Tuesday, May 20, 2008

Poll Results

Does testing team plays an important role in service based company?
YES 8
NO 1
CAN'T SAY 0
Total no of votes = 9

The Ten Principles of Good Software Testing

Testing principle 1: Business risk can be reduced by finding defects.
Testing principle 2: Positive and negative testing contribute to risk reduction.
Testing principle 3: Static and execution testing contribute to risk reduction.
Testing principle 4: Automated test tools can contribute to risk reduction.
Testing principle 5: Make the highest risks the first testing priority.
Testing principle 6: Make the most frequent business activities (the 80/20 rule) the second testing priority.
Testing principle 7: Statistical analyses of defect arrival patterns and other defect characteristics are a very effective way to forecast testing completion.
Testing principle 8: Test the system the way customers will use it.
Testing principle 9: Assume the defects are the result of process and not personality.
Testing principle 10: Testing for defects is an investment as well as a cost.

Saturday, May 10, 2008

Soak Testing

Soak Tests (Also Known as Endurance Testing): Soak testing is running a system at high levels of load for prolonged periods of time. A soak test would normally execute several times more transactions in an entire day (or night) than would be expected in a busy day, to identify any performance problems that appear after a large number of transactions have been executed.
Also, it is possible that a system may ‘stop’ working after a certain number of transactions have been processed due to memory leaks or other defects. Soak tests provide an opportunity to identify such defects, whereas load tests and stress tests may not find such problems due to their relatively short duration. A soak test would run for as long as possible, given the limitations of the testing situation. For example, weekends are often an opportune time for a soak test.

Some typical problems identified during soak tests are listed below:
1.Serious memory leaks that would eventually result in a memory crisis.
2.Failure to close connections between tiers of a multi-tiered system under some circumstances which could stall some or all modules of the system.
3.Failure to close database cursors under some conditions which would eventually result in the entire system stalling.
4.Gradual degradation of response time of some functions as internal data-structures become less efficient during a long test.
Apart from monitoring response time, it is also important to measure CPU usage and available memory. If a server process needs to be available for the application to operate, it is often worthwhile to record it's memory usage at the start and end of a soak test. It is also important to monitor internal memory usages of facilities such as Java Virtual Machines, if applicable.
Long Session Soak Testing: When an application is used for long periods of time each day, the above approach should be modified, because the soak test driver is not Logins and transactions per day, but transactions per active user for each user each day.
This type of situation occurs in internal systems, such as ERP and CRM systems, where users login and stay logged in for many hours, executing a number of business transactions during that time. A soak test for such a system should emulate multiple days of activity in a compacted time-frame rather than just pump multiple days worth of transactions through the system.
Long session soak tests should run with realistic user concurrency, but the focus should be on the number of transactions processed.
Test Duration: The duration of most soak tests is often determined by the available time in the test lab. There are many applications, however, that require extremely long soak tests. Any application that must run, uninterrupted for extended periods of time, may need a soak test to cover all of the activity for a period of time that is agreed to by the stakeholders, such as a month. Most systems have a regular maintenance window, and the time between such windows is usually a key driver for determining the scope of a soak test.

A classic example of a system that requires extensive soak testing is an air traffic control system. A soak test for such a system may have a multi-week or even multi-month duration.

Thursday, May 8, 2008

API Testing

An API (Application Programming Interface) is a collection of software functions and procedures, called API, which can be executed by other software applications.API testing is mostly used for the system which has collection of API that needs to be tested. The system could be system software, application software or libraries.API testing is different from other testing types as GUI is rarely involved in API Testing. Even if GUI is not involved in API testing, we still need to setup initial environment, to invoke API with required set of parameters and then analyze the result.
Setting initial environment become complex because GUI is not involved. It is very easy to setup initial condition in GUI. In case of API this is not the case. This can be divided further in test environment setup and application setup. Things like database should be configured, server should be started are related to test environment setup. On the other hand object should be created before calling non static member of the class falls under application specific setup.Initial condition in API testing also involves creating conditions under which API will be called. API can be called directly or it can be called with the help of some event or in response of some exception. Output of API can be some data or status or it can just wait for some other call to complete in a-synchronized environment. Most of the test cases of API will be based on the output, if API
* Return value based on input condition: - These are relatively simple to test as input can be defined and results can be validated against expected return value. For example, It is very easy to write test cases for int add (int a, int b) kind of API. You can pass different combinations of int a and int b and can validate these against known results.
* Does not return anything: - For cases like these you will probably have some mechanism to check behavior of API on the system. For example, if you need to write test cases for delete (List Element) function you will probably validate size of the list, absence of list element in the list.
* Trigger some other API/event/interrupt: - If API is triggering some event or raising some interrupt, then you need to listen for those events and interrupt listener. Your test suite should call appropriate API and asserts should be on the interrupts and listener.
* Update data structure:-This category is also similar to the API category which does not return anything. Updating data structure will have some effect on the system and that should be validated. If you have other means of accessing the data structure, it should be used to validate that data structure is updated.
* Modify certain resources:-If API call is modifying some resources, for example updating some database, changing registry, killing some process etc, then it should be validated by accessing those resources.
Main Challenges of API Testing can be divided into following categories.
* Parameter Selection
* Parameter combination
* Call sequencing