It is impossible to test a program completely.
What does it “testing a program completely” mean? Ideally, it means that at the end of the test process, the program will have been tested against all possible eventualities, and that there should not remain any errors in program functionality. All existing problems would have been resolved during the testing process.
In reality, this cannot happen. There are simply too many variables. For example:
• It is not possible to verify the reaction of any program to every combination of input information.
• It is not possible to verify every possible sequence of program workflow.
• It is not possible to reveal all design errors.
• The correctness of a program cannot be always be proved logically.
Then why should programs be tested?
Since it can never be said that a program works perfectly, why should it be tested?
A program should be tested to find the errors in it that can be addressed and solved, increasing efficiency in program functionality and confidence in program usage results.
Small or large, all program errors cost you money and time. Our job is to search for and eliminate errors for you.
Testing improves the quality and performance of any program.
When the majority of errors in a program are found and corrected, the quality of program output is improved, and so is your bottom line. This is the real purpose of testing.
We can participate in testing of program product in any development stage of the project:
Design
• preparation for the testing automation
• development of reception tests
• stability of acquisitions analysis
• initial testing plan development
Realization of base functions
• beginning of nonformal testing
• beginning of the formal core product testing
• first nonformal estimations of tasks, resources, time and budget.
Almost alpha
• determination of testing purposes and tasks, time, resources and budget necessary; creation of prototype testing plan
• risk evaluation for the tested project
• fulfillment of basic testing.
Alpha
• testing all program blocks
• testing under real operating conditions
• nonformal testing of the specific program blocks
• planning and the fulfillment of the detailed tests of the selected program blocks
• test plan revision
• analysis of testing manual and testing according to it
• discussion of specification shortcomings
• estimate amount of remaining errors
• beginning of testing for the hardware compatability
• addition of regression tests.
• the beginning of the automation of testing.
Pre-beta
• testing program for compliance to the requirements for the stability and the completeness of Beta-version.
Beta
• final test plan approval
• the continuation of fulfillment and the deepening of test plan and automation of testing
• rapid repeated testing of the corrected program blocks
• complete cycle of hardware testing
• publication of the formal testing results
• the last analysis of user interface, its preparation for the freezing
• Beta- testing out of the company.
Freezing user interface
• regression testing
• test plan fulfillment
• extended hardware testing
Preparation for the final testing
• regression testing with all possible versions of program environment
• complete cycle of tests according to the plan for the last version of program
• hardware testing
• testing the corrections of old errors
• system reliability evaluation
Last test for integrity
• reliability evaluation during the first day of operation
• real mode testing
• test plan and errors analysis
• testing the first releases
Release
• continuous testing during entire production period
• testing finished product
Examples of the tests, which we can make during the functional and system testing:
Collation with the specification
The correspondence of the developed program to each word of specification is checked.
Correctness
Testing how correct program makes necessary calculations and produces reports.
Laboratory tests
Hiring several feature users and watching their work with product. Actually Beta-testing is done in an attempt to obtain the same result, but with Beta-testing it is not possible to watch the process, Beta-testing it is much less effective than laboratory tests.
Extreme values
Testing program reaction to the extreme values input.
Productivity
Measuring time elapsed for different tasks, especially those, which clients will use most frequently.
Switching the modes
Testing how correctly program is switching from one mode to another. It is especially important for multitasking systems.
Real mode operation
We work with the program in the same regime as real users would work. The shortcomings, which were missed during the formal testing or they were considered insignificant, in the real work can prove to be very serious.
Load tests
Testing program reaction to extreme operating conditions:
• testing to the maximum volume of the input information
• testing program reaction to an increased activity
• analysis of resources requirements
Multi-user and multitask work
It is checked how product works with the parallel accomplishment of several objectives and how actions of several users are coordinated.
Working/treatment of errors
Testing program reaction to the improper, nonstandard or not predicted actions of users.
Protection
It is checked to see how complicated is it for an unauthorized user to obtain access to the system.
Compatability and format conversion
Testing ability of two products to work with the same data files or ability of successful co-existence in the computer operating memory.
Hardware configurations
Testing program operation on the computers with diverse configurations.
Installation and maintenance
Testing program installation, how simple and convenient and how long does it take on the average to complete the installation.