SOFTWARE IN PRACTICE
(Alias: acceptance testing, system testing, integration testing, unit testing, module testing )
Testing is the process of executing a program with the intent of finding errors.
Software testing is best defined in terms of testing objectives. Any definition should be suitable for inclusion in the introduction to your Test Plan. Testing objectives need to be stated in a form that testers can understand and achieve. Testing objectives also need to speak to the attitudes of the tester. Fundamentally testers need to expect to find errors. Glenford Myers' definition provides one very important objective but there are several others.
A healthy attitude towards testing is an aggregate of the following perspectives listed in increasing order of maturity. Each attitude is translated into a hard objective you can include in your test planning documentation. Note that the most advanced and effective attitude towards testing is summed up at maturity level 4 Testing is a state of mind.
Attitudes to Testing
Case Study: Chernobyl - Catastrophic Failure of a Test Program
The image above is a photograph taken from space by an American space shuttle mission. At the top of the photograph is the Chernobyl nuclear reactor adjacent to it's cooling pond. The Chernobyl nuclear accident (April, 1986) came about because of an ill conceived and poorly planned and executed test on an unstable reactor design. The result was a disaster. An explosion and fire released large quantities of radioactive contamination into the atmosphere, which spread over much of Western USSR and Europe. It is considered the worst nuclear power plant accident in history.
The reactor design. A graphite moderated and water-cooled reactor controlled by 211 control rods. The reactor was operated with a complex computer control system.
The test objective. The test was to determine whether, after steam had been shut off from the turbo generator, the inertia of the still rotating generator would be sufficient to generate enough electricity to operate auxiliary motors that were part of the reactor emergency cooling system. Note that the nuclear power plant did not rely on its own electricity for operation. Offsite electricity was provided. The objective of the experiment was to see how long the locally generated electricity would power the main pumps that keep the cooling water flowing over the fuel.
What went wrong. The test procedure was not prepared or approved in the proper way and was of low quality. In addition:
A bad attitude to testing. Chernobyl 4 was a model plant. It ran very well. Its operators viewed themselves as an elite crew; they had become overconfident. The operators did not think carefully enough about the impacts of their test on the reactor. They were not risk aware. In other words they were running a test but they didn't expect to find an error. This is generally a bad attitude in a tester. The operators had only seconds to react to the emergency - a bad situation in any safety-critical plant.
Plant management blamed the operators. But scientists from other countries agreed that the unstable nature of the reactor design made it difficult to control. You can not make an unsafe design safe by having clever operators.
Lessons learned. The operators at Chernobyl had become complacent. They slipped into the dangerous attitude that an accident could never happen. The testing process was not capable of demonstrating enhanced reactor functionality or improving reactor safety; it only indicated how unsafe it really was. A few simple steps could have avoided a global disaster:
Testing is an important part of any software quality program but it should NOT be expected to meet the following objectives:
1. Testing does not prove the absence of bugs
2. Testing, in isolation, does not improve software quality