Successful Test Management

Frontmatter

1. Testing and Test Management

Abstract

Automated teller machines that issue the wrong banknotes, Space Shuttles that crash on landing, telephone networks that go down and websites that are vulnerable to hackers are all familiar examples of ICT systems that fail, and there are new examples almost every day.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

2. Risk & Requirement Based Testing

Abstract

Risk management plays an important role in management practice. In test projects, two types of risk are involved: project risks and product risks. Test project risks are related to problems that may endanger the test activities in a project. At the same time, insight into the product risks is important if decisions have to be made on whether to put an information system into production. In consultation with the parties directly involved, the stakeholders, the test manager surveys these risks and their impact.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

3. The Quick Scan

Abstract

It is advisable for the test manager to first carry out a brief feasibility study of the test project. In this “quick scan” phase, the test manager establishes the status of all the important aspects of the test. To do this he employs various components of the Test Management Model. Among other things there are the presence and usability of standards, organisational and budgetary instruments, management involvement and availability of resources. This inventory process should not be too lengthy. Its duration depends on the scale and complexity of the project, but should be no longer than 5 days. The test manager can use the information to start the necessary preparations at an early stage. The results of the study are discussed with the client and other stakeholders and are used to finalise the assignment of the test manager. The quick scan is only used to make an inventory and is not part of the test project preparation.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

4. The Test Management File

Abstract

In order to maintain a good overview of the test project, the test manager must establish and manage all information relating to it. This means not only the documents created by the test manager himself but also, for example, the testware developed by the testers and all other information of importance to the test project. Having a clear structure helps the test manager to provide information to the stakeholders in a clear and efficient manner giving them a better insight into the project. The test management file discussed here equips the test manager with a model for distributing information.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

5. Risk Analysis and Test Strategy

Abstract

“It isn’t possible to test everything!” This is something that must be under­stood by all the stakeholders. But how do you decide what has to be tested? A good test strategy is necessary because, otherwise, time and effort will not be focused in the best possible way and it will be hard to communicate the status of the covered product risks. In the test strategy, the test manager indicates what is to be tested, but also what is not to be tested. This is decided on the basis of the product risks and the priority assigned to them, assessed by the product risk analysis. To implement the test strategy, allowance has to be made within the test project for the project risks: which elements might obstruct the progress of the test project?

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

6. Estimation

Abstract

It is not easy to draw up an estimate at the very beginning of a test project, since only general information is available at that time. There is a choice of methods for estimating, suited to a variety of situations. A number of these methods are discussed in this chapter. The estimate can be refined by keeping metrics during test projects. This provides figures about the history that can be used in subsequent test projects. We will describe how these can be transferred into a model and produce more accurate estimates.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

7. Planning

Abstract

An estimate has been drawn up: the number of hours allocated is linked to the activities that need to be performed. These activities must now be set out in terms of time. It must also be indicated which interdependencies exist between the activities, who is going to carry out the activities and which results are expected from them. Activities are given a start date and a completion date. In the course of the test project, the plan will become increasingly detailed.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

8. The Test Organisation

Abstract

The scope and depth of the test are set out in the test strategy. The test manager has based the planning and estimating on that information. The cluster matrix sets out which test levels will be carried out. A system test will largely be performed within the IT project, while an acceptance test will be carried out by the business side (a users’ organisation or a central test department). The test organisation should be described in the test plans. Sometimes a reference to documentation already present in the organisation is sufficient.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

9. The Dynamics of Testing

Abstract

Following the preparation, it is time to go on to design the testware and eventually to perform tests on the information system. The test strategy and the test plan have been developed and signed off by the stakeholders. Nevertheless, in practice, the analysis and test execution do not always proceed according to the predictions made in the test strategy and test plan and based on the information that was available at the time. The project and its context are always in motion. Within the dynamics of the test execution, the test manager’s attention shifts away from creating conditions to controlling the situation. Everything must be geared towards achieving a successful result, for testing is not something you do “on the side”. It is a specific profession. The management required and the potential problems demand specific knowledge. During this phase, the test manager has the opportunity to demonstrate his added value.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

10. Progress Management

Abstract

The preparations for the test project are completed. The budget is available, the project and product risks have been surveyed and the test organisation is ready. The stakeholders have approved the test strategy, and the plan describes the activities together with their relevant resources and time required. Everyone knows what has to be done. The execution of the test project can begin, starting with the test preparation. This chapter shows how the test manager monitors the test project and how he is able to provide timely information and advice to the client concerning bottlenecks and possible consequences of risks that materialise.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

11. Issue Management

Abstract

In 1947, a moth got into the hardware of the MARK II “Automatic Sequence Controlled Calculator” and was the cause of various failures. Mrs Grace Hopper discovered the “bug” and logged it in her journal — the first “bug” and bug registration. Nowadays, the registration of bugs (also called defects, or issues) is an important part of test execution, not only because it allows the test team to demonstrate whether the information system is up to the required quality, but also because it enables the developers to reproduce and solve failures.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

12. Reporting and Advice

Abstract

“How’s testing going? Is the information system ready to go into production?” As soon as the system is delivered to the test project, all the parties involved are interested in the first test results. The test manager will provide regular insight into the progress, the quality of the information system and any problems surrounding the test project. Through regular reporting, the test project holds the attention of the client and stakeholders. They know then which product risks and related requirements are covered and which risks still have to be tested. With this information they can make a decision on whether to go on to a subsequent test level or to put the information system into production.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

13. Evaluation and Transfer

Abstract

When the information system has successfully gone into production and the test project has been discharged, the test manager disbands the team, takes his leave of the project and moves on to the next one. His responsibilities seem to end here. But is that right? If he never looks back at what went well and what did not during the test project, and why, there will be no improvement in the following project. Only by making an evaluation can he, as well as the organisation, learn from and improve on the test process. And if he fails to organise the maintenance of the testware for the information system, he does himself or a colleague in a subsequent project a grave disservice. In short, the test manager must never forget evaluation of the test process and transfer of the testware, even though the information system is operating successfully in the production environment.

Iris Pinkster, Bob van de Burgt, Dennis Janssen, Erik van Veenendaal

Backmatter