Strategy, Planning and Organization of Test Processes

Frontmatter

1. Test Documents According to IEEE 829

Abstract

IEEE 829 defines basic documents for software testing. The test concept describes the results of the test planning. Different types of test documents document the test contents, the test progress and the results of the test execution.

Frank Witte

2. Test Strategy

Abstract

A test strategy is the prerequisite for planned action and the basis for defining the right test processes and determining the organization. A test strategy applies company-wide or to an organization within the company and not just to a single project. Deviations from the test strategy in one’s own project must be justified. The test strategy must be considered in conjunction with the IT strategy. The use of balanced scorecards is recommended as an instrument for controlling the implementation of project strategies.

Frank Witte

3. Test Objectives

Abstract

Test objectives are a subset of the project objectives and are to be measured using appropriate methods. Suitable metrics should be defined to evaluate the test objectives. For this purpose, a project traffic lights should be used as suitable evaluation scales with which test results can be summarized and presented pictorially during reporting.

Frank Witte

4. Test Planning

Abstract

Test planning determines which work packages are to be completed by whom and in what order, plans the test resources and shows dependencies. Professional tools should be used for test planning. Suitable test methods and a realistic test effort estimate, the observance of time and cost restrictions and an evaluation of the test tasks are to be provided by the test manager within the framework of test planning.

Frank Witte

5. Designation of the Test Concept and Introduction

Abstract

For test concepts, whether they are created generically for the entire company or project-related, some general specifications and the correct versioning of the document must be observed. In the introduction to project-related test concepts, the necessity of the corresponding project must be addressed. Relevant project documents must be referenced in the test concept. Agile projects have some special features that must be taken into account already in the concept creation phase. The test concept must be elaborated at the beginning of the test phase.

Frank Witte

6. Test Organization

Abstract

Successful projects require professional organization. In this context, among other things, detailed test planning, the organization of test resources, regular status meetings and coordination during the project must be considered. The test organization forms a subarea of the project organization and must consider the structures of the project and the organizational unit.

Frank Witte

7. Process Description

Abstract

The test process consists of several phases and is to be accompanied by regular defined coordination meetings. The project members in the test team have certain tasks and contribute to the success of the test through active action and targeted communication. Test requirements necessary for the success of the project must be observed and organized.

Frank Witte

8. Test Objects and Test Phases

Abstract

Test cases are linked to test objects. Test objects determine the content of the test activities. Test activities within the individual test levels take place in several phases. Different influencing factors are responsible for determining the test effort.

Frank Witte

9. Test Levels

Abstract

The test process takes place in several test levels. The test concept defines the contents of the individual test levels and delimits them from one another. A good test strategy ensures efficient test execution, prevents duplicate activities in different test levels and ensures the highest possible test coverage. Special test strategies apply to agile projects.

Frank Witte

10. Performance Characteristics to Be Tested

Abstract

The test objects are described in the test concept. In the process, functional tests, load tests and stress tests must be planned and defined in more detail for the performance features to be tested. For testing the user interface, A/B tests and multivariate tests are used for Internet applications.

Frank Witte

11. Features That Are Not Tested

Abstract

The test concept should also set out which criteria are not demonstrated and clearly delineate the test activities so that no unrealistic expectations of the test are created. In this context, the importance of reviews should be emphasized.

Frank Witte

12. Prioritization of Test Cases

Abstract

The reasonable scope of the test activities must be determined during test planning. In the context of prioritizing test cases, test cases that exhibit high risks in the case of faulty functionality and test cases for the verification of frequently occurring business processes are to be preferred.

Frank Witte

13. Permanent Test Organization

Abstract

A permanent test organization takes over defined test services for several organizational units. The goal of the permanent test organization is to accelerate processes while reducing costs and increasing quality. The division of tasks between the permanent test organization and project-related test activities is defined in test guidelines,

Frank Witte

14. Acceptance Criteria

Abstract

Acceptance criteria are measurable performances or properties of the system that are verified during an acceptance test and are relevant for the release of the test object. For the definition of acceptance criteria, as well as for the preparation and execution of acceptance tests, a number of points must be observed.

Frank Witte

15. Criteria for Test Discontinuation and Test Continuation

Abstract

In certain situations, tests must be aborted or interrupted and continued later. There are certain criteria for this. The resumption after test interruption leads to a rescheduling of the test process.

Frank Witte

16. Test Risks

Abstract

Many projects do not run successfully. Project risks must be identified and assessed from the very beginning. There are numerous possible risks to project success that can lead to significant delays and additional costs.

Frank Witte

17. Test Data

Abstract

Test data management regulates the use and administration of test data. The availability of test data has to be ensured, and special questions have to be answered. Different approaches and tools must be used for this purpose.

Frank Witte

18. Test Documentation

Abstract

The test documentation with standard IEEE 829:2008 distinguishes different integrity levels and different types of documents to be prepared within the test process. The test should be integrated into the organization and the test documentation should follow a structure. For test documentation, suitable metrics are to be defined and collected on a regular basis. The availability of the test system during the project duration should also be documented.

Frank Witte

19. Test Items

Abstract

Responsibilities are to be defined for the individual test tasks. Role descriptions are to be defined for individual areas of activity. Standards (e.g., IEC 62034, IEC 62304) are used to describe regulatory tasks and problem-solving processes. An impact analysis of the change requests is to be performed as part of the regression tests. Suitable test tools are to be used for the optimal completion of the test tasks.

Frank Witte

20. Test Environment

Abstract

Suitable test environments must be set up for test execution. Test environment management includes configuration management, availability management, access management, change management, and release management. External test environments and cloud services such as AWS or private clouds can be used for hosting test environments.

Frank Witte

21. Responsibilities, Accountability and Communication

Abstract

Well-functioning cooperation between the individual team members is crucial for successful tests. Communication in projects becomes increasingly complex as the size of the team increases and represents a particular challenge for test management. The precise definition of responsibilities and the use of a communication matrix help to plan the flow of information within a project in a structured manner. The RACI method serves to improve communication within the company.

Frank Witte

22. Personnel, Familiarization, Training

Abstract

The individual stakeholders in the test project must be defined. In this context, the necessary familiarization and training for new employees must also be considered. Good conflict management and relationship management have a significant influence on the success of the project.

Frank Witte

23. Schedule/Work Plan

Abstract

The COCOMO model is suitable for estimating the test duration. Precise data is required for detailed cost estimation and scheduling. For scheduling, the individual test phases and mutual dependencies must be considered. The determined costs and schedules together with the personnel utilization plans must be permanently reviewed during the project duration. Delays on the “critical path” can lead to a delay of the entire project. The COCOMO model can also be used to determine test productivity.

Frank Witte

24. Planning Risks and Unforeseeable Events

Abstract

In the case of postponements, the target amounts of the alternative courses of action must be made clear and possible alternative decision paths must be evaluated. Methods such as investment appraisal and argument balance are suitable for this. SWOT analyses, consequence tables and decision trees, utility and risk analyses are further helpful methods for evaluating and controlling unforeseen events during projects.

Frank Witte

25. Approval and Release

Abstract

A release recommendation is made in the test completion report, followed by approval of the release. The release process with the recommendation of the use of the tested software is to be defined in advance.

Frank Witte

26. Project Organization

Abstract

In practice, different forms of project organization are encountered, which have an effect on the organization of the test project. The matrix organization is the most common organizational form encountered in practice. Different indices can be used to evaluate the progress of the project.

Frank Witte

27. Test Methods

Abstract

Several models, such as TMMI, TPI or TMap, support the methodical approach during the test process. This allows test processes to be optimized in a targeted manner and with the right priorities. Different maturity levels define the progress of the corresponding test organization.

Frank Witte

28. Maturity Level of Test Management According to TPI Next

Abstract

TPI Next evaluates the maturity level of test processes and provides suitable best practices to sustainably improve a test organization. For this purpose, the test process is divided into different core areas. Certain criteria must be met for each of the maturity levels.

Frank Witte

29. Special Features of the Test Organization in Agile Projects

Abstract

Test management in agile projects must consider the stronger dynamics of the test processes. Organizational forms, roles, and methods necessary for agilization also affect software testing. Special principles must therefore be observed for agile testing.

Frank Witte

30. Artificial Intelligence and Cognitive Testing

Abstract

New trends, such as artificial intelligence and the use of cognitive technologies, will increasingly influence software testing and lead to new test procedures and changed processes.

Frank Witte

Backmatter