Testing a Software Product Line

The software product line approach to the development of software intensive systems has been used by organizations to improve quality, increase productivity and reduce cycle time. These gains require different approaches to a number of the practices in the development organization including testing. The planned variability that facilitates some of the benefits of the product line approach poses a challenge for test-related activities. This chapter provides a comprehensive view of testing at various points in the software development process and describes specific techniques for carrying out the various test-related tasks. These techniques are illustrated using a pedagogical product line developed by the Software Engineering Institute (SEI).

We first introduce the main challenges approached by this chapter, then Section 2 overviews basic software product lines concepts. Section 3 explains, in a product line context, several testing concepts introduced in Chapter 1. Section 4 complements this by introducing Guided Inspections, a technique that applies the discipline of testing to the review of non-software assets typically found in software product lines. The core of the chapter is Section 5, which describes techniques that can be used to test product lines. Section 6 discusses how we can evaluate the benefits of a product line approach for testing, and Section 7 illustrates some of the presented techniques. Finally, we end with a discussion of related issues and research questions (see Section 8), and conclusions (see Section 9) about product line testing and its relation to development.