Improving Test Models for Large Scale Industrial Systems: An Inquisitive Study

Although documentation of software tests is becoming increasingly important, there is little knowledge on whether modeling languages and tools are effective in industrial projects. Recent reports have pointed out that test modeling techniques might be barely used by software developers due to their inability to cover test concepts relevant in real-life large applications. This paper reports an inquisitive multi-phase study aimed at revealing test-relevant concepts not supported by modeling languages. The study encompassed several questionnaire responses and interviews with developers, and observational analyses run over two years in large-scale software projects. Various test concepts were brought forth and they fall in three categories: (i) test cases and software evolution, (ii) interdependencies between test cases, and (iii) categorization and grouping of test cases. Finally, the relevance of the identified test concepts is discussed in terms of an industrial system for inventory and supply control of petroleum products.