ABSTRACT
We investigate how to execute a unit test for all products of a product line without generating each product in isolation in a brute-force fashion. Learning from variability-aware analyses, we (a) design and implement a variability-aware interpreter and, alternatively, (b) reencode variability of the product line to simulate the test cases with a model checker. The interpreter internally reasons about variability, executing paths not affected by variability only once for the whole product line. The model checker achieves similar results by reusing powerful off-the-shelf analyses. We experimented with a prototype implementation for each strategy. We compare both strategies and discuss trade-offs and future directions. In the long run, we aim at finding an efficient testing approach that can be applied to entire product lines with millions of products.
- S. Apel, C. Kästner, A. Größlinger, and C. Lengauer. Type safety for feature-oriented product lines. Automated Software Engineering, 17(3):251--300, 2010. Google ScholarDigital Library
- S. Apel, H. Speidel, P. Wendler, A. von Rhein, and D. Beyer. Detection of feature interactions using feature-aware verification. In Proc. Int'l Conf. Automated Software Engineering (ASE), pages 372--375. IEEE, 2011. Google ScholarDigital Library
- E. Bodden. Position paper: Static flow-sensitive & context-sensitive information-flow analysis for software product lines. In Workshop on Programming Languages and Analysis for Security (PLAS), 2012. Google ScholarDigital Library
- C. Brabrand, M. Ribeiro, T. Tolêdo, and P. Borba. Intraprocedural dataflow analysis for software product lines. In Proc. Int'l Conf. Aspect-Oriented Software Development (AOSD), pages 13--24. ACM, 2012. Google ScholarDigital Library
- I. Cabral, M. B. Cohen, and G. Rothermel. Improving the testing and testability of software product lines. In Proc. Int'l Software Product Line Conference (SPLC), volume 6287 of LNCS, pages 241--255. Springer, 2010. Google ScholarDigital Library
- C. Cadar, P. Godefroid, S. Khurshid, C. S. Păsăreanu, K. Sen, N. Tillmann, and W. Visser. Symbolic execution for software testing in practice: Preliminary assessment. In Proc. Int'l Conf. Software Engineering (ICSE), pages 1066--1071. ACM, 2011. Google ScholarDigital Library
- S. Chen, M. Erwig, and E. Walkingshaw. Extending type inference to variational programs. Technical report (draft), School of EECS, Oregon State University, 2012.Google Scholar
- H. Cichos, S. Oster, M. Lochau, and A. Schürr. Model-based coverage-driven test suite generation for software product lines. In Proc. Int'l Conf. Model Driven Engineering Languages and Systems (MoDELS), volume 6981 of LNCS, pages 425--439. Springer, 2011. Google ScholarDigital Library
- A. Classen, P. Heymans, P.-Y. Schobbens, A. Legay, and J.-F. Raskin. Model checking lots of systems: Efficient verification of temporal properties in software product lines. In Proc. Int'l Conf. Software Engineering (ICSE), pages 335--344. ACM, 2010. Google ScholarDigital Library
- K. Czarnecki and U. Eisenecker. Generative Programming: Methods, Tools, and Applications. ACM Press/Addison-Wesley, New York, 2000. Google ScholarDigital Library
- M. Erwig and E. Walkingshaw. The choice calculus: A representation for software variation. ACM Trans. Softw. Eng. Methodol. (TOSEM), 21(1):Article 6, 2011. Google ScholarDigital Library
- M. Erwig and E. Walkingshaw. Variation programming with the choice calculus. In Proc. Int'l Summer School on Generative and Transformational Techniques in Software Engineering (GTTSE), 2011.Google Scholar
- C. Kästner, S. Apel, T. Thüm, and G. Saake. Type checking annotation-based product lines. ACM Trans. Softw. Eng. Methodol. (TOSEM), 21(3), 2012. Google ScholarDigital Library
- C. Kästner, P. G. Giarrusso, T. Rendel, S. Erdweg, K. Ostermann, and T. Berger. Variability-aware parsing in the presence of lexical macros and conditional compilation. In Proc. Int'l Conf. Object-Oriented Programming, Systems, Languages and Applications (OOPSLA), pages 805--824. ACM, 2011. Google ScholarDigital Library
- C. H. P. Kim, D. S. Batory, and S. Khurshid. Reducing combinatorics in testing product lines. In Proc. Int'l Conf. Aspect-Oriented Software Development (AOSD), pages 57--68. ACM, 2011. Google ScholarDigital Library
- D. R. Kuhn, D. R. Wallace, and A. M. Gallo. Software fault interactions and implications for software testing. IEEE Trans. Softw. Eng. (TSE), 30:418--421, 2004. Google ScholarDigital Library
- K. Lauenroth, K. Pohl, and S. Toehning. Model checking of domain artifacts in product line engineering. In Proc. Int'l Conf. Automated Software Engineering (ASE), pages 269--280. IEEE, 2009. Google ScholarDigital Library
- M. Lochau, I. Schaefer, J. Kamischke, and S. Lity. Incremental model-based testing of delta-oriented software product lines. In Proc. Int'l Conf. Tests and Proofs (TAP), volume 7305 of LNCS, pages 67--82. Springer, 2012. Google ScholarDigital Library
- R. Lopez-Herrejon and D. Batory. A standard problem for evaluating product-line methodologies. In Proc. Int'l Conf. Generative and Component-Based Software Engineering (GCSE), volume 2186 of LNCS, pages 10--24. Springer, 2001. Google ScholarDigital Library
- S. Oster, F. Markert, and P. Ritter. Automated incremental pairwise testing of software product lines. In Proc. Int'l Software Product Line Conference (SPLC), volume 6287 of LNCS, pages 196--210. Springer, 2010. Google ScholarDigital Library
- G. Perrouin, S. Sen, J. Klein, B. Baudry, and Y. le Traon. Automated and scalable t-wise test case generation strategies for software product lines. In Proc. Int'l Conf. Software Testing, Verification, and Validation, pages 459--468. IEEE, 2010. Google ScholarDigital Library
- K. Pohl, G. Böckle, and F. J. van der Linden. Software Product Line Engineering: Foundations, Principles and Techniques. Springer, Berlin/Heidelberg, 2005. Google ScholarDigital Library
- H. Post and C. Sinz. Configuration lifting: Verification meets software configuration. In Proc. Int'l Conf. Automated Software Engineering (ASE), pages 347--350. IEEE, 2008. Google ScholarDigital Library
- J. Shi, M. Cohen, and M. Dwyer. Integration testing of software product lines using compositional symbolic execution. In Proc. Int'l Conf. Fundamental Approaches to Software Engineering, volume 7212 of LNCS, pages 270--284. Springer, 2012. Google ScholarDigital Library
- S. Thaker, D. Batory, D. Kitchin, and W. Cook. Safe composition of product lines. In Proc. Int'l Conf. Generative Programming and Component Engineering (GPCE), pages 95--104. ACM, 2007. Google ScholarDigital Library
- T. Thüm, S. Apel, C. Kästner, M. Kuhlemann, I. Schaefer, and G. Saake. Analysis strategies for software product lines. Technical Report FIN-004-2012, School of Computer Science, University of Magdeburg, 2012.Google Scholar
- T. Thüm, I. Schaefer, S. Apel, and M. Hentschel. Family-based deductive verification of software product lines. In Proc. Int'l Conf. Generative Programming and Component Engineering (GPCE). ACM, 2012. Google ScholarDigital Library
- E. Uzuncaova, D. Garcia, S. Khurshid, and D. Batory. A specification-based approach to testing software product lines. In Proc. Europ. Software Engineering Conf./Foundations of Software Engineering (ESEC/FSE), pages 525--528. ACM, 2007. Google ScholarDigital Library
- W. Visser, K. Havelund, G. P. Brat, S. Park, and F. Lerda. Model checking programs. Autom. Softw. Eng., 10(2):203--232, 2003. Google ScholarDigital Library
- A. von Rhein, S. Apel, and F. Raimondi. Introducing binary decision diagrams in the explicit-state verification of Java code. In Proc. Java Pathfinder Workshop, 2011.Google Scholar
- Toward variability-aware testing
Recommendations
A variability-aware module system
OOPSLA '12: Proceedings of the ACM international conference on Object oriented programming systems languages and applicationsModule systems enable a divide and conquer strategy to software development. To implement compile-time variability in software product lines, modules can be composed in different combinations. However, this way, variability dictates a dominant ...
A variability-aware module system
OOPSLA '12Module systems enable a divide and conquer strategy to software development. To implement compile-time variability in software product lines, modules can be composed in different combinations. However, this way, variability dictates a dominant ...
Investigating Variability-aware Smells in SPLs: An Exploratory Study
SBES '19: Proceedings of the XXXIII Brazilian Symposium on Software EngineeringVariability-aware smell is a concept referring to artifact shortcomings in the context of highly-configurable systems that can degrade aspects such as program comprehension, maintainability, and evolvability. To the best of our knowledge, there is very ...
Comments