Xavier Devroey
Pierre-Yves Schobbens
ABSTRACT
Software Product Lines (SPLs) are inherently difficult to test due to the combinatorial explosion of the number of products to consider. To reduce the number of products to test, sampling techniques such as combinatorial interaction testing have been proposed. They usually start from a feature model and apply a coverage criterion (e.g. pairwise feature interaction or dissimilarity) to generate tractable, fault-finding, lists of configurations to be tested. Prioritization can also be used to sort/generate such lists, optimizing coverage criteria or weights assigned to features. However, current sampling/prioritization techniques barely take product behaviour into account. We explore how ideas of statistical testing, based on a usage model (a Markov chain), can be used to extract configurations of interest according to the likelihood of their executions. These executions are gathered in featured transition systems, compact representation of SPL behaviour. We discuss possible scenarios and give a prioritization procedure validated on a web-based learning management software.
- P. Asirelli, M. H. ter Beek, A. Fantechi, S. Gnesi, and F. Mazzanti. Design and validation of variability in product lines. In PLEASE '11, pages 25--30. ACM, 2011. Google Scholar
Digital Library
- P. Asirelli, M. H. ter Beek, S. Gnesi, and A. Fantechi. Formal description of variability in product families. In SPLC '11, pages 130--139. IEEE, 2011. Google ScholarDigital Library
- Claroline. http://www.claroline.net/.Google Scholar
- A. Classen, M. Cordy, P.-Y. Schobbens, P. Heymans, A. Legay, and J.-F. Raskin. Featured Transition Systems: Foundations for Verifying Variability-Intensive Systems and their Application to LTL Model Checking. TSE, PP(99):1--22, 2013. Google ScholarDigital Library
- A. Classen, P. Heymans, P. Schobbens, and A. Legay. Symbolic model checking of software product lines. In ICSE '11, 2011. Google ScholarDigital Library
- A. Classen, P. Heymans, P. Schobbens, A. Legay, and J. Raskin. Model checking lots of systems: efficient verification of temporal properties in software product lines. In ICSE '10, pages 335--344. ACM, 2010. Google ScholarDigital Library
- M. Cohen, M. Dwyer, and J. Shi. Interaction testing of highly-configurable systems in the presence of constraints. In ISSTA 07, pages 129--139, 2007. Google ScholarDigital Library
- M. B. Cohen, M. B. Dwyer, and J. Shi. Coverage and adequacy in software product line testing. In ROSATEA '06, pages 53--63, 2006. Google ScholarDigital Library
- M. Cordy, P. Heymans, P.-Y. Schobbens, A. M. Sharifloo, C. Ghezzi, and A. Legay. Verification for reliable product lines. arXiv:1311.1343, 2013.Google Scholar
- K. Czarnecki and A. Wasowski. Feature Diagrams and Logics: There and Back Again. In SPLC '07, pages 23--34. IEEE, Sept. 2007. Google ScholarDigital Library
- X. Devroey, M. Cordy, G. Perrouin, E.-Y. Kang, P.-Y. Schobbens, P. Heymans, A. Legay, and B. Baudry. A Vision for Behavioural Model-Driven Validation of Software Product Lines. ISoLA '12, pages 208--222. Springer-Verlag, 2012. Google ScholarDigital Library
- A. Feliachi and H. Le Guen. Generating transition probabilities for automatic model-based test generation. In ICST '10, pages 99--102. IEEE, 2010. Google ScholarDigital Library
- D. Fischbein, S. Uchitel, and V. Braberman. A foundation for behavioural conformance in software product line architectures. In ROSATEA '06, pages 39--48. ACM, 2006. Google ScholarDigital Library
- S.-D. Gouraud, A. Denise, M.-C. Gaudel, and B. Marre. A new way of automating statistical testing methods. In ASE '01. IEEE Computer Society, 2001. Google ScholarDigital Library
- C. Henard, M. Papadakis, G. Perrouin, J. Klein, and Y. L. Traon. Multi-objective test generation for software product lines. SPLC '13, pages 62--71. ACM, 2013. Google ScholarDigital Library
- M. F. Johansen, Ø. Haugen, F. Fleurey, A. G. Eldegard, and T. Syversen. Generating better partial covering arrays by modeling weights on sub-product lines. In MoDELS '12, pages 269--284, 2012. Google ScholarDigital Library
- K. C. Kang, S. G. Cohen, J. A. Hess, W. E. Novak, and A. Spencer Peterson. Feature-Oriented domain analysis (FODA) feasibility study. Technical report, Soft. Eng. Inst., Carnegie Mellon Univ., 1990.Google Scholar
- C. H. P. Kim, S. Khurshid, and D. S. Batory. Shared execution for efficiently testing product lines. In ISSRE '12, pages 221--230, 2012.Google ScholarDigital Library
- K. Lauenroth, K. Pohl, and S. Toehning. Model checking of domain artifacts in product line engineering. In ASE '09, pages 269--280. IEEE, 2009. Google ScholarDigital Library
- M. Lochau, S. Oster, U. Goltz, and A. Schürr. Model-based pairwise testing for feature interaction coverage in software product line engineering. Software Quality Journal, 20(3-4):567--604, 2012. Google ScholarDigital Library
- A. P. Mathur. Foundations of software testing. Pearson Education, 2008. Google ScholarDigital Library
- R. Michel, A. Classen, A. Hubaux, and Q. Boucher. A formal semantics for feature cardinalities in feature diagrams. VaMoS '11, pages 82--89. ACM, 2011. Google ScholarDigital Library
- J. D. Musa, G. Fuoco, N. Irving, D. Kropfl, and B. Juhlin. The operational profile. NATO ASI series F Comp. and Syst. Sc., 154:333--344, 1996.Google Scholar
- S. Oster, A. Wöbbeke, G. Engels, and A. Schürr. Model-based software product lines testing survey. In Model-Based Testing for Embedded Systems, pages 339--382. CRC Press, 2011.Google ScholarCross Ref
- G. Perrouin, S. Oster, S. Sen, J. Klein, B. Baudry, and Y. L. Traon. Pairwise testing for software product lines: comparison of two approaches. Software Quality Journal, 20(3-4):605--643, 2012. Google ScholarDigital Library
- H. Samih and B. Baudry. Relating variability modelling and model-based testing for software product lines testing. In ICTSS '12 DS, 2012.Google Scholar
- S. Sampath, R. Bryce, G. Viswanath, V. Kandimalla, and A. Koru. Prioritizing user-session-based test cases for web applications testing. In ICST '08, pages 141--150, 2008. Google ScholarDigital Library
- R. R. Sarukkai. Link prediction and path analysis using markov chains. Computer Networks, 33(1-6):377--386, 2000. Google ScholarDigital Library
- Scrapy. http://scrapy.org/.Google Scholar
- P. Thévenod-Fosse and H. Waeselynck. An investigation of statistical software testing. Softw. Test., Verif. Reliab., 1(2):5--25, 1991.Google ScholarDigital Library
- J. Tretmans. Model based testing with labelled transition systems. In Formal methods and testing, pages 1--38. Springer-Verlag, 2008. Google ScholarCross Ref
- M. Utting and B. Legeard. Practical model-based testing: a tools approach. Morgan Kaufmann, 2007. Google ScholarDigital Library
- S. Verwer, R. Eyraud, and C. Higuera. Pautomac: a probabilistic automata and hidden markov models learning competition. Machine Learning, pages 1--26, 2013.Google Scholar
- A. von Rhein, S. Apel, C. Kästner, T. Thüm, and I. Schaefer. The PLA model: on the combination of product-line analyses. In VaMoS '13. ACM, 2013. Google ScholarDigital Library
- J. A. Whittaker and M. G. Thomason. A markov chain model for statistical software testing. IEEE TSE, 20(10):812--824, 1994. Google ScholarDigital Library
Index Terms
- Towards statistical prioritization for software product lines testing
Recommendations
Similarity-based prioritization in software product-line testing
SPLC '14: Proceedings of the 18th International Software Product Line Conference - Volume 1Exhaustively testing every product of a software product line (SPL) is a difficult task due to the combinatorial explosion of the number of products. Combinatorial interaction testing is a technique to reduce the number of products under test. However, ...
Delta-oriented test case prioritization for integration testing of software product lines
SPLC '15: Proceedings of the 19th International Conference on Software Product LineSoftware product lines have potential to allow for mass customization of products. Unfortunately, the resulting, vast amount of possible product variants with commonalities and differences leads to new challenges in software testing. Ideally, every ...
Practical pairwise testing for software product lines
SPLC '13: Proceedings of the 17th International Software Product Line ConferenceOne key challenge for software product lines is efficiently managing variability throughout their lifecycle. In this paper, we address the problem of variability in software product lines testing. We (1) identify a set of issues that must be addressed ...
Comments