Renée C. Bryce
ABSTRACT
Event-driven software (EDS) is a widely used class of software that takes sequences of events as input, changes state, and outputs new event sequences. Managing the size of tests suites for EDS is difficult as the number of event combinations and sequences grow exponentially with the number of events. We propose a new testing technique that extends software interaction testing. Traditional software interaction testing systematically examines all t-way interactions of parameters for a program. This paper extends the notion to t-way interactions over sequences of events. The technique applies to many classes of software; we focus on that of EDS. As a proof-of-concept, we prioritize existing test suites for four GUI-based programs by t-way interaction coverage. We compare the rate of fault detection with that of several other prioritization criteria. Results show that prioritization by interaction coverage has the fastest rate of fault detection in half of our experiments, making the most impact when tests have high interaction coverage.
- R. C. Bryce and C. J. Colbourn. The density algorithm for pairwise interaction testing. Journal of Software Testing, Verification, and Reliability, to appear. Google Scholar
Digital Library
- R. C. Bryce, A. Rajan, and M. P. E. Heimdahl. Interaction testing in model-based development: Effect on model-coverage. Proc. of the 13th Asia-Pacific Software Engineering Conf., pages 258--269, Dec. 2006. Google ScholarDigital Library
- C. J. Colbourn. Combinatorial aspects of covering arrays. Le Matematiche (Catania), 58:121--167, 2004.Google Scholar
- S. Elbaum, A. Malishevsky, and G. Rothermel. Test case prioritization: A family of empirical studies. IEEE Trans. on Software Engineering, 18(2):159--182, 2002. Google ScholarDigital Library
- S. Elbaum, G. Rothermel, S. Kanduri, and A. Malishevsky. Selecting a cost-effective test case prioritization technique. Software Quality Journal, 12(3):185--210, 2004. Google ScholarDigital Library
- D. R. Kuhn, D. R. Wallace, and A. M. Gallo. Software fault interactions and implications for software testing. IEEE Trans. on Software Engineering, 30(6):418--421, Oct. 2004. Google ScholarDigital Library
- Atif M. Memon. An event-flow model of gui-based applications for testing. Software Testing, Verification and Reliability, 2007. Google ScholarDigital Library
- Atif M. Memon, Mary Lou Soffa, and Martha E. Pollack. Coverage criteria for GUI testing. In ESEC/FSE-9: Proc. of the 8th European software engineering conf. held jointly with 9th ACM SIGSOFT Int. symposium on Foundations of software engineering, pages 256--267, 2001. Google ScholarDigital Library
- G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Prioritizing test cases for regression testing. ACM Trans. on Software Engineering and Methodology, 27(10):929--948, 2001. Google ScholarDigital Library
- A. Srivastava and J. Thiagarajan. Effectively prioritizing tests in development environment. In Proc. of the Int. Symposium on Software Testing and Analysis, pages 97--106, Jul. 2002. Google ScholarDigital Library
- Kristen R. Walcott, Mary Lou Soffa, Gregory M. Kapfhammer, and Robert S. Roos. Timeaware test suite prioritization. In Proc. of the Int. Symposium on Software Testing and Analysis, pages 1--12, Jul. 2006. Google ScholarDigital Library
- L. White. Regression testing of gui event interactions. In Proc. of the Int. Conf. on Software Maintenance, pages 350--358, Nov. 1996. Google ScholarDigital Library
- L. White and H. Almezen. Generating test cases for gui responsibilities using complete interaction sequences. In Proc. of the Interactional Symposium on Software Reliability Engineering, pages 110--121, 2000. Google ScholarDigital Library
- Qing Xie and Atif M. Memon. Studying the characteristics of a 'good' GUI test suite. In Proc. of the 17th IEEE Int. Symposium on Software Reliability Engineering. IEEE Computer Society Press, 2006. Google ScholarDigital Library
- C. Yilmaz, M. B. Cohen, and A. Porter. Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Trans. on Software Engineering, 31(1):20--34, Jan. 2006. Google ScholarDigital Library
- Xun Yuan and Atif M. Memon. Using GUI run-time state as feedback to generate test cases. In Proc. of the 29th Int. Conf. on Software Engineering, May 2007. Google ScholarDigital Library
Recommendations
Developing a Single Model and Test Prioritization Strategies for Event-Driven Software
Event-Driven Software (EDS) can change state based on incoming events; common examples are GUI and Web applications. These EDSs pose a challenge to testing because there are a large number of possible event sequences that users can invoke through a user ...
Aggregate-strength interaction test suite prioritization
HighlightsWe propose a new category of prioritization methods of interaction test suites.We propose a new dissimilarity measure for combinatorial test cases.Three versions of our method are evaluated using simulations and empirical studies.Our methods ...
Regression test suite prioritization using system models
During regression testing, a modified system is often retested using an existing test suite. Since the size of the test suite may be very large, testers are interested in detecting faults in the modified system as early as possible during this retesting ...
Comments