Shin Yoo
Mark Harman
ABSTRACT
Previous work has treated test case selection as a single objective optimisation problem. This paper introduces the concept of Pareto efficiency to test case selection. The Pareto efficient approach takes multiple objectives such as code coverage, past fault-detection history and execution cost, and constructs a group of non-dominating, equivalently optimal test case subsets. The paper describes the potential bene?ts of Pareto efficient multi-objective test case selection, illustrating with empirical studies of two and three objective formulations.
- S. Bates and S. Horwitz. Incremental program testing using program dependence graphs. In Conference Record of the Twentieth ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages pages 384--396, Charleston, South Carolina, Jan. 10-13, 1993. ACM Press. Google Scholar
Digital Library
- D. Binkley. Reducing the cost of regression testing by semantics guided test case selection. In ICSM '95: Proceedings of the International Conference on Software Maintenance page 251, Washington, DC, USA, 1995. IEEE Computer Society. Google ScholarDigital Library
- C. A. Coello Coello, D. A. Van Veldhuizen, and G. B. Lamont. Evolutionary Algorithms for Solving Multi-Objective Problems KluwerAcademic Publishers, New York, May 2002. Google ScholarDigital Library
- Y. Collette and P. Siarry. Multiobjective Optimization: Principles and Case Studies Springer, 2004.Google Scholar
- K. Deb. Multi-Objective Optimization Using Evolutionary Algorithms Wiley, Chichester, UK, 2001. Google ScholarDigital Library
- K. Deb, S. Agrawal, A. Pratab, and T. Meyarivan. A Fast Elitist Non-Dominated Sorting Genetic Algorithm for Multi-Objective Optimization: NSGA-II. In Proceedings of the Parallel Problem Solving from Nature VI Conference pages 849--858, Paris, France, 2000. Springer. Lecture Notes in Computer Science No. 1917. Google ScholarDigital Library
- H. Do, S. G. Elbaum, and G. Rothermel. Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering: An International Journal 10(4):405--435, 2005. Google ScholarDigital Library
- S. G. Elbaum, A. G. Malishevsky, and G. Rothermel. Prioritizing test cases for regression testing. In International Symposium on Software Testing and Analysis pages 102--112. ACM Press, 2000. Google ScholarDigital Library
- M. J. Harrold, R. Gupta, and M. L. Soffa. A methodology for controlling the size of a test suite. ACM Trans. Softw. Eng. Methodol. 2(3):270--285, 1993. Google ScholarDigital Library
- L. W. H. K. N. Leung. Insight into regressin testing. In Proceedings of the Conference on Software Maintenance October 1989.Google Scholar
- M. Hutchins, H. Foster, T. Goradia, and T. Ostrand. Experiments on the effectiveness of data ?ow-and control-?ow-based test adequacy criteria. In Proceedings of the 16th International Conference on Software Engineering pages 191--200. IEEE Computer Society Press, May 1994. Google ScholarDigital Library
- Z. Li, M. Harman, and R. Hierons. Meta-heuristic search algorithms for regression test case prioritization. IEEE Transactions on Software Engineering To Appear. Google ScholarDigital Library
- A. G. Malishevsky, J. R. Ruthruff, G. Rothermel, and S. Elbaum. Cost-cognizant test case prioritization. Technical report, Department of Computer Science and Engineering, University of Nebraska-Lincoln, March 2006.Google Scholar
- N. Nethercote and J. Seward. Valgrind: A program supervision framework. In Proceedings of the Third Workshop on Runtime Verification Colorado, USA, July 2003. Boulder.Google Scholar
- G. Rothermel and M. J. Harrold. Analyzing regression test selection techniques. IEEE Transactions on Software Engineering 22(8):529--551, Aug. 1996. Google ScholarDigital Library
- G. Rothermel, M. J. Harrold, J. Ostrin, and C. Hong. An empirical study of the effects of minimization on the fault detection capabilities of test suites. In ICSM pages 34--43, 1998. Google ScholarDigital Library
- G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold. Test case prioritization: An empirical study. In Proceedings; IEEE International Conference on Software Maintenance pages 179--188, Los Alamitos, California, USA, 1999. IEEE Computer Society Press. Google ScholarDigital Library
- F. Szidarovsky, M. E. Gershon, and L. Dukstein. Techniques for multiobjective decision making in systems management Elsevier, New York, 1986.Google Scholar
- P. Tonella, P. Avesani, and A. Susi. Using the case-based ranking methodology for test case prioritization. In ICSM '06: Proceedings of the 22nd IEEE International Conference on Software Maintenance pages 123--133, Washington, DC, USA, 2006. IEEE Computer Society. Google ScholarDigital Library
- K. R. Walcott, M. L. Soffa, G. M. Kapfhammer, and R. S. Roos. Time aware test suite prioritization. In ISSTA '06: Proceedings of the 2006 international symposium on Software testing and analysis pages 1--12, New York, NY, USA, 2006. ACM Press. Google ScholarDigital Library
- W. E. Wong, J. R. Horgan, S. London, and A. P. Mathur. Effect of test set minimization on fault detection effectiveness. Software - Practice and Experience 28(4):347--369, 1998. Google ScholarDigital Library
- S. S. Yau and Z. Kishimoto. A method for revalidating modi?ed programs in the maintenance phase. In Proceedings of 11th International Computer Software and Applications Conference (COMPSAC '87)pages pp. 272--277, October 1987.Google Scholar
Index Terms
- Pareto efficient multi-objective test case selection
Recommendations
Empirical evaluation of pareto efficient multi-objective regression test case prioritisation
ISSTA 2015: Proceedings of the 2015 International Symposium on Software Testing and AnalysisThe aim of test case prioritisation is to determine an ordering of test cases that maximises the likelihood of early fault revelation. Previous prioritisation techniques have tended to be single objective, for which the additional greedy algorithm is ...
Runtime analysis of a multi-objective evolutionary algorithm for obtaining finite approximations of Pareto fronts
Previous theoretical analyses of evolutionary multi-objective optimization (EMO) mostly focus on obtaining @?-approximations of Pareto fronts. However, in practical applications, an appropriate value of @? is critical but sometimes, for a multi-...
An effective approach for regression test case selection using pareto based multi-objective harmony search
SBST '18: Proceedings of the 11th International Workshop on Search-Based Software TestingRegression testing is a way of catching bugs in new builds and releases to avoid the product risks. Corrective, progressive, retest all and selective regression testing are strategies to perform regression testing. Retesting all existing test cases is ...
Comments