Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Software Quality Journal
Published in: Software Quality Journal 3/2017

14-07-2016

State-based models in regression test suite prioritization

Authors: Luay Tahat, Bogdan Korel, George Koutsogiannakis, Nada Almasri

Published in: Software Quality Journal | Issue 3/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Testing software products is very expensive and time consuming, especially for large software systems with extensive regression testing. During regression testing, a modified system is often re-tested using an existing test suite. Since test suites can be very large, testers are interested in detecting faults in the modified system as early as possible. Test prioritization tries to order test cases for execution in a way that increases the chances of the early detection of faults. Most of the existing test prioritization methods are based on the code of the system under test, but model-based test prioritization has been lately proposed. Most of the existing model-based test prioritization methods can be used only when models are modified during system maintenance. In this paper, we present model-based prioritization for a class of modifications for which models are not modified (only the source code is modified). After identifying the elements of the model related to the modified source code, information collected during the execution of the model is used to prioritize tests for execution. Here, we present and compare existing and new model-based test prioritization methods focused on this class of modifications. The major motivation for presenting these methods is to provide system developers with simple and yet effective test prioritization techniques for early fault detection. Statistical analysis of the empirical study, which compares the effectiveness of the presented methods in terms of early fault detection, show that compared to random ordering of test cases, model-based test prioritization significantly improve the effectiveness of test prioritization with respect to early fault detection.
previous article A model-based framework encompassing a complete workflow from specification until validation of timing requirements in embedded software systems
next article Quality characteristics and measures for human–computer interaction evaluation in ubiquitous systems

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now
Literature
Androutsopoulos, K., Clark, D., Harman, M., Li, Z., & Tratt, L. (2009a). Control dependence for extended finite state machines. In Proceedings of the Fundamental Approaches to Software Engineering (FASE ‘09) (Vol. 5503, pp. 216–230). LNCS. York: Springer. Accessed 22nd–29th March 2009.
Androutsopoulos, K., Gold, N., Harman, M., Li, Z., & Tratt, L. (2009b). A theoretical and empirical study of EFSM dependence. In Proceedings 25th IEEE international conference on software maintenance (ICSM 2009). Edmonton, AB. 23rd–26th Sept 2009. To appear.
Basanieri, F., Bertolino, A., & Marchetti, E. (2002). The cow suite approach to planning and deriving test suites in UML projects. International conference on the unified modeling language: The language and its applications (Lecture notes in computer science) (pp. 383–397). Berlin: Springer.
Belina, F., & Hogrefe, D. (1989). the CCITT-specification and description language SDL. Computer Networks, 16(3), 11–341.
Briand, L. C., Labiche, Y., & He, S. (2009). Automating regression test selection based on UML designs. Information and Software Technology, 51(1), 16–30.CrossRef
Briand, L. C., Labiche, Y., & Soccar, G. (2002). Automating impact analysis and regression test selection based on UML designs. In International conference on software maintenance (ICSM) (pp. 252–261).
Catal, C., & Mishra, D. (2013). Test case prioritization: A systematic mapping study. Software Quality Journal, 21, 445–478.CrossRef
Chen, Y., Probert, R. L., & Sims, D. P. (2002). Specification based regression test selection with risk analysis. In Proceedings of conference of the center for advance studies on collaborative research.
Chen, Y., Probert, R., & Ural, H. (2007). Model-based regression test suite generation using dependence analysis. In Proceedings of the 3rd ACM workshop on advances in model based testing (A-MOST) (pp. 62–67).
Chen, Y.-P., Probert, R., & Ural, H. (2009). Regression test suite reduction based on SDL models of system requirements. Journal of Software Maintenance and Evolution, 21(6), 379–405.CrossRef
Cheng, K., & Krishnakumar, A. (1993). Automatic functional test generation using the extended finite state machine model. In Proceedings ACM/IEEE design automation conference (pp. 86–91).
de Winter, J. C. (2013). Using the Student’s t-test with extremely small sample sizes. Practical Assessment, Research and Evaluation, 18(10), 1–2.
Deng, D., Sheu, P. C.-Y., Wang, T., & Onoma, A. K. (2004). Model-based testing and maintenance. In Proceedings of the IEEE sixth international symposium on multimedia software engineering (ISMSE) (pp. 278–285).
Dick, J., & Faivre, A. (1992). Automating the generation and sequencing of test case from model-based specification. In Proceedings of the international symposium on formal methods (pp. 268–284).
Do, H., Elbaum, S., & Rothermel, G. (2005). Supporting controlled experimentation with testing techniques: An infrastructure and its potential impact. Empirical Software Engineering, 10(4), 405–435.CrossRef
Do, H., & Rothermel, G. (2006a). On the use of mutation faults in empirical assessments of test case prioritization techniques. IEEE Transactions on Software Engineering, 32(9), 733–752.CrossRef
Do, H., & Rothermel, G. (2006b). On the use of mutation faults in empirical assessments of test case prioritization techniques. IEEE Transactions on Software Engineering (TSE), 32(9), 733–752.CrossRef
Elbaum, S., Malishevsky, A. G., & Rothermel, G. (2000). Prioritizing test cases for regression testing (Vol. 25, no. 5). ACM.
Elbaum, S., Malishevsky, A., & Rothermel, G. (2001). Incorportaing varying test costs and fault severities into test case prioritization. In Proceedings of the 23rd international conference on software engineering, ICSE-01 (pp. 329–338).
Elbaum, S., Malishevsky, A., & Rothermel, G. (2002). Test case prioritization: A family of empirical studies. IEEE Transactions on Software Engineering, 28, 159–182.CrossRef
Farooq, Q., Iqbal, M. Z. Z., Malik, Z. I., & Nadeem, A. (2007). An approach for selective state machine based regression testing. In Proceedings of the 3rd international ACM workshop on advances in model-based testing (pp. 44–52). London.
Ferrante, J., Ottenstein, K., & Warren, J. (1987). The program dependence graph and its use in optimization. ACM Transactions on Programming Languages and Systems, 9(5), 319–349.CrossRefMATH
Fraser, G., & Wotawa, F. (2007). Test-case prioritization with model-checkers. In Proceedings of the 25th IASTED international multi-conference: Software engineering (pp. 267–272). Innsbruck.
Gupta, R., Harrold, M., & Soffa, M. (1992). An approach to regression testing using slices. In Proceedings of the IEEE international conference on software maintenance (pp. 299–308).
Harrold, M. J., & Orso, A. (2008). Retesting software during development and maintenance. In Frontiers of software maintenance (FoSM 2008), international conference on software maintenance (pp. 99–108). Beijing.
Hemmati, H., Arcuri, A., & Briand, L. (2013). Achieving scalable model-based testing through test case diversity. ACM Transaction on Software Engineering, 22(1), 6–48.
Hong, H., & Ural, H. (2000). A test sequence selection method for statecharts. Software Testing, Verification, and Reliability, 10(4), 203–227.CrossRef
ITU-T Recommendation Standard, Q.931. (1998). Digital subscriber signaling system No. 1: Network layer, ISDN User Network Interface Layer 3 Specifications for Basic Call Control.
Jia, Y., & Harman, M. (2011). An analysis and survey of the development of mutation testing. In IEEE transactions on software engineering. in press.
Jiang, B., Tse, T. H., Grieskamp, W., Kicillof, N., Cao, Y., Li, X., et al. (2011). Assuring the model evolution of protocol software specifications by regression testing process improvement. Software: Practice and Experience, 41(10), 1073–1103.
Jiang, B., Zhang, Z., Chan, W. K., & Tse, T. H. (2009). Adaptive random test case prioritization. In Proceedings of the 2009 IEEE/ACM international conference on automated software engineering (ASE ‘09), IEEE computer society (pp. 233–244). Washington, DC.
Kaur, P., Bansal, P., & Sibal, R. (2012). Prioritization of test scenarios derived from UML activity diagram using path complexity. Proceedings of the CUBE international information technology conference (CUBE ‘2012) (pp. 355–359). New York, NY: ACM.CrossRef
Khoshgoftaar, T. M., et al. (1997). Application of neural networks to software quality modeling of a very large telecommunications system. IEEE Transactions on Neural Networks, 8(4), 902–909.CrossRef
Kim, J., & Porter, A. (2002). A history-based test prioritization technique for regression testing in resource constraint environments. In Proceedings of the 24th international conference on software engineering (pp. 119–129).
Kirby, J. (1987). Example NRL/SCR software requirements for an automobile cruise control and monitoring system. In Technical Report TR-87-07, Wang Institute of Graduate Studies.
Korel, B., & Koutsogiannakis, G. (2009). Empirical comparison of code-based and model-based test prioritization. In 5th Workshop on Advances in Model Based Testing, A-MOST 2009, IEEE digital library. Denver.
Korel, B., Koutsogiannakis, G., & Tahat, L. (2007). Model-based test prioritization heuristic methods and their evaluation. In Proceedings of the 3rd ACM workshop on advances in model based testing, A-MOST.
Korel, B., Koutsogiannakis, G., & Tahat, L. (2008). Application of system models in regression test suite prioritization. In IEEE international conference on software maintenance. ICSM 2008 (pp. 247–256). Beijing.
Korel, B., Tahat, L., & Harman, M. (2005). Test prioritization using system models. In Proceedings of the 21st IEEE International Conference on Software Maintenance (ICSM ‘05) (pp. 559–568).
Korel, B., Tahat, L., & Vaysburg, B. (2002). Model based regression test reduction using dependence analysis. In Proceedings of the IEEE international conference on software maintenance (pp. 214–223).
Kundu, D., Sarma, M., Samanta, D., & Mall, R. (2009). System testing for object-oriented systems with test case prioritization. Software Testesting, Verification and Reliability, 19(1), 297–333. doi:10.​1002/​stvr.​407.CrossRef
Le Traon, Y., Jéron, T., Jézéquel, J.-M., & Morel, P. (2000). Efficient object-oriented integration and regression testing. IEEE Transactions on Reliability, 49(1), 12–25.CrossRef
Li, Z., Harman, M., & Hierons, R. (2007). Search algorithms for regression test case prioritization. IEEE Transactions on Software Engineering, 33(4), 225–237.CrossRef
Mei, L., Cai, Y., Jia, C., Jiang, B., Chan, W. K., Zhang, Z., et al. (2014). A subsumption hierarchy of test case prioritization for composite services. IEEE Transactions on Services Computing,. doi:10.​1109/​TSC.​2014.​2331683.
Mei, L., Chan, W. K., Tse, T. H., & Merkel, R. G. (2011a). XML-manipulating test case prioritization for XML-manipulating services. JSS, 84(4), 603–619.
Mei, H., Hao, D., Zhang, L., Zhang, L., Zhou, J., & Rothermel, G. (2011). A static approach to prioritizing JUnit test cases. In IEEE transactions on software engineering.
Muccini, H., Dias, M. S., & Richardson, D. J. (2005). Reasoning about software architecture-based regression testing through a case study. In International computer software and applications conference (COMPSAC) (pp. 189–195).
Mueller C. (2003). Automated interface propping applied to cost component evaluation. Ph.D. Thesis, Illinois Institute of Technology, Chicago, IL.
Offutt, A. J., Lee, A., Rothermel, G., Untch, R., & Zapf, C. (1996). An experimental determination of sufficient mutation operators. ACM Transactions on Software Engineering and Methodology, 5(2), 99–118.CrossRef
Orso, A., Do, H., Rothermel, G., Harrold, M. J., & Rosenblum, D. S. (2007). Using component metadata to regression test component-based software. Software Testing Verification and Reliability (STVR), 17(2), 61–94.CrossRef
Paiva, A. C. R., Faria, J. C. P., Tillmann, N., & Vidal, R. F. A. M. (2005). A model-to-implementation mapping tool for automated model-based GUI testing. In Formal methods and software engineering lecture notes in computer science (vol. 3785, pp. 450–464).
Pilskalns, O., Uyan, G., & Andrews, A. (2006). Regression testing UML designs. In Proceedings of the 22nd IEEE international conference on software maintenance (pp. 254–264).
Rothermel, G., & Harrold, M. (1994). Selecting tests and identifying test coverage requirements for modified software. In Proceedings of the IEEE international conference on software maintenance (pp. 358–367).
Rothermel, G., & Harrold, M. (1997). A safe, efficient regression test selection technique. ACM Transactions on Software Engineering and Methodology, 6(2), 173–210.CrossRef
Rothermel, G., Harrold, M., Ostrin, J., & Hong, C. (1998). An empirical study of the effects of minimization on the fault detection capabilities of test suites. In Proceedings of the international conference on software maintenance (pp. 34–43).
Rothermel, G., Untch, R., Chu, C., & Harrold, M. (1999). Test case prioritization: An empirical study. In Proceedings of the IEEE international conference on software maintenance (pp. 179–188).
Rothermel, G., Untch, R. H., Chu, C., & Harrold, M. J. (2001). Prioritizing test cases for regression testing. IEEE Transactions on Software Engineering, 27(10), 929–948.CrossRef
Saadatmand, M., & Cichetti, A. (2014). Mapping of state machines to code: potentials and challenges. In The Ninth International Conference on Software Engineering Advances, ICSEA-2014 (pp. 247–252).
Sabharwal, S., Sibal, R., & Sharma, C. (2010). Prioritization of test case scenarios derived from activity diagram using genetic algorithm. In Proceeding of the computer and communication technology international conference (pp. 481–485).
Srivastava, A., & Thiagarajan, J. (2002). Effectively prioritizing tests in development environment. In Proceedings of the ACM international symposium on software testing and analysis, ISSTA-02 (pp. 97–106).
Stevens, W. (2001). TCP/IP Illustration, volume I, the Protocol. Boston, MA: Addison Wesley Reading.
Tahat, L., Korel, B., Harman, M., & Ural, H. (2012). Regression test suite prioritization using system models. Journal of Software Testing, Verification, and Reliability, 12(1), 481–506.CrossRef
Thomas, S. W., Hemmati, H., Hassan, A. E., & Blostein, D. (2014). Static test case prioritization using topic models. Empirical Software Engineering, 19(1), 182–212.CrossRef
Ural, H., & Yenigun, H. (2013). Regression test suite selection using dependence analysis. Journal of Software: Evolution and Process, 25(7), 681–709.MATH
Vaysburg, B., Tahat, L.,& Korel, B. (2002). Dependence analysis in reduction of requirement based test suites. In Proceedings of the ACM international symposium on software testing and analysis (pp. 107–111).
Wakinshaw, N., Bogdanov, K., Ali, S., & Holcombe, M. (2008). Automated discovery of state transitions and their functions in source code. Journal of Software Testing, Verification and Reliability, 18(2), 99–121.CrossRef
White, L. (1993). Test manager: A regression testing tool. In Proceedings of the IEEE international conference on software maintenance (pp. 338–347).
Wong, W., Horgan, J., London, S., & Agrawal, H. (1997b). A study of effective regression testing in practice. In Proceedings of the international symposium on software reliability engineering (pp. 230–238).
Wong, W. E., Horgan, J. R., Mathur, A. P., & Pasquini, A. (1997a). Test set size minimization and fault detection effectiveness: A case study in a space application. In Proceedings of 21st annual international computer software and applications conference (pp. 522–528).
Wu, Y., & Offutt, J. (2003). Maintaining evolving component-based software with UML. In Seventh European conference on software maintenance and reengineering (CSMR) (pp. 133–142).
Yenigun, H. (2014). Identifying the effects of modifications as data dependencies. Software Quality Journal, 22, 701–716.CrossRef
Yoo, S., & Harman, M. (2009). Regression testing minimisation, selection and prioritisation: A survey. In Department of Computer Science, King’s College London, Technical Report: TR-09-09.
Yoo, S., & Harman, M. (2012). Regression testing minimization, selection and prioritization: A survey. Software, Testing, Verification, and Reliability, 22(2), 67–120.CrossRef
Yoo, S., Harman, M., Tonella, P., & Susi, A. (2009). Clustering test cases to achieve effective and scalable prioritisation incorporating expert knowledge. In Proceedings of the eighteenth international symposium on Software testing and analysis (pp. 201–212). ACM.
Zhai, K., Jiang, B., & Chan, W. K. (2014). Prioritizing test cases for regression testing of location-based services: Metrics, techniques, and case study. IEEE Transactions on Services Computing, 7(1), 54–67.CrossRef
Metadata
Title
State-based models in regression test suite prioritization
Authors
Luay Tahat
Bogdan Korel
George Koutsogiannakis
Nada Almasri
Publication date
14-07-2016
Publisher
Springer US
Published in
Software Quality Journal / Issue 3/2017
Print ISSN: 0963-9314
Electronic ISSN: 1573-1367
DOI
https://doi.org/10.1007/s11219-016-9330-x

Other articles of this Issue 3/2017

Software Quality Journal 3/2017 Go to the issue

OriginalPaper

A model-based framework encompassing a complete workflow from specification until validation of timing requirements in embedded software systems

OriginalPaper

Toward automatically quantifying the impact of a change in systems

OriginalPaper

APOGEN: automatic page object generator for web testing

EditorialNotes

In this issue

OriginalPaper

Investigating the relation between lexical smells and change- and fault-proneness: an empirical study

OriginalPaper

Automated refinement of models for model-based testing using exploratory testing

Premium Partner

    Image Credits