Top

Published in:

2019 | OriginalPaper | Chapter

Do Null-Type Mutation Operators Help Prevent Null-Type Faults?

Authors : Ali Parsai, Serge Demeyer

Published in: SOFSEM 2019: Theory and Practice of Computer Science

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The null-type is a major source of faults in Java programs, and its overuse has a severe impact on software maintenance. Unfortunately traditional mutation testing operators do not cover null-type faults by default, hence cannot be used as a preventive measure. We address this problem by designing four new mutation operators which model null-type faults explicitly. We show how these mutation operators are capable of revealing the missing tests, and we demonstrate that these mutation operators are useful in practice. For the latter, we analyze the test suites of 15 open-source projects to describe the trade-offs related to the adoption of these operators to strengthen the test suite.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Two-Head Finite-State Acceptors with Translucent Letters

next chapter Towards Combining Multitask and Multilingual Learning

Alexander, I.: Misuse cases: use cases with hostile intent. IEEE Softw. 20(1), 58–66 (2003). https://doi.org/10.1109/ms.2003.1159030CrossRef

Ammann, P., Delamaro, M.E., Offutt, J.: Establishing theoretical minimal sets of mutants. In: 2014 IEEE Seventh International Conference on Software Testing, Verification and Validation, pp. 21–30. IEEE, March 2014. https://doi.org/10.1109/icst.2014.13

Bradbury, J.S., Cordy, J.R., Dingel, J.: Mutation operators for concurrent Java (J2SE 5.0). In: MUTATION 2006, Proceedings of the Second Workshop on Mutation Analysis (Mutation 2006 - ISSRE Workshops 2006), School of Computing, Queen’s University, Kingston, p. 11. IEEE, Washington, DC, November 2006. https://doi.org/10.1109/mutation.2006.10

Coles, H., Laurent, T., Henard, C., Papadakis, M., Ventresque, A.: PIT: a practical mutation testing tool for Java (Demo). In: Proceedings of the 25th International Symposium on Software Testing and Analysis - ISSTA 2016, pp. 449–452. ACM Press, New York (2016). https://doi.org/10.1145/2931037.2948707

DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: help for the practicing programmer. Computer 11(4), 34–41 (1978). https://doi.org/10.1109/c-m.1978.218136CrossRef

Deng, L., Offutt, J., Ammann, P., Mirzaei, N.: Mutation operators for testing Android apps. Inf. Softw. Technol. 81, 154–168 (2017). https://doi.org/10.1016/j.infsof.2016.04.012CrossRef

Ferrari, F.C., Maldonado, J., Rashid, A.: Mutation testing for aspect-oriented programs. In: 2008 International Conference on Software Testing, Verification, and Validation. ICST 2008, Department of Computer System, Sao Paulo University, Sao Carlos, pp. 52–61. IEEE, Washington, DC, April 2008. https://doi.org/10.1109/icst.2008.37

Fowler, M.: Refactoring: Improving the Design of Existing Code. Addison-Wesley, Boston (1999)MATH

Fraser, G., Zeller, A.: Mutation-driven generation of unit tests and oracles. IEEE Trans. Softw. Eng. 38(2), 278–292 (2012). https://doi.org/10.1109/tse.2011.93CrossRef

10.

Gligoric, M., Groce, A., Zhang, C., Sharma, R., Alipour, M.A., Marinov, D.: Comparing non-adequate test suites using coverage criteria. In: Proceedings of the 2013 International Symposium on Software Testing and Analysis - ISSTA 2013, pp. 302–313. ACM Press, New York (2013). https://doi.org/10.1145/2483760.2483769

11.

Gosling, J., Joy, B., Steele, G., Bracha, G., Buckley, A.: The Java Language Specification (Java SE). Oracle, 8th edn. Addison-Wesley, Boston (2014)

12.

Inozemtseva, L., Holmes, R.: Coverage is not strongly correlated with test suite effectiveness. In: Proceedings of the 36th International Conference on Software Engineering - ICSE 2014, pp. 435–445. ACM Press, New York (2014). https://doi.org/10.1145/2568225.2568271

13.

Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011). https://doi.org/10.1109/tse.2010.62CrossRef

14.

Just, R., Jalali, D., Inozemtseva, L., Ernst, M.D., Holmes, R., Fraser, G.: Are mutants a valid substitute for real faults in software testing? In: Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering - FSE 2014, pp. 654–665. ACM Press, New York (2014). https://doi.org/10.1145/2635868.2635929

15.

Kimura, S., Hotta, K., Higo, Y., Igaki, H., Kusumoto, S.: Does return null matter? In: 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE), pp. 244–253. IEEE, February 2014. https://doi.org/10.1109/csmr-wcre.2014.6747176

16.

King, K.N., Offutt, A.J.: A fortran language system for mutation-based software testing. Softw.: Pract. Exp. 21(7), 685–718 (1991). https://doi.org/10.1002/spe.4380210704CrossRef

17.

Kuhn, D.R.: Fault classes and error detection capability of specification-based testing. ACM Trans. Softw. Eng. Methodol. 8(4), 411–424 (1999). https://doi.org/10.1145/322993.322996CrossRef

18.

Kurtz, B., Ammann, P., Offutt, J.: Static analysis of mutant subsumption. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–10. IEEE, April 2015. https://doi.org/10.1109/icstw.2015.7107454

19.

Ma, Y.S., Kwon, Y.R., Offutt, J.: Inter-class mutation operators for Java. In: 13th International Symposium on Software Reliability Engineering, 2002. Proceedings, pp. 352–363. IEEE Computer Society (2002). https://doi.org/10.1109/issre.2002.1173287

20.

Nanavati, J., Wu, F., Harman, M., Jia, Y., Krinke, J.: Mutation testing of memory-related operators. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–10. IEEE, April 2015. https://doi.org/10.1109/icstw.2015.7107449

21.

Offutt, A.J., Lee, A., Rothermel, G., Untch, R.H., Zapf, C.: An experimental determination of sufficient mutant operators. ACM Trans. Softw. Eng. Methodol. 5(2), 99–118 (1996). https://doi.org/10.1145/227607.227610CrossRef

22.

Offutt, A.J., Pan, J.: Automatically detecting equivalent mutants and infeasible paths. Softw. Test. Verif. Reliab. 7(3), 165–192 (1997). https://doi.org/10.1002/(SICI)1099-1689(199709)7:3<165::AID-STVR143>3.0.CO;2-UCrossRef

23.

Oliveira, R.A., Alegroth, E., Gao, Z., Memon, A.: Definition and evaluation of mutation operators for GUI-level mutation analysis. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–10. IEEE, April 2015. https://doi.org/10.1109/icstw.2015.7107457

24.

Osman, H., Leuenberger, M., Lungu, M., Nierstrasz, O.: Tracking null checks in open-source java systems. In: 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), vol. 1, pp. 304–313. IEEE, March 2016. https://doi.org/10.1109/saner.2016.57

25.

Osman, H., Lungu, M., Nierstrasz, O.: Mining frequent bug-fix code changes. In: 2014 Software Evolution Week - IEEE Conference on Software Maintenance, Reengineering, and Reverse Engineering (CSMR-WCRE), pp. 343–347. IEEE, February 2014. https://doi.org/10.1109/csmr-wcre.2014.6747191

26.

Papadakis, M., Henard, C., Harman, M., Jia, Y., Traon, Y.L.: Threats to the validity of mutation-based test assessment. In: Proceedings of the 25th International Symposium on Software Testing and Analysis - ISSTA 2016, pp. 354–365. ACM Press, New York (2016). https://doi.org/10.1145/2931037.2931040

27.

Papadakis, M., Kintis, M., Zhang, J., Jia, Y., Traon, Y.L., Harman, M.: Mutation testing advances: an analysis and survey. In: Advances in Computers (2018). https://doi.org/10.1016/bs.adcom.2018.03.015

28.

Parsai, A.: Replication package. http://parsai.net/files/research/SofSemReplicationPackage.7z

29.

Parsai, A., Demeyer, S., De Busser, S.: C++11/14 mutation operators based on common fault patterns. In: Medina-Bulo, I., Merayo, M.G., Hierons, R. (eds.) ICTSS 2018. LNCS, vol. 11146, pp. 102–118. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99927-2_9CrossRef

30.

Parsai, A., Murgia, A., Demeyer, S.: Evaluating random mutant selection at class-level in projects with non-adequate test suites. In: Proceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering - EASE 2016, pp. 11:1–11:10. ACM Press, New York (2016). https://doi.org/10.1145/2915970.2915992

31.

Parsai, A., Murgia, A., Demeyer, S.: A model to estimate first-order mutation coverage from higher-order mutation coverage. In: 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS), pp. 365–373. IEEE, August 2016. https://doi.org/10.1109/qrs.2016.48

32.

Parsai, A., Murgia, A., Demeyer, S.: LittleDarwin: a feature-rich and extensible mutation testing framework for large and complex Java systems. In: Dastani, M., Sirjani, M. (eds.) FSEN 2017. LNCS, vol. 10522, pp. 148–163. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68972-2_10CrossRef

33.

Parsai, A., Murgia, A., Soetens, Q.D., Demeyer, S.: Mutation testing as a safety net for test code refactoring. In: Scientific Workshop Proceedings of the XP2015 on - XP 2015 workshops, pp. 8:1–8:7. ACM Press, New York (2015). https://doi.org/10.1145/2764979.2764987

34.

Parsai, A., Soetens, Q.D., Murgia, A., Demeyer, S.: Considering polymorphism in change-based test suite reduction. In: Dingsøyr, T., Moe, N.B., Tonelli, R., Counsell, S., Gencel, C., Petersen, K. (eds.) XP 2014. LNBIP, vol. 199, pp. 166–181. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-14358-3_14CrossRef

35.

Schuler, D., Zeller, A.: Javalanche: efficient mutation testing for Java. In: Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering on European Software Engineering Conference and Foundations of Software Engineering Symposium - ESEC/FSE 2009, pp. 297–298. ACM Press, New York (2009). https://doi.org/10.1145/1595696.1595750

36.

Yin, R.K.: Case Study Research: Design and Methods. Applied Social Research Methods. SAGE Publications, Thousand Oaks (2003)

Title: Do Null-Type Mutation Operators Help Prevent Null-Type Faults?
Authors: Ali Parsai
Serge Demeyer
Publisher: Springer International Publishing
Book: SOFSEM 2019: Theory and Practice of Computer Science
Print ISBN: 978-3-030-10800-7

Electronic ISBN: 978-3-030-10801-4

Copyright Year: 2019
DOI: https://doi.org/10.1007/978-3-030-10801-4_33

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner