Skip to main content

2017 | OriginalPaper | Buchkapitel

Integration of Metamorphic Testing with Program Repair Methods Based on Adaptive Search Strategies and Program Equivalence

verfasst von : Tingting Wu, Yunwei Dong, Tsong Yueh Chen, Mingyue Jiang, Man Lau, Fei-Ching Kuo, Sebastian Ng

Erschienen in: Formal Methods and Software Engineering

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Automated program repair (APR) is a promising approach to localize faults and generate patches for program under repair. One of the test suite based APR techniques, a method leveraging Adaptive search strategies and program Equivalence (AE), has been commonly used for program repair. AE assumes the availability of test oracles, which brings the oracle problem to AE repair procedure. Metamorphic Testing (MT) has been introduced to alleviate the test oracle problem, and it tests the correctness of programs through metamorphic relations (MRs) which are relations among multiple inputs and outputs. This paper presents an integration of AE with MT (referred to as AE-MT) to extend the applicability of AE to those applications with oracle problems. To evaluate the repair effectiveness of AE-MT, an empirical study is conducted against IntroClass benchmark. We conclude that AE-MT outperforms AE in terms of success rate, but is of lower repair quality than AE.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Arcuri, A.: On the automation of fixing software bugs. In: Companion of the 30th International Conference on Software Engineering, pp. 1003–1006. ACM (2008) Arcuri, A.: On the automation of fixing software bugs. In: Companion of the 30th International Conference on Software Engineering, pp. 1003–1006. ACM (2008)
2.
Zurück zum Zitat Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The oracle problem in software testing: a survey. IEEE Trans. Softw. Eng. 41(5), 507–525 (2015)CrossRef Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The oracle problem in software testing: a survey. IEEE Trans. Softw. Eng. 41(5), 507–525 (2015)CrossRef
3.
Zurück zum Zitat Cao, Y., Zhou, Z.Q., Chen, T.Y.: On the correlation between the effectiveness of metamorphic relations and dissimilarities of test case executions. In: 2013 13th International Conference on Quality Software (QSIC), pp. 153–162. IEEE (2013) Cao, Y., Zhou, Z.Q., Chen, T.Y.: On the correlation between the effectiveness of metamorphic relations and dissimilarities of test case executions. In: 2013 13th International Conference on Quality Software (QSIC), pp. 153–162. IEEE (2013)
4.
Zurück zum Zitat Chan, F., Chen, T., Cheung, S.C., Lau, M., Yiu, S.: Application of metamorphic testing in numerical analysis. In: Proceedings of the IASTED International Conference on Software Engineering (SE98) (1998) Chan, F., Chen, T., Cheung, S.C., Lau, M., Yiu, S.: Application of metamorphic testing in numerical analysis. In: Proceedings of the IASTED International Conference on Software Engineering (SE98) (1998)
5.
Zurück zum Zitat Chen, T.Y., Cheung, S.C., Yiu, S.M.: Metamorphic testing: a new approach for generating next test cases. Department of Computer Science, Hong Kong University of Science and Technology, Technical report HKUST-CS98-01 (1998) Chen, T.Y., Cheung, S.C., Yiu, S.M.: Metamorphic testing: a new approach for generating next test cases. Department of Computer Science, Hong Kong University of Science and Technology, Technical report HKUST-CS98-01 (1998)
6.
Zurück zum Zitat Chen, T.Y., Huang, D., Tse, T., Zhou, Z.Q.: Case studies on the selection of useful relations in metamorphic testing. In: Proceedings of the 4th Ibero-American Symposium on Software Engineering and Knowledge Engineering (JIISIC 2004), pp. 569–583. Polytechnic University of Madrid (2004) Chen, T.Y., Huang, D., Tse, T., Zhou, Z.Q.: Case studies on the selection of useful relations in metamorphic testing. In: Proceedings of the 4th Ibero-American Symposium on Software Engineering and Knowledge Engineering (JIISIC 2004), pp. 569–583. Polytechnic University of Madrid (2004)
7.
Zurück zum Zitat Chen, T.Y., Tse, T., Zhou, Z.Q.: Semi-proving: an integrated method for program proving, testing, and debugging. IEEE Trans. Softw. Eng. 37(1), 109–125 (2011)CrossRef Chen, T.Y., Tse, T., Zhou, Z.Q.: Semi-proving: an integrated method for program proving, testing, and debugging. IEEE Trans. Softw. Eng. 37(1), 109–125 (2011)CrossRef
8.
Zurück zum Zitat Forrest, S., Nguyen, T., Weimer, W., Le Goues, C.: A genetic programming approach to automated software repair. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pp. 947–954. ACM (2009) Forrest, S., Nguyen, T., Weimer, W., Le Goues, C.: A genetic programming approach to automated software repair. In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation, pp. 947–954. ACM (2009)
9.
Zurück zum Zitat Gotlieb, A., Botella, B.: Automated metamorphic testing. In: 2003 27th Annual International Computer Software and Applications Conference (COMPSAC), pp. 34–40. IEEE (2003) Gotlieb, A., Botella, B.: Automated metamorphic testing. In: 2003 27th Annual International Computer Software and Applications Conference (COMPSAC), pp. 34–40. IEEE (2003)
10.
Zurück zum Zitat Arcuri, A., Briand, L.: A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: 2011 33rd International Conference on Software Engineering (ICSE), pp. 1–10. IEEE (2011) Arcuri, A., Briand, L.: A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: 2011 33rd International Conference on Software Engineering (ICSE), pp. 1–10. IEEE (2011)
11.
Zurück zum Zitat Jiang, M., Chen, T.Y., Kuo, F.C., Ding, Z., Choi, E.H., Mizuno, O.: A revisit of the integration of metamorphic testing and test suite based automated program repair. In: The 2nd International Workshop on Metamorphic Testing, pp. 14–20. IEEE (2017) Jiang, M., Chen, T.Y., Kuo, F.C., Ding, Z., Choi, E.H., Mizuno, O.: A revisit of the integration of metamorphic testing and test suite based automated program repair. In: The 2nd International Workshop on Metamorphic Testing, pp. 14–20. IEEE (2017)
12.
Zurück zum Zitat Jiang, M., Chen, T.Y., Kuo, F.C., Towey, D., Ding, Z.: A metamorphic testing approach for supporting program repair without the need for a test oracle. J. Syst. Softw. 126, 127–140 (2016)CrossRef Jiang, M., Chen, T.Y., Kuo, F.C., Towey, D., Ding, Z.: A metamorphic testing approach for supporting program repair without the need for a test oracle. J. Syst. Softw. 126, 127–140 (2016)CrossRef
13.
Zurück zum Zitat Jiang, M., Chen, T.Y., Kuo, F.C., Zhou, Z., Ding, Z.: Testing model transformation programs using metamorphic testing. In: The 26th International Conference on Software Engineering and Knowledge Engineering (SEKE), pp. 94–99 (2014) Jiang, M., Chen, T.Y., Kuo, F.C., Zhou, Z., Ding, Z.: Testing model transformation programs using metamorphic testing. In: The 26th International Conference on Software Engineering and Knowledge Engineering (SEKE), pp. 94–99 (2014)
14.
Zurück zum Zitat Le Goues, C., Holtschulte, N., Smith, E.K., Brun, Y., Devanbu, P., Forrest, S., Weimer, W.: The manybugs and introclass benchmarks for automated repair of c programs. IEEE Trans. Softw. Eng. 41(12), 1236–1256 (2015)CrossRef Le Goues, C., Holtschulte, N., Smith, E.K., Brun, Y., Devanbu, P., Forrest, S., Weimer, W.: The manybugs and introclass benchmarks for automated repair of c programs. IEEE Trans. Softw. Eng. 41(12), 1236–1256 (2015)CrossRef
15.
Zurück zum Zitat Liu, H., Kuo, F.C., Towey, D., Chen, T.Y.: How effectively does metamorphic testing alleviate the oracle problem? IEEE Trans. Softw. Eng. 40(1), 4–22 (2014)CrossRef Liu, H., Kuo, F.C., Towey, D., Chen, T.Y.: How effectively does metamorphic testing alleviate the oracle problem? IEEE Trans. Softw. Eng. 40(1), 4–22 (2014)CrossRef
16.
Zurück zum Zitat Mechtaev, S., Yi, J., Roychoudhury, A.: Angelix: scalable multiline program patch synthesis via symbolic analysis. In: Proceedings of the 38th International Conference on Software Engineering, pp. 691–701. ACM (2016) Mechtaev, S., Yi, J., Roychoudhury, A.: Angelix: scalable multiline program patch synthesis via symbolic analysis. In: Proceedings of the 38th International Conference on Software Engineering, pp. 691–701. ACM (2016)
17.
Zurück zum Zitat Murphy, C., Shen, K., Kaiser, G.: Automatic system testing of programs without test oracles. In: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, pp. 189–200. ACM (2009) Murphy, C., Shen, K., Kaiser, G.: Automatic system testing of programs without test oracles. In: Proceedings of the Eighteenth International Symposium on Software Testing and Analysis, pp. 189–200. ACM (2009)
18.
Zurück zum Zitat Nguyen, H.D.T., Qi, D., Roychoudhury, A., Chandra, S.: Semfix: program repair via semantic analysis. In: Proceedings of the 2013 International Conference on Software Engineering, pp. 772–781. IEEE Press (2013) Nguyen, H.D.T., Qi, D., Roychoudhury, A., Chandra, S.: Semfix: program repair via semantic analysis. In: Proceedings of the 2013 International Conference on Software Engineering, pp. 772–781. IEEE Press (2013)
19.
Zurück zum Zitat Nguyen, T.: Automating program verification and repair using invariant analysis and test input generation. Ph.D. thesis (2014) Nguyen, T.: Automating program verification and repair using invariant analysis and test input generation. Ph.D. thesis (2014)
20.
Zurück zum Zitat Qi, Y., Mao, X., Lei, Y.: Efficient automated program repair through fault-recorded testing prioritization. In: 2013 29th IEEE International Conference on Software Maintenance (ICSM), pp. 180–189. IEEE (2013) Qi, Y., Mao, X., Lei, Y.: Efficient automated program repair through fault-recorded testing prioritization. In: 2013 29th IEEE International Conference on Software Maintenance (ICSM), pp. 180–189. IEEE (2013)
21.
Zurück zum Zitat Qi, Z., Long, F., Achour, S., Rinard, M.: An analysis of patch plausibility and correctness for generate-and-validate patch generation systems. In: Proceedings of the 2015 International Symposium on Software Testing and Analysis, pp. 24–36. ACM (2015) Qi, Z., Long, F., Achour, S., Rinard, M.: An analysis of patch plausibility and correctness for generate-and-validate patch generation systems. In: Proceedings of the 2015 International Symposium on Software Testing and Analysis, pp. 24–36. ACM (2015)
22.
Zurück zum Zitat Segura, S., Fraser, G., Sanchez, A.B., Ruiz-Cortés, A.: A survey on metamorphic testing. IEEE Trans. Softw. Eng. 42(9), 805–824 (2016)CrossRef Segura, S., Fraser, G., Sanchez, A.B., Ruiz-Cortés, A.: A survey on metamorphic testing. IEEE Trans. Softw. Eng. 42(9), 805–824 (2016)CrossRef
23.
Zurück zum Zitat Sidiroglou-Douskos, S., Lahtinen, E., Long, F., Rinard, M.: Automatic error elimination by horizontal code transfer across multiple applications. ACM SIGPLAN Not. 50, 43–54 (2015). ACMCrossRef Sidiroglou-Douskos, S., Lahtinen, E., Long, F., Rinard, M.: Automatic error elimination by horizontal code transfer across multiple applications. ACM SIGPLAN Not. 50, 43–54 (2015). ACMCrossRef
24.
Zurück zum Zitat Weimer, W., Fry, Z.P., Forrest, S.: Leveraging program equivalence for adaptive program repair: models and first results. In: 2013 IEEE/ACM 28th International Conference on Automated Software Engineering (ASE), pp. 356–366. IEEE (2013) Weimer, W., Fry, Z.P., Forrest, S.: Leveraging program equivalence for adaptive program repair: models and first results. In: 2013 IEEE/ACM 28th International Conference on Automated Software Engineering (ASE), pp. 356–366. IEEE (2013)
25.
Zurück zum Zitat Weimer, W., Nguyen, T., Le Goues, C., Forrest, S.: Automatically finding patches using genetic programming. In: Proceedings of the 31st International Conference on Software Engineering, pp. 364–374. IEEE Computer Society (2009) Weimer, W., Nguyen, T., Le Goues, C., Forrest, S.: Automatically finding patches using genetic programming. In: Proceedings of the 31st International Conference on Software Engineering, pp. 364–374. IEEE Computer Society (2009)
26.
Zurück zum Zitat Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bull. 1(6), 80–83 (1945)CrossRef Wilcoxon, F.: Individual comparisons by ranking methods. Biometrics Bull. 1(6), 80–83 (1945)CrossRef
27.
Zurück zum Zitat Xie, X., Wong, W.E., Chen, T.Y., Xu, B.: Metamorphic slice: an application in spectrum-based fault localization. Inf. Softw. Technol. 55(5), 866–879 (2013)CrossRef Xie, X., Wong, W.E., Chen, T.Y., Xu, B.: Metamorphic slice: an application in spectrum-based fault localization. Inf. Softw. Technol. 55(5), 866–879 (2013)CrossRef
Metadaten
Titel
Integration of Metamorphic Testing with Program Repair Methods Based on Adaptive Search Strategies and Program Equivalence
verfasst von
Tingting Wu
Yunwei Dong
Tsong Yueh Chen
Mingyue Jiang
Man Lau
Fei-Ching Kuo
Sebastian Ng
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-68690-5_25

Premium Partner