nach oben

Empirical Software Engineering

Erschienen in:

01.07.2021

An empirical study on the use of SZZ for identifying inducing changes of non-functional bugs

verfasst von: Sophia Quach, Maxime Lamothe, Yasutaka Kamei, Weiyi Shang

Erschienen in: Empirical Software Engineering | Ausgabe 4/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Non-functional bugs, e.g., performance bugs and security bugs, bear a heavy cost on both software developers and end-users. For example, IBM estimates the cost of a single data breach to be millions of dollars. Tools to reduce the occurrence, impact, and repair time of non-functional bugs can therefore provide key assistance for software developers racing to fix these issues. Identifying bug-inducing changes is a critical step in software quality assurance. In particular, the SZZ approach is commonly used to identify bug-inducing commits. However, the fixes to non-functional bugs may be scattered and separate from their bug-inducing locations in the source code. The nature of non-functional bugs may therefore make the SZZ approach a sub-optimal approach for identifying bug-inducing changes. Yet, prior studies that leverage or evaluate the SZZ approach do not consider non-functional bugs, leading to potential bias on the results. In this paper, we conduct an empirical study on the results of the SZZ approach when used to identify the inducing changes of the non-functional bugs in the NFBugs dataset. We eliminate a majority of the bug-inducing commits as they are not in the same method or class level. We manually examine whether each identified bug-inducing change is indeed the correct bug-inducing change. Our manual study shows that a large portion of non-functional bugs cannot be properly identified by the SZZ approach. By manually identifying the root causes of the falsely detected bug-inducing changes, we uncover root causes for false detection that have not been found by previous studies. We evaluate the identified bug-inducing changes based on three criteria from prior research, i.e., the earliest bug appearance, the future impact of changes, and the realism of bug introduction. We find that prior criteria may be irrelevant for non-functional bugs. Our results may be used to assist in future research on non-functional bugs, and highlight the need to complement SZZ to accommodate the unique characteristics of non-functional bugs.

Vorheriger Artikel Security assurance cases—state of the art of an emerging approach

Nächster Artikel Foreword to the Special Issue on Configurable Systems

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Our extension of the NFBugs dataset is publicly available and can be found at: https://github.com/senseconcordia/NFBugsExtended

Bryant RE, O’Hallaron DR (2015) Computer systems: a programmer’s perspective, 3rd edn. Pearson

Borg M, Svensson O, Berg K, Hansson D (2019) Szz unleashed: an open implementation of the szz algorithm—featuring example usage in a study of just-in-time bug prediction for the jenkins project. In: Proceedings of the 3rd ACM SIGSOFT international workshop on machine learning techniques for software quality evaluation, ser. MaLTeSQuE 2019. Association for Computing Machinery, New York, pp 7–12. [Online]. Available: https://doi.org/10.1145/3340482.3342742

da Costa DA, McIntosh S, Shang W, Kulesza U, Coelho R, Hassan AE (2017) A framework for evaluating the results of the szz approach for identifying bug-introducing changes. IEEE Trans Softw Eng 43(7):641–657CrossRef

Davies S, Roper M, Wood M (2014) mparing text-based and dependence-based approaches for determining the origins of bugs. J Softw: Evol Process 26:01CrossRef

Fan Y, Xia X, Alencar da Costa D, Lo D, Hassan AE, Li S (2019) The impact of changes mislabeled by szz on just-in-time defect prediction. IEEE Trans Softw Eng 1–1

Glinz M (2007) On non-functional requirements. In: 15th IEEE international requirements engineering conference (RE 2007), pp 21–26

Grubb P, Takang A (2003) Software maintenance—concepts and practice (2nd edn). 01

Guindon C Swt: the standard widget toolkit. [Online]. Available: https://www.eclipse.org/swt/

Gyimothy T, Ferenc R, Siket I (2005) Empirical validation of object-oriented metrics on open source software for fault prediction. IEEE Trans Softw Eng 31(10):897–910CrossRef

Hamill M, Goseva-Popstojanova K (2014) Exploring the missing link: an empirical study of software fixes. Softw Test Verif Reliab 24(8):684–705. [Online]. Available: https://doi.org/10.1002/stvr.1518CrossRef

Hassan AE (2009) Predicting faults using the complexity of code changes. In: 2009 IEEE 31st international conference on software engineering, pp 78–88

Jin G, Song L, Shi X, Scherpelz J, Lu S (2012) Understanding and detecting real-world performance bugs. SIGPLAN Not 47(6):77–88CrossRef

Jpace (2018) jpace/diffj. [Online]. Available: https://github.com/jpace/dij

Kamei Y, Shihab E, Adams B, Hassan AE, Mockus A, Sinha A, Ubayashi N (2013) A large-scale empirical study of just-in-time quality assurance. IEEE Trans Softw Eng 39(6):757–773CrossRef

Kim M, Lee E (2018) Are information retrieval-based bug localization techniques trustworthy?. In: Proceedings of the 40th international conference on software engineering: companion proceedings, ser. ICSE ’18. Association for Computing Machinery, New York, pp 248–249. [Online]. Available: https://doi.org/10.1145/3183440.3194954

Kim S, Whitehead EJ Jr (2006) How long did it take to fix bugs?. In: Proceedings of the 2006 international workshop on mining software repositories, ser. MSR ’06. ACM, New York, pp 173–174

Kim S, Zimmermann T, Pan K, Whitehead EJ Jr (2006) Automatic identification of bug-introducing changes. In: 21st IEEE/ACM international conference on automated software engineering (ASE’06), pp 81–90

Kotonya G, Sommerville I (1998) Requirements engineering: processes and techniques, 1st edn, Wiley Publishing

LaToza TD, Venolia G, DeLine R (2006) Maintaining mental models: a study of developer work habits. In: Proceedings of the 28th international conference on software engineering, ser. ICSE ’06. ACM, New York, pp 492–501

Mahrous H, Malhotra B (2018) Managing publicly known security vulnerabilities in software systems. In: 2018 16th Annual conference on privacy, security and trust (PST), pp 1–10

McHugh M (2012) Interrater reliability: the kappa statistic. Biochemia medica: časopis Hrvatskoga društva medicinskih biokemičara / HDMB 22:276–82, 10CrossRef

McIntosh S, Kamei Y (2018) Are fix-inducing changes a moving target? A longitudinal case study of just-in-time defect prediction. IEEE Trans Softw Eng 44(5):412–428CrossRef

Molyneaux I (2009) The art of application performance testing: help for programmers and quality assurance, 1st edn. O’Reilly Media, Inc.

Neto C, Barbalho E (2018) Enhancing the szz algorithm to deal with refactoring changes

Neto E, Costa D, Kulesza U (2018) The impact of refactoring changes on the szz algorithm: an empirical study. 03

Nistor A, Jiang T, Tan L (2013) Discovering, reporting, and fixing performance bugs. In: 2013 10th Working conference on mining software repositories (MSR), pp 237–246

Nugroho YS, Hata H, Matsumoto K (2019) How different are different diff algorithms in git? Empir Softw Eng 25(1):790–823. [Online]. Available: https://doi.org/10.1007/s10664-019-09772-zCrossRef

Ohira M, Kashiwa Y, Yamatani Y, Yoshiyuki H, Maeda Y, Limsettho N, Fujino K, Hata H, Ihara A, Matsumoto K (2015) A dataset of high impact bugs: manually-classified issue reports. In: Proceedings of the 12th working conference on mining software repositories, ser. MSR ’15. IEEE Press, Piscataway, pp 518–521

Pan K, Kim S, Whitehead EJ Jr (2009) Toward an understanding of bug fix patterns. Empir Softw Eng 14(3):286–315CrossRef

Ping L, Jin S, Xinfeng Y (2011) Research on software security vulnerability detection technology. In: Proceedings of 2011 international conference on computer science and network technology, vol 3, pp 1873–1876

Radu A, Nadi S (2019) A dataset of non-functional bugs. In: Proceedings of the 16th international conference on mining software repositories, ser. MSR ’19. IEEE Press, Piscataway, pp 399–403

Rosen C, Grawi B, Shihab E (2015) Commit guru: analytics and risk prediction of software commits. In: Proceedings of the 2015 10th joint meeting on foundations of software engineering, ser. ESEC/FSE 2015. Association for Computing Machinery, New York, pp 966–969. [Online]. Available: https://doi.org/10.1145/2786805.2803183

Śliwerski J, Zimmermann T, Zeller A (2005) When do changes induce fixes? SIGSOFT Softw Eng Notes 30(4):1–5CrossRef

Steidl D, Hummel B, Juergens E (2014) Incremental origin analysis of source code files. In: Proceedings of the 11th working conference on mining software repositories, ser. MSR 2014. Association for Computing Machinery, New York, pp 42–51. [Online]. Available: https://doi.org/10.1145/2597073.2597111

Team J. Eclipse java development tools (jdt). [Online]. Available: https://www.eclipse.org/jdt/

Williams C, Spacco J (2008) Szz revisited: verifying when changes induce fixes. In: Proceedings of the 2008 workshop on defects in large software systems, ser. DEFECTS ’08. ACM, New York, pp 32–36

Williams L, McGraw G, Migues S (2018) Engineering security vulnerability prevention, detection, and response. IEEE Softw 35(5):76–80CrossRef

Zaman S, Adams B, Hassan AE (2011) Security versus performance bugs: a case study on firefox. In: Proceedings of the 8th working conference on mining software repositories, ser. MSR ’11. ACM, New York, pp 93–102

Titel: An empirical study on the use of SZZ for identifying inducing changes of non-functional bugs
verfasst von: Sophia Quach
Maxime Lamothe
Yasutaka Kamei
Weiyi Shang
Publikationsdatum: 01.07.2021
Verlag: Springer US
Erschienen in: Empirical Software Engineering / Ausgabe 4/2021
Print ISSN: 1382-3256
Elektronische ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-021-09970-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Weitere Artikel der Ausgabe 4/2021

Automatic team recommendation for collaborative software development

Promises and challenges of microservices: an exploratory study

Assessing practitioner beliefs about software engineering

Analysing Time-Stamped Co-Editing Networks in Software Development Teams using git2net

Foreword to the Special Issue on Configurable Systems

Reliability of self-rated experience and confidence as predictors for students’ performance in software engineering

Premium Partner