Top

Empirical Software Engineering

Published in:

01-12-2022

The role of bug report evolution in reliable fixing estimation

Authors: Renan G. Vieira, César Lincoln C. Mattos, Lincoln S. Rocha, João Paulo P. Gomes, Matheus Paixão

Published in: Empirical Software Engineering | Issue 7/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Context

Bug reports contain information that can be used by researchers and practitioners to better understand the bug fixing process and to enable the estimation of the effort necessary to fix bugs. In general, estimation models are built using the data (e.g., fixing time, severity, number of comments, number of attachments, and number of patches) present in the reports of fixed bugs (i.e., the report final’s state). However, we claim that this approach is not reliable in a real setting. Effort estimation is necessary for bug fix scheduling and team allocation tasks, which happens closer to the bug report opening than its closing. At that moment, the data available in the bug report is less informative than the data used to build the model, which may lead to an unrealistic estimation.

Objective

We propose a new approach to estimate bug-fixing time, i.e., the time span between the moment the bug was first reported until the bug is considered fixed. We consider not only the final state of the bug report to create our estimation model but all the previous available states, different from some previous studies that do not consider the reports’ updates. The concept of bug report evolution is used to create a dataset containing all investigated report states.

Method

First, we verify how often the bug reports and their fields are updated. Next, we evaluate our approach using different machine learning methods as a classification problem, with distinct output configurations, and class balancing techniques. The experimental analysis is performed with data from the JIRA issue tracking system of ten open-source projects. By leveraging the best models (considering all possible configurations) for the different states of the evolution of a bug report, we can assess whether there are significant differences in the models’ estimation ability due to the report’s state.

Results

We gathered evidence that the reports’ fields are updated often, which characterizes the reports’ evolution, impacting the building of bug-fixing estimation models. The models’ evaluation shows promising results 0.44 up to 0.85, precision values from 0.34 up to 0.74 and recall values from 0.62 up to 0.99, depending on the project.

Conclusions

Our experiments show that field updates have a meaningful impact on the models’ performance. Furthermore, we present a new approach to deal with the bug report evolution by considering each report version as an independent report. Finally, we also make available our dataset to the community.

previous article On the documentation of self-admitted technical debt in issues

next article How Scrum adds value to achieving software quality?

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

An issue could represent a story, a bug, a task, or another issue type in the project.

In Mozilla’s (Firefox) case, it is partly automated, see https://hacks.mozilla.org/2019/04/teaching-machines-to-triage-firefox-bugs/

https://zenodo.org/record/5338495#.YS0bdVtv9H4

https://figshare.com/articles/Replication_Package_-_PROMISE_19/8852084

https://scikit-learn.org/

https://www.gpflow.org/

https://keras.io/

Akbarinasaji S, Caglayan B, Bener A (2018) Predicting bug-fixing time: a replication study using an open source software project. J Syst Softw 136:173–186. https://doi.org/10.1016/j.jss.2017.02.021. http://www.sciencedirect.com/science/article/pii/S0164121217300365CrossRef

Al-Zubaidi W H A, Dam H K, Ghose A, Li X (2017) Multi-objective search-based approach to estimate issue resolution time. In: Proceedings of the 13th international conference on predictive models and data analytics in software engineering, PROMISE. https://doi.org/10.1145/3127005.3127011. Association for Computing Machinery, New York, pp 53–62

Ardimento P, Bilancia M, Monopoli S (2016) Predicting bug-fix time: using standard versus topic-based text categorization techniques. In: Calders T, Ceci M, Malerba D (eds) Discovery science. Springer International Publishing, Cham, pp 167–182

Assar S, Borg M, Pfahl D (2016) Using text clustering to predict defect resolution time: a conceptual replication and an evaluation of prediction accuracy. Empirical Softw Engg 21(4):1437–1475. https://doi.org/10.1007/s10664-015-9391-7CrossRef

Baysal O, Holmes R, Godfrey M W (2013) Situational awareness: personalizing issue tracking systems. In: Proceedings of the 2013 international conference on software engineering, ICSE ’13. IEEE Press, pp 1185–1188

Bhattacharya P, Neamtiu I (2011) Bug-fix time prediction models: can we do better?. In: Proceedings of the 8th working conference on mining software repositories, MSR ’11. https://doi.org/10.1145/1985441.1985472. Association for Computing Machinery, New York, pp 207–210

Brady F (2013) Cambridge university report on cost of software faults, press release. http://www.prweb.com/releases/2013/1/prweb10298185.htm. Accessed 2020-01-02

Chawla N V, Bowyer K W, Hall L O, Kegelmeyer W P (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357CrossRefMATH

Ebrahimi N, Trabelsi A, Islam M S, Hamou-Lhadj A, Khanmohammadi K (2019) An hmm-based approach for automatic detection and classification of duplicate bug reports. Inf Softw Technol 113:98–109. https://doi.org/10.1016/j.infsof.2019.05.007. http://www.sciencedirect.com/science/article/pii/S095058491930117XCrossRef

Guo P J, Zimmermann T, Nagappan N, Murphy B (2011) “Not my bug!” and other reasons for software bug report reassignments. In: Proceedings of the ACM 2011 conference on computer supported cooperative work, CSCW ’11. https://doi.org/10.1145/1958824.1958887. Association for Computing Machinery, New York, pp 395–404

Habayeb M, Murtaza S S, Miranskyy A, Bener A B (2018) On the use of hidden markov model to predict the time to fix bugs. IEEE Trans Softw Eng 44(12):1224–1244. https://doi.org/10.1109/TSE.2017.2757480CrossRef

Hamill M, Goseva-Popstojanova K (2017) Analyzing and predicting effort associated with finding and fixing software faults. Inf Softw Technol 87:1–18. https://doi.org/10.1016/j.infsof.2017.01.002. http://www.sciencedirect.com/science/article/pii/S0950584917300290CrossRef

Hauge O, Ayala C, Conradi R (2010) Adoption of open source software in software-intensive organizations—a systematic literature review. Inf Softw Technol 52(11):1133–1154. https://doi.org/10.1016/j.infsof.2010.05.008CrossRef

Hensman J, Fusi N, Lawrence N D (2013) Gaussian processes for big data. In: Proceedings of the twenty-ninth conference on uncertainty in artificial intelligence, pp 282–290

Herzig K, Just S, Zeller A (2013) It’s not a bug, it’s a feature: how misclassification impacts bug prediction. In: Proceedings of the 2013 international conference on software engineering, ICSE ’13. IEEE Press, pp 392–401

Hooimeijer P, Weimer W (2007) Modeling bug report quality. In: Proceedings of the twenty-second IEEE/ACM international conference on automated software engineering, ASE ’07. https://doi.org/10.1145/1321631.1321639. Association for Computing Machinery, New York, pp 34–43

Hu H, Zhang H, Xuan J, Sun W (2014) Effective bug triage based on historical bug-fix information. In: 2014 IEEE 25th international symposium on software reliability engineering. https://doi.org/10.1109/ISSRE.2014.17, pp 122–132

Kim S, Whitehead E J (2006) How long did it take to fix bugs?. In: Proceedings of the 2006 international workshop on mining software repositories, MSR ’06. https://doi.org/10.1145/1137983.1138027. Association for Computing Machinery, New York, pp 173–174

Lazar A, Ritchey S, Sharif B (2014) Improving the accuracy of duplicate bug report detection using textual similarity measures. In: Proceedings of the 11th working conference on mining software repositories, MSR 2014, p 308–311. https://doi.org/10.1145/2597073.2597088. Association for Computing Machinery, New York

Lenarduzzi V, Taibi D, Tosi D, Lavazza L, Morasca S (2020) Open source software evaluation, selection, and adoption: a systematic literature review. In: 2020 46th Euromicro conference on software engineering and advanced applications (SEAA). https://doi.org/10.1109/SEAA51224.2020.00076, pp 437–444

Raja U (2013) All complaints are not created equal: text analysis of open source software defect reports. Empir Softw Eng 18(1):117–138. https://doi.org/10.1007/s10664-012-9197-9CrossRef

Serrano N, Ciordia I (2005) Bugzilla, itracker, and other bug trackers. IEEE Softw 22(2):11–13. https://doi.org/10.1109/MS.2005.32CrossRef

Shokripour R, Anvik J, Kasirun Z M, Zamani S (2015) A time-based approach to automatic bug report assignment. J Syst Softw 102(C):109–122. https://doi.org/10.1016/j.jss.2014.12.049CrossRef

Thung F (2016) Automatic prediction of bug fixing effort measured by code churn size. In: Proceedings of the 5th international workshop on software mining, SoftwareMining 2016. https://doi.org/10.1145/2975961.2975964. Association for Computing Machinery, New York, pp 18–23

Tian Y, Lo D, Xia X, Sun C (2015) Automated prediction of bug report priority using multi-factor analysis. Empirical Softw Engg 20(5):1354–1383. https://doi.org/10.1007/s10664-014-9331-yCrossRef

Vieira R, da Silva A, Rocha L, Gomes JAP (2019) From reports to bug-fix commits: a 10 years dataset of bug-fixing activity from 55 apache’s open source projects. In: Proceedings of the fifteenth international conference on predictive models and data analytics in software engineering, PROMISE’19. https://doi.org/10.1145/3345629.3345639. http://doi.acm.org/10.1145/3345629.3345639. ACM, New York, pp 80–89

Weiss C, Premraj R, Zimmermann T, Zeller A (2007) How long will it take to fix this bug?. In: Fourth international workshop on mining software repositories (MSR’07:ICSE workshops 2007). https://doi.org/10.1109/MSR.2007.13, pp 1–1

Wohlin C, Runeson P, Hst M, Ohlsson M C, Regnell B, Wessln A (2012) Experimentation in software engineering. Springer Publishing Company, Incorporated

Zhang H, Gong L, Versteeg S (2013) Predicting bug-fixing time: an empirical study of commercial software projects. In: 2013 35th International conference on software engineering (ICSE). https://doi.org/10.1109/ICSE.2013.6606654, pp 1042–1051

Zhang X, Yao L, Huang C, Sheng Q Z, Wang X (2017) Intent recognition in smart living through deep recurrent neural networks. In: Liu D, Xie S, Li Y, Zhao D, El-Alfy E S M (eds) Neural information processing. Springer International Publishing, Cham, pp 748–758

Zhang X, Chen X, Yao L, Ge C, Dong M (2019) Deep neural network hyperparameter optimization with orthogonal array tuning. In: Gedeon T, Wong K W, Lee M (eds) Neural information processing. Springer International Publishing, Cham, pp 287–295

Zimmermann T, Premraj R, Bettenburg N, Just S, Schroter A, Weiss C (2010) What makes a good bug report? IEEE Trans Softw Eng 36 (5):618–643. https://doi.org/10.1109/TSE.2010.63CrossRef

Title: The role of bug report evolution in reliable fixing estimation
Authors: Renan G. Vieira
César Lincoln C. Mattos
Lincoln S. Rocha
João Paulo P. Gomes
Matheus Paixão
Publication date: 01-12-2022
Publisher: Springer US
Published in: Empirical Software Engineering / Issue 7/2022
Print ISSN: 1382-3256
Electronic ISSN: 1573-7616
DOI: https://doi.org/10.1007/s10664-022-10213-7

Springer Professional

Abstract

Context

Objective

Method

Results

Conclusions

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 7/2022

Intensifying the search-based optimization of product line architectures with crossover operators

Basic block coverage for search-based unit testing and crash reproduction

Sources of software development task friction

Developers’ perception matters: machine learning to detect developer-sensitive smells

Test smells 20 years later: detectability, validity, and reliability

An empirical study of Android behavioural code smells detection

Premium Partner