Emanuel Giger
Marco D'Ambros
Martin Pinzger
Harald C. Gall
ABSTRACT
Researchers proposed a wide range of approaches to build effective bug prediction models that take into account multiple aspects of the software development process. Such models achieved good prediction performance, guiding developers towards those parts of their system where a large share of bugs can be expected. However, most of those approaches predict bugs on file-level. This often leaves developers with a considerable amount of effort to examine all methods of a file until a bug is located. This particular problem is reinforced by the fact that large files are typically predicted as the most bug-prone. In this paper, we present bug prediction models at the level of individual methods rather than at file-level. This increases the granularity of the prediction and thus reduces manual inspection efforts for developers. The models are based on change metrics and source code metrics that are typically used in bug prediction. Our experiments---performed on 21 Java open-source (sub-)systems---show that our prediction models reach a precision and recall of 84% and 88%, respectively. Furthermore, the results indicate that change metrics significantly outperform source code metrics.
- G. Antoniol, K. Ayari, M. D. Penta, F. Khomh, and Y.-G. Gueheneuc. Is it a bug or an enhancement? a text-based approach to classify change requests. In Proc. Conf. of the center for advanced studies on collaborative research: meeting of minds, pages 304--318, 2008. Google Scholar
Digital Library
- E. Arisholm and L. Briand. Predicting fault-prone components in a java legacy system. In Proc. Int'l Symp. on Empir. Softw. Eng., pages 8--17, 2006. Google ScholarDigital Library
- V. Basili, L. Briand, and W. Melo. A validation of object-oriented design metrics as quality indicators. IEEE Trans. Softw. Eng., 22:751--761, October 1996. Google ScholarDigital Library
- A. Bernstein, J. Ekanayake, and M. Pinzger. Improving defect prediction using temporal features and non linear models. In Proc. Int'l Workshop on Principles of Softw. Evolution, pages 11--18, 2007. Google ScholarDigital Library
- C. Bird, A. Bachmann, E. Aune, J. Duffy, A. Bernstein, V. Filkov, and P. Devanbu. Fair and balanced?: bias in bug-fix datasets. In Proc. Joint Eur. Softw. Eng. Conf. and Symp. on the Found. of Softw. Eng., pages 121--130, 2009. Google ScholarDigital Library
- C. Bird, N. Nagappan, P. Devanbu, H. Gall, and B. Murphy. Does distributed development affect software quality? an empirical case study of windows vista. In Proc. Int'l Conf. on Softw. Eng., pages 518--528, 2009. Google ScholarDigital Library
- C. Bird, N. Nagappan, B. Murphy, H. Gall, and P. Devanbu. Don't Touch My Code! Examining the Effects of Ownership on Software Quality. In Proc. Joint Eur Softw. Eng. Conf. and Symp. on the Found. of Softw. Eng., pages 4--14, 2011. Google ScholarDigital Library
- S. R. Chidamber and C. F. Kemerer. A metrics suite for object oriented design. IEEE Trans. Softw. Eng., 20(6):476--493, June 1994. Google ScholarDigital Library
- M. D'Ambros, M. Lanza, and R. Robbes. Evaluating defect prediction approaches: a benchmark and an extensive comparison. Empir. Softw. Eng., pages 1--47, 2011. Google ScholarDigital Library
- G. Denaro and M. Pezze. An empirical evaluation of fault-proneness models. In Proc. Int'l Conf. on Softw. Eng., pages 241--251, 2002. Google ScholarDigital Library
- K. E. Emam, S. Benlarbi, N. Goel, and S. Rai. The confounding effect of class size on the validity of object-oriented metrics. IEEE Trans. on Softw. Eng., 27(7):630--650, July 2001. Google ScholarDigital Library
- B. Fluri, M. Wursch, M. Pinzger, and H. C. Gall. Change Distilling: Tree Differencing for Fine-Grained Source Code Change Extraction. IEEE Trans. on Softw. Eng., 33(11):725--743, November 2007. Google ScholarDigital Library
- B. Fluri, J. Zuberbuehler, and H. C. Gall. Recommending method invocation context changes. In Proc. Int'l Workshop on Recomm. Syst. for Softw. Eng., pages 1--5, 2008. Google ScholarDigital Library
- H. C. Gall, B. Fluri, and M. Pinzger. Change analysis with evolizer and changedistiller. IEEE Software, 26(1):26--33, January/February 2009. Google ScholarDigital Library
- G. Ghezzi and H. Gall. Sofas: A lightweight architecture for software analysis as a service. In Proc. Working Conf. on Softw. Architecture, pages 93--102, 2011. Google ScholarDigital Library
- E. Giger, M. Pinzger, and H. C. Gall. Comparing fine-grained source code changes and code churn for bug prediction. In Proc. Int'l Workshop on Mining Softw. Repos., pages 83--92, 2011. Google ScholarDigital Library
- T. Graves, A. Karr, J. Marron, and H. Siy. Predicting fault incidence using software change history. IEEE Trans. Softw. Eng., 26:653--661, July 2000. Google ScholarDigital Library
- T. Gyimothy, R. Ferenc, and I. Siket. Empirical validation of object-oriented metrics on open source software for fault prediction. IEEE Trans. Softw. Eng., 31:897--910, 2005. Google ScholarDigital Library
- A. Hassan. Predicting faults using the complexity of code changes. In Proc. Int'l Conf. on Softw. Eng., pages 78--88, 2009. Google ScholarDigital Library
- Y. Kamei, S. Matsumoto, A. Monden, K. Matsumoto, B. Adams, and A. Hassan. Revisiting common bug prediction findings using effort-aware models. In Proc. Int'l Conf. on Softw. Maint., pages 1--10, 2010. Google ScholarDigital Library
- D. Kawrykow and M. P. Robillard. Non-essential changes in version histories. In Proc. Int'l Conf. on Softw. Eng., pages 351--360, 2011. Google ScholarDigital Library
- S. Kim, H. Zhang, R. Wu, and L. Gong. Dealing with noise in defect prediction. In Proc. Int'l Conf. on Softw. Eng., pages 481--490, 2011. Google ScholarDigital Library
- S. Kim, T. Zimmermann, J. Whitehead, and A. Zeller. Predicting faults from cached history. In Proc. Int'l Conf. on Softw. Eng., pages 489--498, 2007. Google ScholarDigital Library
- P. Knab, M. Pinzger, and A. Bernstein. Predicting defect densities in source code files with decision tree learners. In Proc. Int'l Workshop on Mining Softw. Repos., pages 119--125, 2006. Google ScholarDigital Library
- T. Lee, J. Nam, D. Han, S. Kim, and H. P. In. Micro interaction metrics for defect prediction. In Proc. Joint Eur. Softw. Eng. Conf. and Symp. on the Found. of Softw. Eng., pages 311--321, 2011. Google ScholarDigital Library
- S. Lessmann, B. Baesens, C. M. Swantje, and Pietsch. Benchmarking classification models for software defect prediction: A proposed framework and novel findings. IEEE Trans. on Softw. Eng., 34:485--496, July 2008. Google ScholarDigital Library
- T. Menzies, J. Greenwald, and A. Frank. Data mining static code attributes to learn defect predictors. IEEE Trans. on Softw. Eng., 33:2--13, January 2007. Google ScholarDigital Library
- T. Menzies, Z. Milton, B. Turhan, B. Cukic, Y. Jiang, and A. Bener. Defect prediction from static code features: current results, limitations, new approaches. Automated Softw. Eng., 17(4):375--407, 2010. Google ScholarDigital Library
- I. Mierswa, M. Wurst, R. Klinkenberg, M. Scholz, and T. Euler. Yale: Rapid prototyping for complex data mining tasks. In Proc. Int'l Conf. on Knowl. Discovery and Data Mining, pages 935--940, 2006. Google ScholarDigital Library
- R. Moser, W. Pedrycz, and G. Succi. A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction. In Proc. Int'l Conf. on Softw. Eng., pages 181--190, 2008. Google ScholarDigital Library
- N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In Proc. Int'l Conf. on Softw. Eng., pages 284--292, 2005. Google ScholarDigital Library
- N. Nagappan, T. Ball, and A. Zeller. Mining metrics to predict component failures. In Proc. Int'l Conf. on Softw. Eng., pages 452--461, 2006. Google ScholarDigital Library
- N. Nagappan, B. Murphy, and V. Basili. The influence of organizational structure on software quality: an empirical case study. In Proc. Int'l Conf. on Softw. Eng., pages 521--530, 2008. Google ScholarDigital Library
- N. Nagappan, A. Zeller, T. Zimmermann, K. Herzig, and B. Murphy. Change bursts as defect predictors. In Proc. Int'l Symp. on Softw. Reliability Eng., 2010. Google ScholarDigital Library
- T. Nguyen, B. Adams, and A. Hassan. Studying the impact of dependency network measures on software quality. In Int'l Conf. on Softw. Maint., pages 1--10, 2010. Google ScholarDigital Library
- T. Ostrand, E. Weyuker, and R. Bell. Predicting the location and number of faults in large software systems. IEEE Trans. Softw. Eng., 31(4):340--355, 2005. Google ScholarDigital Library
- M. Pinzger, N. Nagappan, and B. Murphy. Can developer-module networks predict failures? In Proc. Symp. on the Found. of Softw. Eng., pages 2--12, 2008. Google ScholarDigital Library
- D. Posnett, V. Filkov, and P. Devanbu. Ecological inference in empirical software engineering. In Proc. Int'l Conf. on Automated Softw. Eng., pages 362--371, 2011. Google ScholarDigital Library
- R. Purushothaman and D. Perry. Toward understanding the rhetoric of small source code changes. IEEE Trans. Softw. Eng., 31(6):511--526, June 2005. Google ScholarDigital Library
- F. Rahman and P. Devanbu. Ownership, experience and defects: a fine-grained study of authorship. In Proc. Int'l Conf. on Softw. Eng., pages 491--500, 2011. Google ScholarDigital Library
- E. Shihab, M. Jiang, W. Ibrahim, B. Adams, and A. Hassan. Understanding the impact of code and process metrics on post-release defects: a case study on the eclipse project. In Proc. Int'l Symp. on Empir. Softw. Eng. and Meas., pages 1--10, 2010. Google ScholarDigital Library
- E. Shihab, A. Mockus, Y. Kamei, B. Adams, and A. Hassan. High-impact defects: a study of breakage and surprise defects. In Proc. Joint Eur. Softw. Eng. Conf. and Symp. on the Found. of Softw. Eng., pages 300--310, 2011. Google ScholarDigital Library
- J. Sliwerski, T. Zimmermann, and A. Zeller. When do changes induce fixes? In Proc. Int'l Workshop on Mining Softw. Repos., pages 1--5, 2005. Google ScholarDigital Library
- R. Subramanyam and M. Krishnan. Empirical analysis of ck metrics for object-oriented design complexity: Implications for software defects. IEEE Trans. Softw. Eng., 29(4):297--310, 2003. Google ScholarDigital Library
- R. Wu, H. Zhang, S. Kim, and S.-C. Cheung. Relink: Recovering links between bugs and changes. In Proc. Joint Eur. Softw. Eng. Conf. and Symp. on the Found. of Softw. Eng., pages 15--25, 2011. Google ScholarDigital Library
- H. Zhang. An investigation of the relationships between lines of code and defects. In Proc. Int'l Conf. on Softw. Maint., pages 274--283, 2009.Google ScholarCross Ref
- T. Zimmermann and N. Nagappan. Predicting defects using network analysis on dependency graphs. In Proc. Int'l Conf. on Softw. Eng., pages 531--540, 2008. Google ScholarDigital Library
- T. Zimmermann, R. Premraj, and A. Zeller. Predicting defects for eclipse. In Proc. Int'l Workshop on Predictor Models in Softw. Eng., pages 9--15, 2007. Google ScholarDigital Library
Index Terms
- Method-level bug prediction
Recommendations
An exploratory study of bug prediction at the method level
Abstract Context:During the past decades, researchers have proposed numerous studies to predict bugs at different granularity levels, such as the file level, package level, module level, etc. However, the prediction models at the ...
Method-level Bug Prediction: Problems and Promises
Fixing software bugs can be colossally expensive, especially if they are discovered in the later phases of the software development life cycle. As such, bug prediction has been a classic problem for the research community. As of now, the Google Scholar ...
Empirical Evaluation of Hunk Metrics as Bug Predictors
IWSM '09 /Mensura '09: Proceedings of the International Conferences on Software Process and Product MeasurementReducing the number of bugs is a crucial issue during software development and maintenance. Software process and product metrics are good indicators of software complexity. These metrics have been used to build bug predictor models to help developers ...
Comments