Coping with an open bug repository

Reviewer: Phillip A. Laplante

With respect to the need for flexibility in humans, science fiction writer Robert Heinlein wrote, "specialization is for insects" [1]. Upon reading this paper, I think you will conclude that specialization is also for bug reporting. This conclusion follows from the study reported in this work, in which the data from the bug repositories for two open source projects are characterized, and an automated triage tool proposed. Indeed, this paper is recommended reading for those involved in open source software. The authors selected two well-known projects, Eclipse V3.0 and Firefox V1.0, for their study. This choice was appropriate, as there are a large number of bugs reported daily in these repositories (around 1,000 per year), and a variety of developers available to resolve the bugs. These large projects surely helped to provide amplification of the kinds of problems found in most other bug repositories. Among the problems the authors identified in managing the bug repository were: the large number of bugs reported daily, and the varied quality of the reports, leading to a great deal of wasted time and effort; a great deal of duplication in the bug reports; and difficulty in assigning the right person to resolve the bug report. Together, these problems face the bug triager (a person who must assign the bug report to an appropriate developer for investigation and resolution). These challenges are compounded by the fact that most bugs are resolved as, cryptically, "INVALID," "DUPLICATE," "WORKSFORME," or "WONTFIX," so that when a "new" bug is reported, the triager must possess a great deal of working knowledge to determine if a new bug has indeed been found. The authors found that, even after a bug had been resolved, determining the developer who resolved a report was nontrivial. While knowing who resolved a particular bug is clearly important for a human triager, this information is particularly vital to automating bug triage. In fact, the authors propose that bug triaging difficulties can be resolved via an artificial intelligence approach. They note, for example, that using spam filtering-like rules can help triage automatically. But, more powerfully, the authors are investigating a machine learning approach using a support vector machine (SVM), which involves separating a multi-dimensional feature space by discriminating hyperplanes. The authors' bug reporting system apparently uses a statistical database of knowledge from past bug reports and the SVM to help cut down on duplication, through better classification of bug types. The system also uses domain analysis techniques in order to assign bugs to an appropriate person for resolution. This system can be incorporated into an existing integrated development environment (IDE) such as Eclipse. Unfortunately, the paper provides little other information on this automated system, and, in fact, it is unclear if the system has even been built. This brings me back to my original statement about specialization: what is not explicitly stated in this work, but what is clearly implied, is that a better, more fine-grained bug classification system is needed to reduce bug report duplication, and assist the machine learning algorithms in assignment. Therefore, I hope the authors eventually develop and report a fully working system, replete with a rich entomological taxonomy. Online Computing Reviews Service