Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Software developers spend a significant portion of their resources handling user-submitted bug reports. For software that is widely deployed, the number of bug reports typically outstrips the resources available to triage them. As a result, some reports may be dealt with too slowly or not at all.

We present a descriptive model of bug report quality based on a statistical analysis of surface features of over 27,000 publicly available bug reports for the Mozilla Firefox project. The model predicts whether a bug report is triaged within a given amount of time. Our analysis of this model has implications for bug reporting systems and suggests features that should be emphasized when composing bug reports.

We evaluate our model empirically based on its hypothetical performance as an automatic filter of incoming bug reports. Our results show that our model performs significantly better than chance in terms of precision and recall. In addition, we show that our modelcan reduce the overall cost of software maintenance in a setting where the average cost of addressing a bug report is more than 2% of the cost of ignoring an important bug report.

REFERENCES

• 1: G. Antoniol, M. D. Penta, H. Gall, and M. Pinzger. Towards the integration of versioning systems, bug reports and source code meta-models. Electr. Notes Theor. Comput. Sci., 127(3):87--99, 2005.
• 2: John Anvik , Lyndon Hiew , Gail C. Murphy, Coping with an open bug repository, Proceedings of the 2005 OOPSLA workshop on Eclipse technology eXchange, p.35-39, October 16-17, 2005, San Diego, California [doi>10.1145/1117696.1117704]
• 3: John Anvik , Lyndon Hiew , Gail C. Murphy, Who should fix this bug?, Proceedings of the 28th international conference on Software engineering, May 20-28, 2006, Shanghai, China [doi>10.1145/1134285.1134336]
• 4: Ron Bekkerman , Ran El-Yaniv , Naftali Tishby , Yoad Winter, Distributional word clusters vs. words for text categorization, The Journal of Machine Learning Research, 3, 3/1/2003
• 5: Barry Boehm , Victor R. Basili, Software Defect Reduction Top 10 List, Computer, v.34 n.1, p.135-137, January 2001 [doi>10.1109/2.962984]
• 6: Marco D'Ambros , Michele Lanza, Software Bugs and Evolution: A Visual Approach to Uncover Their Relationship, Proceedings of the Conference on Software Maintenance and Reengineering, p.229-238, March 22-24, 2006
• 7: Michael Fischer , Martin Pinzger , Harald Gall, Analyzing and Relating Bug Report Data for Feature Tracking, Proceedings of the 10th Working Conference on Reverse Engineering, p.90, November 13-17, 2003
• 8: Sunghun Kim , E. James Whitehead, Jr., How long did it take to fix bugs?, Proceedings of the 2006 international workshop on Mining software repositories, May 22-23, 2006, Shanghai, China [doi>10.1145/1137983.1138027]
• 9: R. Kohavi. A study of cross-validation and bootstrap for accuracy estimation and model selection. International Joint Conference on Artificial Intelligence, 14(2):1137--1145, 1995.
• 10: Ben Liblit , Alex Aiken , Alice X. Zheng , Michael I. Jordan, Bug isolation via remote program sampling, Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, June 09-11, 2003, San Diego, California, USA
• 11: Douglas R. McCallum , James L. Peterson, Computer-based readability indexes, Proceedings of the ACM '82 conference, p.44-48, January 1982 [doi>10.1145/800174.809754]
• 12: C. V. Ramamoorthy , Wei-Tek Tsai, Advances in Software Engineering, Computer, v.29 n.10, p.47-58, October 1996 [doi>10.1109/2.539720]
• 13: E. S. Raymond. The cathedral and the bazaar: musings on linux and open source by an accidental revolutionary. Inf. Res., 6(4), 2001.
• 14: C. R. Reis and R. P. de Mattos Fortes. An overview of the software engineering process and tools in the Mozilla project. In Proceedings of the Open Source software Development Workshop, pages 155--175, 2002.
• 15: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 16: Westley Weimer, Patches as better bug reports, Proceedings of the 5th international conference on Generative programming and component engineering, October 22-26, 2006, Portland, Oregon, USA [doi>10.1145/1173706.1173734]
• 17: Cathrin Weiss , Rahul Premraj , Thomas Zimmermann , Andreas Zeller, How Long Will It Take to Fix This Bug?, Proceedings of the Fourth International Workshop on Mining Software Repositories, p.1, May 20-26, 2007 [doi>10.1109/MSR.2007.13]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Debugging aids

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.9 Management
          Subjects: Time estimation

General Terms:
Economics, Experimentation, Human Factors, Management, Measurement

Keywords:
bug report triage, information retrieval, issue tracking, statistical model