Is it a bug or an enhancement?: a text-based approach to classify change requests

ABSTRACT

Bug tracking systems are valuable assets for managing maintenance activities. They are widely used in open-source projects as well as in the software industry. They collect many different kinds of issues: requests for defect fixing, enhancements, refactoring/restructuring activities and organizational issues. These different kinds of issues are simply labeled as "bug" for lack of a better classification support or of knowledge about the possible kinds.

This paper investigates whether the text of the issues posted in bug tracking systems is enough to classify them into corrective maintenance and other kinds of activities.

We show that alternating decision trees, naive Bayes classifiers, and logistic regression can be used to accurately distinguish bugs from other kinds of issues. Results from empirical studies performed on issues for Mozilla, Eclipse, and JBoss indicate that issues can be classified with between 77% and 82% of correct decisions.

REFERENCES

• 1: Ethem Alpaydin, Introduction to Machine Learning (Adaptive Computation and Machine Learning), The MIT Press, 2004
• 2: K. Ayari , P. Meshkinfam , G. Antoniol , M. Di Penta, Threats on building models from CVS and Bugzilla repositories: the Mozilla case study, Proceedings of the 2007 conference of the center for advanced studies on Collaborative research, October 22-25, 2007, Richmond Hill, Ontario, Canada [doi>10.1145/1321211.1321234]
• 3: V. Basili, G. Caldiera, and D. H. Rombach. The Goal Question Metric Paradigm Encyclopedia of Software Engineering. John Wiley and Sons, 1994.
• 4: Lionel C. Briand , Sandro Morasca , Victor R. Basili, Measuring and Assessing Maintainability at the End of High Level Design, Proceedings of the Conference on Software Maintenance, p.88-97, September 01, 1993
• 5: S. E. Robertson C. J. van Rijsbergen and M. F. Porter. New models in probabilistic information retrieval. London: British Library, Research and Development Report, no. 5587, 1980.
• 6: Rumelhart D. E., Hinton G. E., and Williams R. J. Learning representations by back-propagating errors. Nature, 323:533--536, 1986.
• 7: Norman E. Fenton , Martin Neil, A Critique of Software Defect Prediction Models, IEEE Transactions on Software Engineering, v.25 n.5, p.675-689, September 1999 [doi>10.1109/32.815326]
• 8: Michael Fischer , Martin Pinzger , Harald Gall, Populating a Release History Database from Version Control and Bug Tracking Systems, Proceedings of the International Conference on Software Maintenance, p.23, September 22-26, 2003
• 9: W. B. Frakes and R. Baeza-Yates. Information Retrieval: Data Structures and Algorithms. Prentice-Hall, Englewood Cliffs, NJ, 1992.
• 10: Harald Gall , Karin Hajek , Mehdi Jazayeri, Detection of Logical Coupling Based on Product Release History, Proceedings of the International Conference on Software Maintenance, p.190, March 16-19, 1998
• 11: Daniel M. German, An empirical study of fine-grained software modifications, Empirical Software Engineering, v.11 n.3, p.369-393, September 2006 [doi>10.1007/s10664-006-9004-6]
• 12: Tibor Gyimothy , Rudolf Ferenc , Istvan Siket, Empirical Validation of Object-Oriented Metrics on Open Source Software for Fault Prediction, IEEE Transactions on Software Engineering, v.31 n.10, p.897-910, October 2005 [doi>10.1109/TSE.2005.112]
• 13: Norio Kurishima , Hiroshi Oikawa , Jun-ichi Nakamura , Kazuo Amari , Masanobu Fujioka, Quantitative Analysis of Errors in Telecommunications Software, Proceedings of the Conference on Software Maintenance, p.190-198, September 01, 1993
• 14: Thomas M. Mitchell, Machine Learning, McGraw-Hill Higher Education, 1997
• 15: J. Ross Quinlan, C4.5: programs for machine learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1993
• 16: J. O. Rawlings, S. G. Pandula, and D. A. Dickey. Applied Regression Analysis a Research Tool. Springer Texts in Statistics. New York: Springer-Verlag, second edition edition, 1998.
• 17: Jacek Śliwerski , Thomas Zimmermann , Andreas Zeller, When do changes induce fixes?, Proceedings of the 2005 international workshop on Mining software repositories, p.1-5, May 17-17, 2005, St. Louis, Missouri
• 18: M. Stone. Cross-validatory choice and assesment of statistical predictions (with discussion). Journal of the Royal Statistical Society B, 36:111--147, 1974.
• 19: Marek Vokac, Defect Frequency and Design Patterns: An Empirical Study of Industrial Code, IEEE Transactions on Software Engineering, v.30 n.12, p.904-917, December 2004 [doi>10.1109/TSE.2004.99]
• 20: Xiaoyin Wang , Lu Zhang , Tao Xie , John Anvik , Jiasu Sun, An approach to detecting duplicate bug reports using natural language and execution information, Proceedings of the 30th international conference on Software engineering, May 10-18, 2008, Leipzig, Germany [doi>10.1145/1368088.1368151]
• 21: Peter Weißgerber , Stephan Diehl, Are refactorings less error-prone than other changes?, Proceedings of the 2006 international workshop on Mining software repositories, May 22-23, 2006, Shanghai, China [doi>10.1145/1137983.1138011]
• 22: Ian H. Witten , Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005
• 23: R. K. Yin. Case Study Research: Design and Methods - Third Edition. SAGE Publications, London, 2002.
• 24: Annie T. T. Ying , Gail C. Murphy , Raymond Ng , Mark C. Chu-Carroll, Predicting Source Code Changes by Mining Change History, IEEE Transactions on Software Engineering, v.30 n.9, p.574-586, September 2004 [doi>10.1109/TSE.2004.52]
• 25: Thomas Zimmermann , Peter Weisgerber , Stephan Diehl , Andreas Zeller, Mining Version Histories to Guide Software Changes, Proceedings of the 26th International Conference on Software Engineering, p.563-572, May 23-28, 2004

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Text processing

General Terms:
Algorithms, Experimentation, Performance