Predicting buggy changes inside an integrated development environment

ABSTRACT

We present a tool that predicts whether the software under development inside an IDE has a bug. An IDE plugin performs this prediction, using the Change Classification technique to classify source code changes as buggy or clean during the editing session. Change Classification uses Support Vector Machines (SVM), a machine learning classifier algorithm, to classify changes to projects mined from their configuration management repository. This technique, besides being language independent and relatively accurate, can (a) classify a change immediately upon its completion and (b) use features extracted solely from the change delta (added, deleted) and the source code to predict buggy changes. Thus, integrating change classification within an IDE can predict potential bugs in the software as the developer edits the source code, ideally reducing the amount of time spent on fixing bugs later. To this end, we have developed a Change Classification plugin for Eclipse based on client-server architecture, described in this paper.

REFERENCES

• 1: C. Artho. Jlint - Find bugs in Java programs, 2006.
• 2: Jennifer Bevan , E. James Whitehead, Jr. , Sunghun Kim , Michael Godfrey, Facilitating software evolution research with kenyon, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal
• 3: James F. Bowring , James M. Rehg , Mary Jean Harrold, Active learning for automatic classification of software behavior, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 4: Yuriy Brun , Michael D. Ernst, Finding Latent Code Errors via Machine Learning over Program Executions, Proceedings of the 26th International Conference on Software Engineering, p.480-490, May 23-28, 2004
• 5: William R. Bush , Jonathan D. Pincus , David J. Sielaff, A static analyzer for finding dynamic programming errors, Software—Practice & Experience, v.30 n.7, p.775-802, June 2000 [doi>10.1002/(SICI)1097-024X(200006)30:7<775::AID-SPE309>3.0.CO;2-H]
• 6: Gerardo Canfora , Luigi Cerulo, Jimpa: An Eclipse Plug-in for Impact Analysis, Proceedings of the Conference on Software Maintenance and Reengineering, p.341-342, March 22-24, 2006
• 7: T. Copeland. PMD Applied: Centennial Books. 2005.
• 8: Valentin Dallmeier , Christian Lindig , Andreas Zeller, Lightweight bug localization with AMPLE, Proceedings of the sixth international symposium on Automated analysis-driven debugging, p.99-104, September 19-21, 2005, Monterey, California, USA [doi>10.1145/1085130.1085143]
• 9: William Dickinson , David Leon , Andy Podgurski, Finding failures by cluster analysis of execution profiles, Proceedings of the 23rd International Conference on Software Engineering, p.339-348, May 12-19, 2001, Toronto, Ontario, Canada
• 10: Michael D. Ernst , Jake Cockrell , William G. Griswold , David Notkin, Dynamically Discovering Likely Program Invariants to Support Program Evolution, IEEE Transactions on Software Engineering, v.27 n.2, p.99-123, February 2001 [doi>10.1109/32.908957]
• 11: Cormac Flanagan , K. Rustan M. Leino , Mark Lillibridge , Greg Nelson , James B. Saxe , Raymie Stata, Extended static checking for Java, Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation, June 17-19, 2002, Berlin, Germany
• 12: Tibor Gyimothy , Rudolf Ferenc , Istvan Siket, Empirical Validation of Object-Oriented Metrics on Open Source Software for Fault Prediction, IEEE Transactions on Software Engineering, v.31 n.10, p.897-910, October 2005 [doi>10.1109/TSE.2005.112]
• 13: Murali Haran , Alan Karr , Alessandro Orso , Adam Porter , Ashish Sanil, Applying classification techniques to remotely-collected program execution data, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal
• 14: Ahmed E. Hassan , Richard C. Holt, The Top Ten List: Dynamic Fault Prediction, Proceedings of the 21st IEEE International Conference on Software Maintenance, p.263-272, September 25-30, 2005 [doi>10.1109/ICSM.2005.91]
• 15: David Hovemeyer , William Pugh, Finding bugs is easy, Companion to the 19th annual ACM SIGPLAN conference on Object-oriented programming systems, languages, and applications, October 24-28, 2004, Vancouver, BC, CANADA [doi>10.1145/1028664.1028717]
• 16: Thorsten Joachims, Text Categorization with Suport Vector Machines: Learning with Many Relevant Features, Proceedings of the 10th European Conference on Machine Learning, p.137-142, April 21-23, 1998
• 17: Taghi M. Khoshgoftaar , Edward B. Allen, Ordering Fault-Prone Software Modules, Software Quality Control, v.11 n.1, p.19-37, May 2003 [doi>10.1023/A:1023632027907]
• 18: Sunghun Kim , Kai Pan , E. E. James Whitehead, Jr., Memories of bug fixes, Proceedings of the 14th ACM SIGSOFT international symposium on Foundations of software engineering, November 05-11, 2006, Portland, Oregon, USA [doi>10.1145/1181775.1181781]
• 19: S. Kim, E. J. Whitehead, Jr., and Y. Zhang. Classifying software changes: Clean or buggy. IEEE Trans. Softw. Engin., in review. Manuscript available at http://www.cs.ucsc.edu/~ejw/papers/cc.pdf.
• 20: V. Livshits. Turning Eclipse against itself: Finding bugs in eclipse code using lightweight static analysis. Eclipsecon '05 Research Exchange, March 2005.
• 21: A. Mockus and D. M. Weiss. Predicting risk of software changes. Bell Labs Tech. J., 5(2):169--180, April-June 2000.
• 22: Thomas J. Ostrand , Elaine J. Weyuker , Robert M. Bell, Predicting the Location and Number of Faults in Large Software Systems, IEEE Transactions on Software Engineering, v.31 n.4, p.340-355, April 2005 [doi>10.1109/TSE.2005.49]
• 23: Kai Pan , Sunghun Kim , E. James Whitehead, Jr., Bug Classification Using Program Slicing Metrics, Proceedings of the Sixth IEEE International Workshop on Source Code Analysis and Manipulation, p.31-42, September 27-29, 2006 [doi>10.1109/SCAM.2006.6]
• 24: Xiaoxia Ren , Fenil Shah , Frank Tip , Barbara G. Ryder , Ophelia Chesley, Chianti: a tool for change impact analysis of java programs, Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, October 24-28, 2004, Vancouver, BC, Canada
• 25: Sam Scott , Stan Matwin, Feature Engineering for Text Classification, Proceedings of the Sixteenth International Conference on Machine Learning, p.379-388, June 27-30, 1999
• 26: Jacek Śliwerski , Thomas Zimmermann , Andreas Zeller, HATARI: raising risk awareness, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal
• 27: Ian H. Witten , Eibe Frank, Data Mining: Practical Machine Learning Tools and Techniques, Second Edition (Morgan Kaufmann Series in Data Management Systems), Morgan Kaufmann Publishers Inc., San Francisco, CA, 2005
• 28: Thomas Zimmermann , Peter Weisgerber , Stephan Diehl , Andreas Zeller, Mining Version Histories to Guide Software Changes, Proceedings of the 26th International Conference on Software Engineering, p.563-572, May 23-28, 2004

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement
          Subjects: Version control

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.6 Programming Environments
          Subjects: Programmer workbench; Integrated environments

General Terms:
Algorithms, Experimentation, Measurement

Keywords:
bug prediction, integrated development environments