Mining metrics to predict component failures

ABSTRACT

What is it that makes software fail? In an empirical study of the post-release defect history of five Microsoft software systems, we found that failure-prone software entities are statistically correlated with code complexity measures. However, there is no single set of complexity metrics that could act as a universally best defect predictor. Using principal component analysis on the code metrics, we built regression models that accurately predict the likelihood of post-release defects for new entities. The approach can easily be generalized to arbitrary projects; in particular, predictors obtained from one project can also be significant for new, similar projects.

REFERENCES

• 1: E. N. Adams, "Optimizing Preventive Service of Software Products", IBM Journal of Research and Development, 28(1), pp. 2--14, 1984.
• 2: Victor R. Basili , Forrest Shull , Filippo Lanubile, Building Knowledge through Families of Experiments, IEEE Transactions on Software Engineering, v.25 n.4, p.456-473, July 1999 [doi>10.1109/32.799939]
• 3: Victor R. Basili , Lionel C. Briand , Walcélio L. Melo, A Validation of Object-Oriented Design Metrics as Quality Indicators, IEEE Transactions on Software Engineering, v.22 n.10, p.751-761, October 1996 [doi>10.1109/32.544352]
• 4: Aaron B. Binkley , Stephen R. Schach, Validation of the coupling dependency metric as a predictor of run-time failures and maintenance measures, Proceedings of the 20th international conference on Software engineering, p.452-455, April 19-25, 1998, Kyoto, Japan
• 5: S. Biyani , P. Santhanam, Exploring Defect Data from Development and Customer Usage on Software Modules over Multiple Releases, Proceedings of the The Ninth International Symposium on Software Reliability Engineering, p.316, November 04-07, 1998
• 6: Barry Boehm , Richard Turner, Using Risk to Balance Agile and Plan-Driven Methods, Computer, v.36 n.6, p.57-66, June 2003 [doi>10.1109/MC.2003.1204376]
• 7: Fernando Brito e. Abreu , Walcelio Melo, Evaluating the Impact of Object-Oriented Design on Software Quality, Proceedings of the 3rd International Symposium on Software Metrics: From Measurement to Empirical Results, p.90, March 25-26, 1996
• 8: S. R. Chidamber , C. F. Kemerer, A Metrics Suite for Object Oriented Design, IEEE Transactions on Software Engineering, v.20 n.6, p.476-493, June 1994 [doi>10.1109/32.295895]
• 9: Davor Čubranić , Gail C. Murphy, Hipikat: recommending pertinent software development artifacts, Proceedings of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon
• 10: Norman E. Fenton , Martin Neil, A Critique of Software Defect Prediction Models, IEEE Transactions on Software Engineering, v.25 n.5, p.675-689, September 1999 [doi>10.1109/32.815326]
• 11: Norman E. Fenton , Shari Lawrence Pfleeger, Software Metrics: A Rigorous and Practical Approach, PWS Publishing Co., Boston, MA, 1998
• 12: Michael Fischer , Martin Pinzger , Harald Gall, Populating a Release History Database from Version Control and Bug Tracking Systems, Proceedings of the International Conference on Software Maintenance, p.23, September 22-26, 2003
• 13: John P. Hudepohl , Stephen J. Aud , Taghi M. Khoshgoftaar , Edward B. Allen , Jean Mayrand, Emerald: Software Metrics and Models on the Desktop, IEEE Software, v.13 n.5, p.56-60, September 1996 [doi>10.1109/52.536459]
• 14: E. J. Jackson, A User's Guide to Principal Components. Hoboken, NJ: John Wiley & Sons Inc., 2003.
• 15: Lucas Layman , Laurie Williams , Lynn Cunningham, Exploring Extreme Programming in Context: An Industrial Case Study, Proceedings of the Agile Development Conference (ADC'04), p.32-41, June 22-26, 2004
• 16: Audris Mockus , Ping Zhang , Paul Luo Li, Predictors of customer perceived software quality, Proceedings of the 27th international conference on Software engineering, p.225-233, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062506]
• 17: Nachiappan Nagappan , Thomas Ball, Use of relative code churn measures to predict system defect density, Proceedings of the 27th international conference on Software engineering, p.284-292, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062514]
• 18: Niclas Ohlsson , Hans Alberg, Predicting Fault-Prone Software Modules in Telephone Switches, IEEE Transactions on Software Engineering, v.22 n.12, p.886-894, December 1996 [doi>10.1109/32.553637]
• 19: Thomas J. Ostrand , Elaine J. Weyuker , Robert M. Bell, Predicting the Location and Number of Faults in Large Software Systems, IEEE Transactions on Software Engineering, v.31 n.4, p.340-355, April 2005 [doi>10.1109/TSE.2005.49]
• 20: Jacek Śliwerski , Thomas Zimmermann , Andreas Zeller, When do changes induce fixes?, Proceedings of the 2005 international workshop on Mining software repositories, p.1-5, May 17-17, 2005, St. Louis, Missouri
• 21: Ramanath Subramanyam , M. S. Krishnan, Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects, IEEE Transactions on Software Engineering, v.29 n.4, p.297-310, April 2003 [doi>10.1109/TSE.2003.1191795]
• 22: Thomas Zimmermann , Peter Weissgerber , Stephan Diehl , Andreas Zeller, Mining Version Histories to Guide Software Changes, IEEE Transactions on Software Engineering, v.31 n.6, p.429-445, June 2005 [doi>10.1109/TSE.2005.72]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement
          Subjects: Version control

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Product metrics; Process metrics; Performance measures
      D.2.9 Management
          Subjects: Software quality assurance (SQA)

General Terms:
Design, Measurement, Reliability

Keywords:
bug database, complexity metrics, empirical study, principal component analysis, regression model