Use of relative code churn measures to predict system defect density

ABSTRACT

Software systems evolve over time due to changes in requirements, optimization of code, fixes for security and reliability bugs etc. Code churn, which measures the changes made to a component over a period of time, quantifies the extent of this change. We present a technique for early prediction of system defect density using a set of relative code churn measures that relate the amount of churn to other variables such as component size and the temporal extent of churn.Using statistical regression models, we show that while absolute measures of code churn are poor predictors of defect density, our set of relative measures of code churn is highly predictive of defect density. A case study performed on Windows Server 2003 indicates the validity of the relative code churn measures as early indicators of system defect density. Furthermore, our code churn metric suite is able to discriminate between fault and not fault-prone binaries with an accuracy of 89.0 percent.

REFERENCES

• 1: ANSI/IEEE, "IEEE Standard Glossary of Software Engineering Terminology, Standard 729," 1983.
• 2: Victor R. Basili , Forrest Shull , Filippo Lanubile, Building Knowledge through Families of Experiments, IEEE Transactions on Software Engineering, v.25 n.4, p.456-473, July 1999 [doi>10.1109/32.799939]
• 3: Barry W. Boehm, Software Engineering Economics, Prentice Hall PTR, Upper Saddle River, NJ, 1981
• 4: Brace, N., Kemp, R., Snelgar, R., SPSS for Psychologists: Palgrave Macmillan, 2003.
• 5: Fernando Brito e. Abreu , Walcelio Melo, Evaluating the Impact of Object-Oriented Design on Software Quality, Proceedings of the 3rd International Symposium on Software Metrics: From Measurement to Empirical Results, p.90, March 25-26, 1996
• 6: Giovanni Denaro , Mauro Pezzè, An empirical evaluation of fault-proneness models, Proceedings of the 24th International Conference on Software Engineering, May 19-25, 2002, Orlando, Florida [doi>10.1145/581339.581371]
• 7: Norman E. Fenton , Niclas Ohlsson, Quantitative Analysis of Faults and Failures in a Complex Software System, IEEE Transactions on Software Engineering, v.26 n.8, p.797-814, August 2000 [doi>10.1109/32.879815]
• 8: Fenton, N. E., Pfleeger, S.L., Software Metrics. Boston, MA: International Thompson Publishing, 1997.
• 9: Todd L. Graves , Alan F. Karr , J. S. Marron , Harvey Siy, Predicting Fault Incidence Using Software Change History, IEEE Transactions on Software Engineering, v.26 n.7, p.653-661, July 2000 [doi>10.1109/32.859533]
• 10: Jackson, E. J., A User's Guide to Principal Components: John Wiley & Sons, Inc., 1991.
• 11: Kaiser, H. F., "An Index of Factorial Simplicity," Psychometrika, Vol. 39, No., pp. 31--36, 1974.
• 12: Karunanithi, N., "A Neural Network approach for Software Reliability Growth Modeling in the Presence of Code Churn," Proceedings of International Symposium on Software Reliability Engineering, 1993, pp. 310--317.
• 13: T. M. Khoshgoftaar , E. B. Allen , N. Goel , A. Nandi , J. McMullan, Detection of software modules with high debug code churn in a very large legacy system, Proceedings of the The Seventh International Symposium on Software Reliability Engineering (ISSRE '96), p.364, October 30-November 02, 1996
• 14: Khoshgoftaar, T. M., Allen, E.B., Kalaichelvan, K.S., Goel, N., Hudepohl, J.P., Mayrand, J., "Detection of fault-prone program modules in a very large telecommunications system," Proceedings of International Symposium Software Reliability Engineering, 1995, pp. 24--33.
• 15: Taghi M. Khoshgoftaar , Robert M. Szabo, Improving Code Churn Predictions During the System Test and Maintenance Phases, Proceedings of the International Conference on Software Maintenance, p.58-67, September 01, 1994
• 16: D. G. Kleinbaum , L. L. Kupper , K. E. Muller, Applied regression analysis and other multivariable methods, PWS Publishing Co., Boston, MA, 1988
• 17: J. C. Munson , S. G. Elbaum, Code Churn: A Measure for Estimating the Impact of Code Change, Proceedings of the International Conference on Software Maintenance, p.24, March 16-19, 1998
• 18: John C. Munson , Taghi M. Khoshgoftaar, The Detection of Fault-Prone Programs, IEEE Transactions on Software Engineering, v.18 n.5, p.423-433, May 1992 [doi>10.1109/32.135775]
• 19: Ohlsson, M. C., von Mayrhauser, A., McGuire, B., Wohlin, C., "Code Decay Analysis of Legacy Software through Successive Releases," Proceedings of IEEE Aerospace Conference, 1999, pp. 69--81.
• 20: Thomas J. Ostrand , Elaine J. Weyuker , Robert M. Bell, Where the bugs are, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement
          Subjects: Version control

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Performance measures; Process metrics; Product metrics

Keywords:
defect density, fault-proneness, multiple regression, principal component analysis, relative code churn