Static analysis tools as early indicators of pre-release defect density

ABSTRACT

During software development it is helpful to obtain early estimates of the defect density of software components. Such estimates identify fault-prone areas of code requiring further testing. We present an empirical approach for the early prediction of pre-release defect density based on the defects found using static analysis tools. The defects identified by two different static analysis tools are used to fit and predict the actual pre-release defect density for Windows Server 2003. We show that there exists a strong positive correlation between the static analysis defect density and the pre-release defect density determined by testing. Further, the predicted pre-release defect density and the actual pre-release defect density are strongly correlated at a high degree of statistical significance. Discriminant analysis shows that the results of static analysis tools can be used to separate high and low quality components with an overall classification rate of 82.91%.

REFERENCES

• 1: Victor R. Basili , Lionel C. Briand , Walcélio L. Melo, A Validation of Object-Oriented Design Metrics as Quality Indicators, IEEE Transactions on Software Engineering, v.22 n.10, p.751-761, October 1996 [doi>10.1109/32.544352]
• 2: Brace, N., Kemp, R., Snelgar, R., SPSS for Psychologists: Palgrave Macmillan, 2003.
• 3: Lionel C. Briand , William M. Thomas , Christopher J. Hetmanski, Modeling and managing risk early in software development, Proceedings of the 15th international conference on Software Engineering, p.55-65, May 17-21, 1993, Baltimore, Maryland, United States
• 4: Lionel C. Briand , Jürgen Wüst , John W. Daly , D. Victor Porter, Exploring the relationship between design measures and software quality in object-oriented systems, Journal of Systems and Software, v.51 n.3, p.245-273, May 2000 [doi>10.1016/S0164-1212(99)00102-8]
• 5: Lionel C. Briand , Jürgen Wüst , Stefan V. Ikonomovski , Hakim Lounis, Investigating quality factors in object-oriented designs: an industrial case study, Proceedings of the 21st international conference on Software engineering, p.345-354, May 16-22, 1999, Los Angeles, California, United States [doi>10.1145/302405.302654]
• 6: William R. Bush , Jonathan D. Pincus , David J. Sielaff, A static analyzer for finding dynamic programming errors, Software—Practice & Experience, v.30 n.7, p.775-802, June 2000 [doi>10.1002/(SICI)1097-024X(200006)30:7<775::AID-SPE309>3.0.CO;2-H]
• 7: S. R. Chidamber , C. F. Kemerer, A Metrics Suite for Object Oriented Design, IEEE Transactions on Software Engineering, v.20 n.6, p.476-493, June 1994 [doi>10.1109/32.295895]
• 8: Giovanni Denaro , Sandro Morasca , Mauro Pezzè, Deriving models of software fault-proneness, Proceedings of the 14th international conference on Software engineering and knowledge engineering, July 15-19, 2002, Ischia, Italy [doi>10.1145/568760.568824]
• 9: Giovanni Denaro , Mauro Pezzè, An empirical evaluation of fault-proneness models, Proceedings of the 24th International Conference on Software Engineering, May 19-25, 2002, Orlando, Florida [doi>10.1145/581339.581371]
• 10: Engler, D., Chelf, B., Chou, A., Hallem, S., "Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions," Proceedings of OSDI 2000, 2000.
• 11: David Evans , John Guttag , James Horning , Yang Meng Tan, LCLint: a tool for using specifications to check code, Proceedings of the 2nd ACM SIGSOFT symposium on Foundations of software engineering, p.87-96, December 06-09, 1994, New Orleans, Louisiana, United States
• 12: Fenton, N. E., Pfleeger, S.L., Software Metrics. Boston, MA: International Thompson Publishing, 1997.Fenton, N. E., Pfleeger, S.L., Software Metrics. Boston, MA: International Thompson Publishing, 1997.
• 13: Khoshgoftaar, T. M., Allen, E.B., Deng,J., "Using Regression Trees to Classify Fault-Prone Software Modules," IEEE Transactions on Reliability, Vol. 51, No. 4, pp. 455--462, 2002.
• 14: T. M. Khoshgoftaar , E. B. Allen , N. Goel , A. Nandi , J. McMullan, Detection of software modules with high debug code churn in a very large legacy system, Proceedings of the The Seventh International Symposium on Software Reliability Engineering (ISSRE '96), p.364, October 30-November 02, 1996
• 15: Khoshgoftaar, T. M., Allen, E.B., Hudepohl, J.P., Aud, S.J., "Application of neural networks to software quality modeling of a very large telecommunications system," IEEE Transactions on Neural Networks, Vol. 8, No. 4, pp. 902--909, 1997.
• 16: Khoshgoftaar, T. M., Allen, E.B., Jones,W.D., Hudepohl, J.P., "Classification-Tree Models of Software Quality Over Multiple Releases," IEEE Transactions on Reliability, Vol. 49, No. 1, pp. 4--11, 2000.
• 17: Khoshgoftaar, T. M., Allen, E.B., Kalaichelvan, K.S., Goel, N., Hudepohl, J.P., Mayrand, J., "Detection of fault-prone program modules in a very large telecommunications system," Proceedings of International Symposium Software Reliability Engineering, 1995, pp. 24--33.
• 18: Taghi M. Khoshgoftaar , John C. Munson , David L. Lanning, A Comparative Study of Predictive Models for Program Changes During System Testing and Maintenance, Proceedings of the Conference on Software Maintenance, p.72-79, September 01, 1993
• 19: Taghi M. Khoshgoftaar , Naeem Seliya, Fault Prediction Modeling for Software Quality Estimation: Comparing Commonly Used Techniques, Empirical Software Engineering, v.8 n.3, p.255-283, September 2003 [doi>10.1023/A:1024424811345]
• 20: Larus, J. R., Ball, T., Das, M., DeLine, R., Fahndrich, M., Pincus, J., Rajamani, S.K., Venkatapathy, R., "Righting Software," in IEEE Software, vol. 21, 2004, pp. 92--100.
• 21: John C. Munson , Taghi M. Khoshgoftaar, The Detection of Fault-Prone Programs, IEEE Transactions on Software Engineering, v.18 n.5, p.423-433, May 1992 [doi>10.1109/32.135775]
• 22: J. Munson , Y. M. Khoshgoftaar, Regression modelling of software quality: empirical investigation, Journal of Electronic Materials, v.19 n.6, p.106-114, Jun. 1990
• 23: Nachiappan Nagappan , Laurie Williams , John Hudepohl , Will Snipes , Mladen Vouk, Preliminary Results On Using Static Analysis Tools For Software Inspection, Proceedings of the 15th International Symposium on Software Reliability Engineering (ISSRE'04), p.429-439, November 02-05, 2004 [doi>10.1109/ISSRE.2004.30]
• 24: Norman F. Schneidewind, Methodology for Validating Software Metrics, IEEE Transactions on Software Engineering, v.18 n.5, p.410-422, May 1992 [doi>10.1109/32.135774]
• 25: Ramanath Subramanyam , M. S. Krishnan, Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects, IEEE Transactions on Software Engineering, v.29 n.4, p.297-310, April 2003 [doi>10.1109/TSE.2003.1191795]
• 26: Mei-Huei Tang , Ming-Hung Kao , Mei-Hwa Chen, An Empirical Study on Object-Oriented Metrics, Proceedings of the 6th International Symposium on Software Metrics, p.242, November 04-06, 1999

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Symbolic execution

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)
      D.2.8 Metrics
          Subjects: Process metrics; Performance measures; Product metrics
      D.2.9 Management
          Subjects: Software quality assurance (SQA)

Keywords:
defect density, fault-proneness, static analysis tools, statistical methods