Predicting failures with developer networks and social network analysis

ABSTRACT

Software fails and fixing it is expensive. Research in failure prediction has been highly successful at modeling software failures. Few models, however, consider the key cause of failures in software: people. Understanding the structure of developer collaboration could explain a lot about the reliability of the final product. We examine this collaboration structure with the developer network derived from code churn information that can predict failures at the file level. We conducted a case study involving a mature Nortel networking product of over three million lines of code. Failure prediction models were developed using test and post-release failure data from two releases, then validated against a subsequent release. One model's prioritization revealed 58% of the failures in 20% of the files compared with the optimal prioritization that would have found 61% in 20% of the files, indicating that a significant correlation exists between file-based developer network metrics and failures.

REFERENCES

• 1: Network Analysis: Methodological Foundations. Berlin: Springer, 2005.
• 2: Allen, T. J., Managing the Flow of Technology: MIT Press, 1977.
• 3: Erik Arisholm , Lionel C. Briand, Predicting fault-prone components in a java legacy system, Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering, September 21-22, 2006, Rio de Janeiro, Brazil [doi>10.1145/1159733.1159738]
• 4: Erik Arisholm , Lionel C. Briand , Magnus Fuglerud, Data Mining Techniques for Building Fault-proneness Models in Telecom Java Software, Proceedings of the The 18th IEEE International Symposium on Software Reliability, p.215-224, November 05-09, 2007 [doi>10.1109/ISSRE.2007.8]
• 5: Barabasi, A.-L. and Albert, R., "Emergence of scaling in random networks," Science, vol. 286, no. 5439, pp. 509--512, 1999.
• 6: Barabasi, A.-L. and Oltvai, Z. N., "Network Biology: Understanding the Cell's Functional Organization," Nature Reviews Genetics, vol. 5, no. 2, pp. 101--113, 2004.
• 7: Yoshua Bengio , Yves Grandvalet, No Unbiased Estimator of the Variance of K-Fold Cross-Validation, The Journal of Machine Learning Research, 5, p.1089-1105, 12/1/2004
• 8: Abraham Bernstein , Jayalath Ekanayake , Martin Pinzger, Improving defect prediction using temporal features and non linear models, Ninth international workshop on Principles of software evolution: in conjunction with the 6th ESEC/FSE joint meeting, September 03-04, 2007, Dubrovnik, Croatia [doi>10.1145/1294948.1294953]
• 9: Christian Bird , Alex Gourley , Prem Devanbu , Michael Gertz , Anand Swaminathan, Mining email social networks in Postgres, Proceedings of the 2006 international workshop on Mining software repositories, May 22-23, 2006, Shanghai, China [doi>10.1145/1137983.1138033]
• 10: Barry W. Boehm, Software Engineering Economics, Prentice Hall PTR, Upper Saddle River, NJ, 1981
• 11: Giovanni Denaro , Mauro Pezzè, An empirical evaluation of fault-proneness models, Proceedings of the 24th International Conference on Software Engineering, May 19-25, 2002, Orlando, Florida [doi>10.1145/581339.581371]
• 12: Gao, K. and Khoshgoftaar, T. M., "A Comprehensive Empirical Study of Count Models for Software Fault Prediction," Reliability, IEEE Transactions on, vol. 56, no. 2, pp. 223--236, June, 2007.
• 13: Girvan, M. and Newman, M. E. J., "Community Structure in Social and Biological Networks," The Proceedings of the National Academy of Sciences, vol. 99, no. 12, pp. 7821--7826, 2001.
• 14: Gonzales-Barahona, J. M., Lopez-Fernandez, L., and Robles, G., "Applying Social Network Analysis to the Information in CVS Repositories," in 2005 International Workshop on Mining Software Repositories, 2004.
• 15: Shih-Kun Huang , Kang-min Liu, Mining version histories to verify the learning process of Legitimate Peripheral Participants, Proceedings of the 2005 international workshop on Mining software repositories, p.1-5, May 17-17, 2005, St. Louis, Missouri
• 16: John P. Hudepohl , Stephen J. Aud , Taghi M. Khoshgoftaar , Edward B. Allen , Jean Mayrand, Emerald: Software Metrics and Models on the Desktop, IEEE Software, v.13 n.5, p.56-60, September 1996 [doi>10.1109/52.536459]
• 17: Lave, J. and Wenger, E., Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge University Press, 1991.
• 18: Mockus, A. and Weiss, D. M., "Predicting Risk of Software Changes," Bell Labs Technical Journal, vol. 5, pp. 169--180, 2002.
• 19: Audris Mockus , David M. Weiss , Ping Zhang, Understanding and predicting effort in software projects, Proceedings of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon
• 20: Nachiappan Nagappan , Thomas Ball, Static analysis tools as early indicators of pre-release defect density, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062558]
• 21: Nachiappan Nagappan , Thomas Ball, Use of relative code churn measures to predict system defect density, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062514]
• 22: Nachiappan Nagappan , Thomas Ball , Andreas Zeller, Mining metrics to predict component failures, Proceedings of the 28th international conference on Software engineering, May 20-28, 2006, Shanghai, China [doi>10.1145/1134285.1134349]
• 23: Masao Ohira , Naoki Ohsugi , Tetsuya Ohoka , Ken-ichi Matsumoto, Accelerating cross-project knowledge collaboration using collaborative filtering and social networks, Proceedings of the 2005 international workshop on Mining software repositories, p.1-5, May 17-17, 2005, St. Louis, Missouri
• 24: Thomas J. Ostrand , Elaine J. Weyuker , Robert M. Bell, Locating where faults will be, Proceedings of the 2005 conference on Diversity in computing, October 19-22, 2005, Albuquerque, New Mexico, USA [doi>10.1145/1095242.1095262]
• 25: Elaine J. Weyuker , Thomas J. Ostrand , Robert M. Bell, Using Developer Information as a Factor for Fault Prediction, Proceedings of the Third International Workshop on Predictor Models in Software Engineering, p.8, May 20-26, 2007 [doi>10.1109/PROMISE.2007.14]
• 26: Liguo Yu , Srini Ramaswamy, Mining CVS Repositories to Understand Open-Source Project Developer Roles, Proceedings of the Fourth International Workshop on Mining Software Repositories, p.8, May 20-26, 2007 [doi>10.1109/MSR.2007.19]
• 27: Thomas Zimmermann , Nachiappan Nagappan, Predicting defects using network analysis on dependency graphs, Proceedings of the 30th international conference on Software engineering, May 10-18, 2008, Leipzig, Germany [doi>10.1145/1368088.1368161]
• 28: Thomas Zimmermann , Rahul Premraj , Andreas Zeller, Predicting Defects for Eclipse, Proceedings of the Third International Workshop on Predictor Models in Software Engineering, p.9, May 20-26, 2007 [doi>10.1109/PROMISE.2007.10]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Process metrics

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Product metrics

General Terms:
Human Factors, Reliability, Verification

Keywords:
developer network, failure prediction, logistic regression, negative binomial regression, social network analysis