Predicting vulnerable software components

ABSTRACT

Where do most vulnerabilities occur in software? Our Vulture tool automatically mines existing vulnerability databases and version archives to map past vulnerabilities to components. The resulting ranking of the most vulnerable components is a perfect base for further investigations on what makes components vulnerable.

In an investigation of the Mozilla vulnerability history, we surprisingly found that components that had a single vulnerability in the past were generally not likely to have further vulnerabilities. However, components that had similar imports or function calls were likely to be vulnerable.

Based on this observation, we were able to extend Vulture by a simple predictor that correctly predicts about half of all vulnerable components, and about two thirds of all predictions are correct. This allows developers and project managers to focus their their efforts where it is needed most: "We should look at nsXPInstallManager because it is likely to contain yet unknown vulnerabilities.".

REFERENCES

• 1: Rakesh Agrawal , Ramakrishnan Srikant, Fast Algorithms for Mining Association Rules in Large Databases, Proceedings of the 20th International Conference on Very Large Data Bases, p.487-499, September 12-15, 1994
• 2: Omar Alhazmi, Yashwant Malaiya, and Indrajit Ray. Security Vulnerabilities in Software Systems: A Quantitative Perspective, volume 3645/2005 of Lecture Notes in Computer Science, pages 281--294. Springer Verlag, Berlin, Heidelberg, August 2005.
• 3: Hao Chen, Drew Dean, and David Wagner. Model checking one million lines of C code. In Proc. 11th Annual Network and Distributed System Security Symposium (NDSS), pages 171--185, February 2004.
• 4: Hao Chen , David Wagner, MOPS: an infrastructure for examining security properties of software, Proceedings of the 9th ACM conference on Computer and communications security, November 18-22, 2002, Washington, DC, USA [doi>10.1145/586110.586142]
• 5: Crispin Cowan. Apparmor linux application security. http://www.novell.com/linux/security/apparmor/, January 2007.
• 6: Crispin Cowan , Calton Pu , Dave Maier , Heather Hintony , Jonathan Walpole , Peat Bakke , Steve Beattie , Aaron Grier , Perry Wagle , Qian Zhang, StackGuard: automatic adaptive detection and prevention of buffer-overflow attacks, Proceedings of the 7th conference on USENIX Security Symposium, p.5-5, January 26-29, 1998, San Antonio, Texas
• 7: Davor Cubranic , Gail C. Murphy , Janice Singer , Kellogg S. Booth, Hipikat: A Project Memory for Software Development, IEEE Transactions on Software Engineering, v.31 n.6, p.446-465, June 2005 [doi>10.1109/TSE.2005.71]
• 8: Dan DaCosta , Christopher Dahn , Spiros Mancoridis , Vassilis Prevelakis, Characterizing the 'Security Vulnerability Likelihood' of Software Functions, Proceedings of the International Conference on Software Maintenance, p.266, September 22-26, 2003
• 9: Evgenia Dimitriadou, Kurt Hornik, Friedrich Leisch, David Meyer, and Andreas Weingessel. e1071: Misc Functions Department of Statistics (e1071), TU Wien, 2006. R package version 1.5-13.
• 10: Michael Fischer , Martin Pinzger , Harald Gall, Populating a Release History Database from Version Control and Bug Tracking Systems, Proceedings of the International Conference on Software Maintenance, p.23, September 22-26, 2003
• 11: Pascal Fradet , Ronan Caugne , Daniel Le Métayer, Static Detection of Pointer Errors: An Axiomatisation and a Checking Algorithm, Proceedings of the 6th European Symposium on Programming Languages and Systems, p.125-140, April 22-24, 1996
• 12: Vinod Ganapathy , Somesh Jha , David Chandler , David Melski , David Vitek, Buffer overrun detection using linear programming and static analysis, Proceedings of the 10th ACM conference on Computer and communications security, October 27-30, 2003, Washington D.C., USA [doi>10.1145/948109.948155]
• 13: Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics. Springer Verlag, 2001.
• 14: Nenad Jovanovic , Christopher Kruegel , Engin Kirda, Pixy: A Static Analysis Tool for Detecting Web Application Vulnerabilities (Short Paper), Proceedings of the 2006 IEEE Symposium on Security and Privacy, p.258-263, May 21-24, 2006 [doi>10.1109/SP.2006.29]
• 15: Roger Koenker and Pin Ng. SparseM: Sparse Linear Algebra. R package version 0.73.
• 16: David Larochelle , David Evans, Statically detecting likely buffer overflow vulnerabilities, Proceedings of the 10th conference on USENIX Security Symposium, p.14-14, August 13-17, 2001, Washington, D.C.
• 17: Zhenmin Li , Lin Tan , Xuanhui Wang , Shan Lu , Yuanyuan Zhou , Chengxiang Zhai, Have things changed now?: an empirical study of bug characteristics in modern open source software, Proceedings of the 1st workshop on Architectural and system support for improving software dependability, p.25-33, October 21-21, 2006, San Jose, California [doi>10.1145/1181309.1181314]
• 18: Heikki Mannila, Hannu Toivonen, and A. Inkeri Verkamo. Efficient algorithms for discovering association rules. In Knowledge Discovery in Databases: Papers from the 1994 AAAI Workshop, pages 181--192, 1994.
• 19: Barton P. Miller , Louis Fredriksen , Bryan So, An empirical study of the reliability of UNIX utilities, Communications of the ACM, v.33 n.12, p.32-44, Dec. 1990 [doi>10.1145/96267.96279]
• 20: Keith W. Miller , Larry J. Morell , Robert E. Noonan , Stephen K. Park , David M. Nicol , Branson W. Murrill , Jeffrey M. Voas, Estimating the Probability of Failure When Testing Reveals No Failures, IEEE Transactions on Software Engineering, v.18 n.1, p.33-43, January 1992 [doi>10.1109/32.120314]
• 21: Nachiappan Nagappan, Thomas Ball, and Andreas Zeller. Mining metrics to predict component failures. In Proc. 29th Int'l Conf. on Software Engineering. ACM Press, November 2005.
• 22: National Security Agency. Security-enhanced linux. http://www.nsa.gov/selinux/, January 2007.
• 23: Andy Ozment , Stuart E. Schechter, Milk or wine: does software security improve with age?, Proceedings of the 15th conference on USENIX Security Symposium, p.7-7, July 31-August 04, 2006, Vancouver, B.C., Canada
• 24: R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2006. ISBN 3-900051-07-0.
• 25: Eric Rescorla, Is Finding Security Holes a Good Idea?, IEEE Security and Privacy, v.3 n.1, p.14-19, January 2005 [doi>10.1109/MSP.2005.17]
• 26: Radu Rugina , Martin Rinard, Symbolic bounds analysis of pointers, array indices, and accessed memory regions, Proceedings of the ACM SIGPLAN 2000 conference on Programming language design and implementation, p.182-195, June 18-21, 2000, Vancouver, British Columbia, Canada
• 27: Bruce Schneier. Do we really need a security industry? Wired, May 2007. http://www.wired.com/politics/security/commentary/securitymatters/2007/%05/securitymatters_0503.
• 28: Berhard Scholz , Johann Blieberger , Thomas Fahringer, Symbolic pointer analysis for detecting memory leaks, Proceedings of the 2000 ACM SIGPLAN workshop on Partial evaluation and semantics-based program manipulation, p.104-113, January 22-23, 2000, Boston, Massachusetts, United States
• 29: Adrian Schröter , Thomas Zimmermann , Andreas Zeller, Predicting component failures at design time, Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering, September 21-22, 2006, Rio de Janeiro, Brazil [doi>10.1145/1159733.1159739]
• 30: Jacek Śliwerski, Thomas Zimmermann, and Andreas Zeller. When do changes induce fixes? In Proc. Second Int'l Workshop on Mining Software Repositories, pages 24--28, May 2005.
• 31: Torsten Robschink , Gregor Snelting, Efficient path conditions in dependence graphs, Proceedings of the 24th International Conference on Software Engineering, May 19-25, 2002, Orlando, Florida [doi>10.1145/581339.581398]
• 32: The Mozilla Foundation. Bugzilla. http://www.bugzilla.org, January 2007.
• 33: The Mozilla Foundation. Mozilla foundation security advisories. http://www.mozilla.org/projects/security/known-vulnerabilities.html, January 2007.
• 34: The Mozilla Foundation. Mozilla project website. http://www.mozilla.org/, January 2007.
• 35: Chris Tofts and Brian Monahan. Towards an analytic model of security flaws. Technical Report 2004-224, HP Trusted Systems Laboratory, Bristol, UK, December 2004.
• 36: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 37: John Viega , J. T. Bloch , Tadayoshi Kohno , Gary McGraw, Token-based scanning of source code for security problems, ACM Transactions on Information and System Security (TISSEC), v.5 n.3, p.238-261, August 2002 [doi>10.1145/545186.545188]
• 38: Jeffrey M. Voas , Gary McGraw, Software fault injection: inoculating programs against errors, John Wiley & Sons, Inc., New York, NY, 1997
• 39: Jian Yin, Chunqiang Tang, Xiaolan Zhang, and Michael McIntosh. On estimating the security risks of composite software services. In Proc. PASSWORD Workshop, June 2006.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Statistical methods

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)
  D.4 OPERATING SYSTEMS
      D.4.6 Security and Protection
          Subjects: Invasive software (e.g., viruses, worms, Trojan horses)

General Terms:
Experimentation, Measurement, Security

Keywords:
prediction, software security