Improving vulnerability discovery models

ABSTRACT

Security researchers are applying software reliability models to vulnerability data, in an attempt to model the vulnerability discovery process. I show that most current work on these vulnerability discovery models (VDMs) is theoretically unsound. I propose a standard set of definitions relevant to measuring characteristics of vulnerabilities and their discovery process. I then describe the theoretical requirements of VDMs and highlight the shortcomings of existing work, particularly the assumption that vulnerability discovery is an independent process.

REFERENCES

• 1: AIAA/ANSI. Recommended Practice: Software Reliability. ANSI, 1993. R-013-1992.
• 2: O. H. Alhazmi , Y. K. Malaiya, Modeling the Vulnerability Discovery Process, Proceedings of the 16th IEEE International Symposium on Software Reliability Engineering, p.129-138, November 08-11, 2005 [doi>10.1109/ISSRE.2005.30]
• 3: O. H. Alhazmi and Y. K. Malaiya. Quantitative vulnerability assessment of systems software. In Proceedings of the IEEE Reliability and Maintainability Symposium (RAMS'05), pages 615--620, Alexandria, VA, USA, 2005.
• 4: Omar H. Alhazmi , Yashwant K. Malaiya, Measuring and Enhancing Prediction Capabilities of Vulnerability Discovery Models for Apache and IIS HTTP Servers, Proceedings of the 17th International Symposium on Software Reliability Engineering, p.343-352, November 07-10, 2006 [doi>10.1109/ISSRE.2006.26]
• 5: O. H. Alhazmi , Y. K. Malaiya, Prediction capabilities of vulnerability discovery models, Proceedings of the RAMS '06. Annual Reliability and Maintainability Symposium, 2006., p.86-91, June 14-16, 2006 [doi>10.1109/RAMS.2006.1677355]
• 6: William A. Arbaugh , William L. Fithen , John McHugh, Windows of Vulnerability: A Case Study Analysis, Computer, v.33 n.12, p.52-59, December 2000 [doi>10.1109/2.889093]
• 7: Algirdas Avizienis , Jean-Claude Laprie , Brian Randell , Carl Landwehr, Basic Concepts and Taxonomy of Dependable and Secure Computing, IEEE Transactions on Dependable and Secure Computing, v.1 n.1, p.11-33, January 2004 [doi>10.1109/TDSC.2004.2]
• 8: S. Brocklehurst, B. Littlewood, T. Olovsson, and E. Jonsson. On measurement of operational security. Technical Report 160, Predictably Dependable Computing Systems, Apr. 1994.
• 9: Computer Science and Telecommunications Board. Computers at Risk: Safe Computing In the Information Age. National Academy Press, Washington, DC, 2001.
• 10: R. Gopalakrishna and E. H. Spafford. A trend analysis of vulnerabilities. Technical Report 2005-05, CERIAS, Purdue University, May 2005.
• 11: R. Gopalakrishna, E. H. Spafford, and J. Vitek. Vulnerability likelihood: A probabilistic approach to software assurance. Technical Report 2005-06, CERIAS, Purdue University, 2005. 2005-06.
• 12: K. Goseva-Popstonjanova and K. S. Trivedi. Failure correlation in software reliability models. Technical Report 00/04, Center for Advanced Computing and Communication (CACC), 2000.
• 13: IEEE. IEEE standard glossary of software engineering terminology, Sept. 1990.
• 14: Barbara Kitchenham , Steve Linkman, Validation, Verification, and Testing: Diversity Rules, IEEE Software, v.15 n.4, p.46-49, July 1998 [doi>10.1109/52.687944]
• 15: Ivan Victor Krsul , Eugene H. Spafford, Software vulnerability analysis, Purdue University, West Lafayette, IN, 1998
• 16: Michael R. Lyu, Introduction, Handbook of software reliability and system reliability, McGraw-Hill, Inc., Hightstown, NJ, 1996
• 17: John D. Musa , Anthony Iannino , Kazuhira Okumoto, Software reliability: measurement, prediction, application, McGraw-Hill, Inc., New York, NY, 1987
• 18: A. Ozment. The likelihood of vulnerability rediscovery and the social utility of vulnerability hunting. In Workshop on the Economics of Information Security (WEIS), June 2005. Cambridge, MA, USA.
• 19: A. Ozment. Software security growth modeling: Examining vulnerabilities with reliability growth models. In D. Gollmann, F. Massacci, and A. Yautsiukhin, editors, Quality Of Protection: Security Measurements and Metrics, Milan, Italy, 2006. Springer.
• 20: A. Ozment. Vulnerability Discovery & Software Security. PhD thesis, University of Cambridge, 2007.
• 21: Andy Ozment , Stuart E. Schechter, Milk or wine: does software security improve with age?, Proceedings of the 15th conference on USENIX Security Symposium, p.7-7, July 31-August 04, 2006, Vancouver, B.C., Canada
• 22: Eric Rescorla, Is Finding Security Holes a Good Idea?, IEEE Security and Privacy, v.3 n.1, p.14-19, January 2005 [doi>10.1109/MSP.2005.17]
• 23: S.-W. Woo, O. H. Alhazmi, and Y. K. Malaiya. An analysis of the vulnerability discovery process in web browsers, Nov. 2006.
• 24: Sung-Whan Woo , Omar H. Alhazmi , Yashwant K. Malaiya, Assessing Vulnerabilities in Apache and IIS HTTP Servers, Proceedings of the 2nd IEEE International Symposium on Dependable, Autonomic and Secure Computing, p.103-110, September 29-October 01, 2006 [doi>10.1109/DASC.2006.21]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Product metrics

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Reliability and life testing

General Terms:
Measurement, Reliability, Security

Keywords:
measuring software security, measuring vulnerabilities, security metrics, vulnerability discovery model