Prioritizing software security fortification throughcode-level metrics

ABSTRACT

Limited resources preclude software engineers from finding and fixing all vulnerabilities in a software system. We create predictive models to identify which components are likely to have the most security risk. Software engineers can use these models to make measurement-based risk management decisions and to prioritize software security fortification efforts, such as redesign and additional inspection and testing. We mined and analyzed data from a large commercial telecommunications software system containing over one million lines of code that had been deployed to the field for two years. Using recursive partitioning, we built attack-prone prediction models with the following code-level metrics: static analysis tool alert density, code churn, and count of source lines of code. One model identified 100% of the attack-prone components (40% of the total number of components) with an 8% false positive rate. As such, the model could be used to prioritize fortification efforts in the system.

REFERENCES

• 1: S. Barnum and M. Gegick, "Design Principles," https://buildsecurityin.us--cert.gov/portal/article/knowledge/Principles, 2005.
• 2: Victor R. Basili , Lionel C. Briand , Walcélio L. Melo, A Validation of Object-Oriented Design Metrics as Quality Indicators, IEEE Transactions on Software Engineering, v.22 n.10, p.751-761, October 1996 [doi>10.1109/32.544352]
• 3: Pravir Chandra , Brian Chess , John Steven, Putting the Tools to Work: How to Succeed with Source Code Analysis, IEEE Security and Privacy, v.4 n.3, p.80-83, May 2006 [doi>10.1109/MSP.2006.77]
• 4: R. A. DeMillo , R. J. Lipton , F. G. Sayward, Hints on Test Data Selection: Help for the Practicing Programmer, Computer, v.11 n.4, p.34-41, April 1978 [doi>10.1109/C-M.1978.218136]
• 5: Giovanni Denaro, Estimating software fault-proneness for tuning testing activities, Proceedings of the 22nd international conference on Software engineering, p.704-706, June 04-11, 2000, Limerick, Ireland [doi>10.1145/337180.337592]
• 6: E. Dijkstra, Structured Programming, Brussels, Belgium, 1970.
• 7: M. Gegick and L. Williams, "Toward the Use of Static Analysis Alerts for Early Identification of Vulnerability- and Attack-prone Components," First International Workshop on Systems Vulnerabilities (SYVUL'07) Santa Clara, CA, July 1--6 2007.
• 8: T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning, New York, Springer, 2001.
• 9: S. Heckman and L. Williams, "Automated adaptive ranking and filtering of static analysis alerts," Fast abstract at the International Symposium on Software Reliability Engineering, Raleigh, NC, November 2006.
• 10: ISO, "ISO/IEC DIS 14598--1 Information Technology - Software Product Evaluation - Part 1: General Overview," October 28 1996.
• 11: ISO/IEC 24765, "Software and Systems Engineering Vocabulary," 2006.
• 12: Taghi M. Khoshgoftaar , Edward B. Allen , Wendell D. Jones , John P. Hudepohl, Classification Tree Models of Software Quality Over Multiple Releases, Proceedings of the 10th International Symposium on Software Reliability Engineering, p.116, November 01-04, 1999
• 13: Ivan Victor Krsul , Eugene H. Spafford, Software vulnerability analysis, Purdue University, West Lafayette, IN, 1998
• 14: R. J. Lipton and F. G. Sayward, "The Status of Research on Program Mutation," In Digest for the Workshop on Software Testing and Test Documentation, pp. 355--373, December 1978.
• 15: John C. Munson , Taghi M. Khoshgoftaar, The Detection of Fault-Prone Programs, IEEE Transactions on Software Engineering, v.18 n.5, p.423-433, May 1992 [doi>10.1109/32.135775]
• 16: Nachiappan Nagappan , Thomas Ball, Static analysis tools as early indicators of pre-release defect density, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062558]
• 17: Nachiappan Nagappan , Thomas Ball, Use of relative code churn measures to predict system defect density, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062514]
• 18: Stephan Neuhaus , Thomas Zimmermann , Christian Holler , Andreas Zeller, Predicting vulnerable software components, Proceedings of the 14th ACM conference on Computer and communications security, October 28-31, 2007, Alexandria, Virginia, USA [doi>10.1145/1315245.1315311]
• 19: A. Offutt, The coupling effect: fact or fiction, Proceedings of the ACM SIGSOFT '89 third symposium on Software testing, analysis, and verification, p.131-140, December 13-15, 1989, Key West, Florida, United States
• 20: Thomas J. Ostrand , Elaine J. Weyuker , Robert M. Bell, Where the bugs are, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 21: V. Prevelakis and D. Spinellis, "The Athens Affair," IEEE Spectrum, vol. 44, no. 7, pp. 26--33, July, 2007.
• 22: Adrian Schröter , Thomas Zimmermann , Andreas Zeller, Predicting component failures at design time, Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering, September 21-22, 2006, Rio de Janeiro, Brazil [doi>10.1145/1159733.1159739]
• 23: Jiang Zheng , Laurie Williams , Nachiappan Nagappan , Will Snipes , John P. Hudepohl , Mladen A. Vouk, On the Value of Static Analysis for Fault Detection in Software, IEEE Transactions on Software Engineering, v.32 n.4, p.240-253, April 2006 [doi>10.1109/TSE.2006.38]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Statistical methods

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Product metrics

General Terms:
Measurement, Security

Keywords:
attack-prone, vulnerability-prone