Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

A proper security metrics and measurement process is thus a means to automatize security decisions. Unfortunately, so far automatic security evaluation techniques have failed to achieve the performance of security experts. In this paper we argue security metrics are by nature highly unstable in time. Moreover, their effectiveness depends on specific target of evaluation. In this paper we elaborate this finding and we describe our experimental framework with its results.

REFERENCES

• 1: Rakesh Agrawal , Tomasz Imieliński , Arun Swami, Mining association rules between sets of items in large databases, Proceedings of the 1993 ACM SIGMOD international conference on Management of data, p.207-216, May 25-28, 1993, Washington, D.C., United States [doi>10.1145/170035.170072]
• 2: Marco D. Aime , Andrea Atzeni , Paolo C. Pomi, AMBRA: automated model-based risk analysis, Proceedings of the 2007 ACM workshop on Quality of protection, October 29-29, 2007, Alexandria, Virginia, USA [doi>10.1145/1314257.1314272]
• 3: A. Atzeni and A. Lioy. Why to adopt a security metric? A brief survey. In QoP2005, First Int. Workshop on Quality of Protection, pages 1--12, 15 September 2005.
• 4: A. Avizienis, J. C. Laprie, and B. Randell. Dependability and its threats: A taxonomy. In Proc. of IFIP 18th World Computer Congress, pages 91--120, 22--27 August 2004.
• 5: M. Dacier. Towards Quantitative Evaluation of Computer Security. PhD thesis, Institute National Politechnique de Toulose, 1994.
• 6: FIRST. Common Vulnerability Scoring System (CVSS). http://www.first.org/cvss/cvss-guide.html.
• 7: K. Fowler and J. Schmalzel. Why do we care about measurement? Instrumentation & Measurement Magazine, IEEE, 7(1):38--46, March 2004.
• 8: D. T. Lee , S. P. Shieh , J. D. Tygar, Computer Security in the 21st Century, Springer-Verlag New York, Inc., Secaucus, NJ, 2005
• 9: ISO/IEC 27004. Information technology - Security techniques - Information security management - Measurements - draft. ISO/IEC, 2006.
• 10: F. Iwu and I. Toyn. Modelling and analysing fault propagation in safety-related systems. In 28th Annual NASA Goddard Software Engineering Workshop, York (UK), pages 167--174, 3--4 Dec 2003.
• 11: N. Kavantzas, D. Burdett, G. Ritzinger, T. Fletcher, Y. Lafon, and C. Barreto. Web services choreography description language version 1.0. W3C Recommendation, http://www.w3.org/TR/ws-cdl-10/, November 2005.
• 12: N. Leveson. A new accident model for engineering safer systems. In Safety ScienceVolume 42, Issue 4, pages 237--270, April 2004.
• 13: B. Martin, C. Sullo, and J. Kouns. OSVDB: Open Source Vulnerability Database. http://www.osvdb.org/database-info.php.
• 14: Ministerio de Administraciones Publicas. Methodology for information systems risk analysis and management (MAGERIT) version 2. http://www.csae.map.es/.
• 15: David M. Nicol , William H. Sanders , Kishor S. Trivedi, Model-Based Evaluation: From Dependability to Security, IEEE Transactions on Dependable and Secure Computing, v.1 n.1, p.48-65, January 2004 [doi>10.1109/TDSC.2004.11]
• 16: NIST. National vulnerability database. http://nvd.nist.gov/.
• 17: Xinming Ou , Wayne F. Boyer , Miles A. McQueen, A scalable approach to attack graph generation, Proceedings of the 13th ACM conference on Computer and communications security, October 30-November 03, 2006, Alexandria, Virginia, USA [doi>10.1145/1180405.1180446]
• 18: POSITIF Project. System Description Language (PSDL). http://www.positif.org/.
• 19: Ronald W. Ritchey , Paul Ammann, Using Model Checking to Analyze Network Vulnerabilities, Proceedings of the 2000 IEEE Symposium on Security and Privacy, p.156, May 14-17, 2000
• 20: R. Sawilla and X. Ou. Googling attack graphs. In Technical Report for Defence R&D Canada Ottawa, September 2007.
• 21: The DMTF Technical Committee. The Common Information Model. http://www.dmtf.org/standards/cim.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.5 Security and Protection (D.4.6, K.4.2)

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Modeling techniques; Measurement techniques

General Terms:
Management, Reliability, Security

Keywords:
runtime risk analysis, security measurement