Using instruction block signatures to counter code injection attacks

ABSTRACT

With more computing platforms connected to the Internet each day, computer system security has become a critical issue. One of the major security problems is execution of malicious injected code. In this paper we propose new processor extensions that allow execution of trusted instructions only. The proposed extensions verify instruction block signatures in run-time. Signatures are generated during a trusted installation process, using a multiple input signature register (MISR), and stored in an encrypted form. The coefficients of the MISR and the key used for signature encryption are based on a hidden processor key. Signature verification is done in the background, concurrently with program execution, thus reducing negative impact on performance. The preliminary results indicate that the proposed processor extensions will prevent execution of any unauthorized code at a relatively small increase in system complexity and execution time.

REFERENCES

• 1: J. Wilander and M. Kamkar, "A Comparison of Publicly Available Tools for Dynamic Buffer Overflow Prevention," Proceedings of the 10th Network and Distributed System Security Symposium, San Diego, California, 2003, pp. 149--162.
• 2: T. Newsham, "Format string attacks", September 2000, <u><http://www.securityfocus.com/guest/3342></u> (January 2004).
• 3: Milena Milenković , Aleksandar Milenković , Emil Jovanov, A framework for trusted instruction execution via basic block signature verification, Proceedings of the 42nd annual Southeast regional conference, April 02-03, 2004, Huntsville, Alabama [doi>10.1145/986537.986582]
• 4: William Landi, Undecidability of static analysis, ACM Letters on Programming Languages and Systems (LOPLAS), v.1 n.4, p.323-337, Dec. 1992 [doi>10.1145/161494.161501]
• 5: D. Wagner, J. S. Foster, E. A. Brewer, and A. Aiken, "A first step towards automated detection of buffer overrun vulnerabilities," Network and Distributed System Security Symposium (NDCS), San Diego, CA, 2000.
• 6: D. Larochelle and D. Evans, "Statically detecting likely buffer overflow vulnerabilities," Proceedings of the 10th USENIX Security Symposium, Washington, D.C, 2001, pp. 177--189.
• 7: Nurit Dor , Michael Rodeh , Mooly Sagiv, CSSV: towards a realistic tool for statically detecting all buffer overflows in C, Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, June 09-11, 2003, San Diego, California, USA
• 8: C. Cowan, C. Pu, D. Maier, J. Walpole, P. Bakke, S. Beattie, A. Grier, P. Wagle, Q. Zhang, and H. Hinton, "StackGuard: Automatic adaptive detection and prevention of buffer overflowattacks," 7th USENIX Security Conference, San Antonio, Texas, 1998, pp. 63--78.
• 9: Kyung-suk Lhee , Steve J. Chapin, Type-Assisted Dynamic Buffer Overflow Detection, Proceedings of the 11th USENIX Security Symposium, p.81-88, August 05-09, 2002
• 10: C. Fetzer and Z. Xiao, "Detecting heap smashing attacks through fault containment wrappers," 20th IEEE Symposium on Reliable Distributed Systems, New Orleans, LA, USA, 2001, pp. 80--89.
• 11: M. Prasad and T.-c. Chiueh, "A Binary Rewriting Defense Against Stack-based Buffer Overflow Attacks," Usenix Annual Technical Conference, San Antonio, TX, 2003, pp. 211--224.
• 12: Joseph L. Steffen, Adding run-time checking to the portable C compiler, Software—Practice & Experience, v.22 n.4, p.305-316, April 1992 [doi>10.1002/spe.4380220403]
• 13: Trevor Jim , J. Greg Morrisett , Dan Grossman , Michael W. Hicks , James Cheney , Yanling Wang, Cyclone: A Safe Dialect of C, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.275-288, June 10-15, 2002
• 14: Massimo Bernaschi , Emanuele Gabrielli , Luigi V. Mancini, Operating system enhancements to prevent the misuse of system calls, Proceedings of the 7th ACM conference on Computer and communications security, p.174-183, November 01-04, 2000, Athens, Greece [doi>10.1145/352600.352624]
• 15: <u>"http://pax.grsecurity.net/"</u>, (February 2004).
• 16: P. Busser, "Memory Protection with PaX and the Stack Smashing Protector: Breaking out Peace," Linux Magazine, pp. 36--39, 2004.
• 17: S. Bhatkar, D. C. DuVarney, and R. Sekar, "Address Obfuscation: An Approach to Combat Buffer Overflows, Format-String Attacks, and More," 12th USENIX Security Symposium, Washington, DC, 2003, pp.
• 18: Elena Gabriela Barrantes , David H. Ackley , Trek S. Palmer , Darko Stefanovic , Dino Dai Zovi, Randomized instruction set emulation to disrupt binary code injection attacks, Proceedings of the 10th ACM conference on Computer and communications security, October 27-30, 2003, Washington D.C., USA [doi>10.1145/948109.948147]
• 19: Suan Hsi Yong , Susan Horwitz, Protecting C programs from attacks via invalid pointer dereferences, Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering, September 01-05, 2003, Helsinki, Finland
• 20: R. Sekar , Thomas F. Bowen , Mark E. Segal, On Preventing Intrusions by Process Behavior Monitoring, Proceedings of the Workshop on Intrusion Detection and Network Monitoring, p.29-40, April 09-12, 1999
• 21: C. Warrender, S. Forrest, and B. Pearlmutter, "Detecting Instructions Using System Calls: Alternative Data Models," IEEE Symposium on Security and Privacy, Oakland, CA, 1999, pp. 133--145.
• 22: I. Sato, Y. Okazaki, and S. Goto, "An Improved Intrusion Detection Method Based on Process Profiling," IPSJ Journal, vol. 43, pp. 3316--3326, 2002.
• 23: S. A. Hofmeyr, S. Forrest, and A. Somayaji, "Intrusion Detection using Sequences of System Calls" Journal of Computer Security, vol. 6, pp. 151--180, 1998.
• 24: D. L. Oppenheimer and M. R. Martonosi, "Performance Signatures: A Mechanism for Intrusion Detection," Proceedings of the 1997 IEEE Information Survivability Workshop, San Diego, California, 1997.
• 25: J. Xu, Z. Kalbarczyk, S. Patel, and R. K. Iyer, "Architecture Support for Defending Against Buffer Overflow Attacks," Workshop on Evaluating and Architecting System dependability (EASY), San Jose, California, 2002.
• 26: R. B. Lee, D. K. Karig, J. P. McGregor, and Z. Shi, "Enlisting Hardware Architecture to Thwart Malicious Code Injection," Security in Pervasive Computing, Boppard, Germany, 2003, pp. 237--252.
• 27: H. Ozdoganoglu, C. E. Brodley, T. N. Vijaykumar, B. A. Kuperman, and A. Jalote, "SmashGuard: A Hardware Solution to Prevent Security Attacks on the Function Return Address," Purdue University TR-ECE 03-13, November 22, 2003.
• 28: Gaurav S. Kc , Angelos D. Keromytis , Vassilis Prevelakis, Countering code-injection attacks with instruction-set randomization, Proceedings of the 10th ACM conference on Computer and communications security, October 27-30, 2003, Washington D.C., USA [doi>10.1145/948109.948146]
• 29: Darko Kirovski , Milenko Drinić , Miodrag Potkonjak, Enabling trusted software integrity, Proceedings of the 10th international conference on Architectural support for programming languages and operating systems, October 05-09, 2002, San Jose, California
• 30: G. Edward Suh , Jae W. Lee , David Zhang , Srinivas Devadas, Secure program execution via dynamic information flow tracking, Proceedings of the 11th international conference on Architectural support for programming languages and operating systems, October 07-13, 2004, Boston, MA, USA
• 31: A. Mahmood , E. J. McCluskey, Concurrent Error Detection Using Watchdog Processors-A Survey, IEEE Transactions on Computers, v.37 n.2, p.160-174, February 1988 [doi>10.1109/12.2145]
• 32: M. K. Joseph and A. Avizienis, "A fault tolerance approach to computer viruses," Proceedings of the 1988 IEEE Symposium on Security and Privacy, Oakland, California, USA, 1988, pp. 52--58.
• 33: A. Milenkovic and M. Milenkovic, "Exploiting Streams in Instruction and Data Address Trace Compression," Proceedings of IEEE 6th Annual Workshop on Workload Characterization, Austin, TX, 2003, pp. 99--107.
• 34: D. Burger and T. Austin, "The SimpleScalar Tool Set Version 2.0," University of Wisconsin, Technical Report CS-TR-97-1342, 1997.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.6 Security and Protection
          Subjects: Cryptographic controls

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.5 Local and Wide-Area Networks
          Subjects: Access schemes

  E. Data
  E.3 DATA ENCRYPTION
      Subjects: Public key cryptosystems

General Terms:
Design, Security