Mining specifications of malicious behavior

ABSTRACT

Malware detectors require a specification of malicious behavior. Typically, these specifications are manually constructed by investigating known malware. We present an automatic technique to overcome this laborious manual process. Our technique derives such a specification by comparing the execution behavior of a known malware against the execution behaviors of a set of benign programs. In other words, we mine the malicious behavior present in a known malware that is not present in a set of benign programs. The output of our algorithm can be used by malware detectors to detect malware variants. Since our algorithm provides a succinct description of malicious behavior present in a malware, it can also be used by security analysts for understanding the malware. We have implemented a prototype based on our algorithm and tested it on several malware programs. Experimental results obtained from our prototype indicate that our algorithm is effective in extracting malicious behaviors that can be used to detect malware variants.

REFERENCES

• 1: Glenn Ammons , Rastislav Bodík , James R. Larus, Mining specifications, Proceedings of the 29th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, p.4-16, January 16-18, 2002, Portland, Oregon
• 2: Taweesup Apiwattanapong , Alessandro Orso , Mary Jean Harrold, A Differencing Algorithm for Object-Oriented Programs, Proceedings of the 19th IEEE international conference on Automated software engineering, p.2-13, September 20-24, 2004 [doi>10.1109/ASE.2004.5]
• 3: Hamid Abdul Basit , Stan Jarzabek, Detecting higher-level similarity patterns in programs, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal
• 4: Sandeep Bhatkar , Abhishek Chaturvedi , R. Sekar, Dataflow Anomaly Detection, Proceedings of the 2006 IEEE Symposium on Security and Privacy, p.48-62, May 21-24, 2006 [doi>10.1109/SP.2006.12]
• 5: BindView. Strace for NT. Published online at http://www.bindview.com/Services/RAZOR/Utilities/Windows/strace_readme.cfm (accessed 9 Sep. 2006).
• 6: Mihai Christodorescu , Somesh Jha, Testing malware detectors, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 7: Mihai Christodorescu , Somesh Jha , Sanjit A. Seshia , Dawn Song , Randal E. Bryant, Semantics-Aware Malware Detection, Proceedings of the 2005 IEEE Symposium on Security and Privacy, p.32-46, May 08-11, 2005 [doi>10.1109/SP.2005.20]
• 8: J. T. Giffin, S. Jha, and B. P. Miller. Efficient context-sensitive intrusion detection. In Proc. 11th Network and Distributed System Security Symposium (NDSS'04), 2004.
• 9: Susan Horwitz, Identifying the semantic and textual differences between two versions of a program, Proceedings of the ACM SIGPLAN 1990 conference on Programming language design and implementation, p.234-245, June 1990, White Plains, New York, United States
• 10: Daniel Jackson , David A. Ladd, Semantic Diff: A Tool for Summarizing the Effects of Modifications, Proceedings of the International Conference on Software Maintenance, p.243-252, September 01, 1994
• 11: Toshihiro Kamiya , Shinji Kusumoto , Katsuro Inoue, CCFinder: a multilinguistic token-based code clone detection system for large scale source code, IEEE Transactions on Software Engineering, v.28 n.7, p.654-670, July 2002 [doi>10.1109/TSE.2002.1019480]
• 12: J. Kinder, S. Katzenbeisser, C. Schallhart, and H. Veith. Detecting malicious code by model checking. In Proc. 2nd Conference on Detection of Intrusions and Malware & Vulnerability Assessment (DIMVA'05), pages 174--187, 2005.
• 13: Raghavan Komondoor , Susan Horwitz, Using Slicing to Identify Duplication in Source Code, Proceedings of the 8th International Symposium on Static Analysis, p.40-56, July 16-18, 2001
• 14: C. Kruegel, D. Mutz, W. Robertson, G. Vigna, and R. Kemmerer. Reverse engineering of network signatures. In AusCERT Asia Pacific IT Security Conference, 2005.
• 15: C. Kruegel, D. Mutz, F. Valeur, and G. Vigna. On the detection of anomalous system call arguments. In Proc. 8th European Symposium on Research in Computer Security (ESORICS'03), pages 101--118, 2003.
• 16: Christopher Kruegel , William Robertson , Giovanni Vigna, Detecting Kernel-Level Rootkits Through Binary Analysis, Proceedings of the 20th Annual Computer Security Applications Conference, p.91-100, December 06-10, 2004 [doi>10.1109/CSAC.2004.19]
• 17: J. Laski and W. Szermer. Identification of program modifications and its applications in software maintenance. In Proc. Conference on Software Maintenance, pages 282--290, Nov. 9-12 1992.
• 18: Zhenmin Li , Yuanyuan Zhou, PR-Miner: automatically extracting implicit programming rules and detecting violations in large software code, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal
• 19: A. Marinescu. Russian doll. Virus Bulletin, 15(8):7--9, Aug. 2003.
• 20: Andreas Moser , Christopher Kruegel , Engin Kirda, Exploring Multiple Execution Paths for Malware Analysis, Proceedings of the 2007 IEEE Symposium on Security and Privacy, p.231-245, May 20-23, 2007 [doi>10.1109/SP.2007.17]
• 21: Steven S. Muchnick, Advanced compiler design and implementation, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 22: Carey Nachenberg, Computer virus-antivirus coevolution, Communications of the ACM, v.40 n.1, p.46-51, Jan. 1997 [doi>10.1145/242857.242869]
• 23: R. Sekar , M. Bendre , D. Dhurjati , P. Bollineni, A Fast Automaton-Based Method for Detecting Anomalous Program Behaviors, Proceedings of the 2001 IEEE Symposium on Security and Privacy, p.144, May 14-16, 2001
• 24: Symantec Antivirus Research Center. Expanded threat list and virus encyclopedia. Published online at http://www.symantec.com/enterprise/security_response/threatexplorer/index.jsp (accessed 9 Sep. 2006).
• 25: P. Szor and P. Ferrie. Hunting for metamorphic. In Virus Bulletin Conference, pages 123 -- 144, 2001.
• 26: R. M. H. Ting and J. Bailey. Mining minimal contrast subgraph patterns. In 6th SIAM International Conference on Data Mining, pages 638--642, 2006.
• 27: W. Weimer and G. C. Necula. Mining temporal specifications for error detection. In Proc. 11th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS'05), pages 461--476, 2005.
• 28: I. Whalley, B. Arnold, D. Chess, J. Morar, and A. Segal. An environment for controlled worm replication & analysis (Internet-inna-Box). In Virus Bulletin Conference, 2000.
• 29: z0mbie. z0mbie's homepage. Published online at http://z0mbie.host.sk (accessed 16 Jan. 2004).
• 30: Xiangyu Zhang , Rajiv Gupta, Matching execution histories of program versions, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.6 Security and Protection
          Subjects: Invasive software (e.g., viruses, worms, Trojan horses)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.1 Requirements/Specifications

General Terms:
Experimentation, Languages, Measurement, Security

Keywords:
behavior-based detection, differential analysis, malspec