VMM-based hidden process detection and identification using Lycosid

ABSTRACT

Use of stealth rootkit techniques to hide long-lived malicious processes is a current and alarming security issue. In this paper, we describe, implement, and evaluate a novel VMM-based hidden process detection and identification service called Lycosid that is based on the cross-view validation principle. Like previous VMM-based security services, Lycosid benefits from its protected location. In contrast top revious VMM-based hidden process detectors, Lycosid obtains guest process information implicitly. Using implicit information reduces its susceptibility to guest evasion attacks and decouples it from specific guest operating system versions and patch levels. The implicit information Lycosid depends on, however, can be noisy and unreliable. Statistical inference techniques like hypothesis testing and line arregression allow Lycosid to trade time for accuracy. Despite low quality inputs, Lycosid provides a robust, highly accurate service usable even insecurity environments where the consequences for wrong decisions can behig.

REFERENCES

• 1: 90210. Bypassing klister 0.4 with no hooks or running a controlled thread scheduler. hi-tech.nsys.by/33/.
• 2: J. Butler, J.L. Undercoffer, and J. Pinkston. Hidden processes: The implication for intrusion detection. In Proceedings of the 2003 IEEE Workshop on Information Assurance, pages 116--121, June 2003.
• 3: J. Clemens. Knark: Linux kernel subversion. www.sans.org/ resources/ idfaq/ knark.php.
• 4: B. Cogswell and M. Russinovich. Pslist. www.sysinternals.com.
• 5: B. Cogswell and M. Russinovich. Rootkit revealer. www.sysinternals.com.
• 6: John R. Douceur , William J. Bolosky, Progress-based regulation of low-importance processes, Proceedings of the seventeenth ACM symposium on Operating systems principles, p.247-260, December 12-15, 1999, Charleston, South Carolina, United States
• 7: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 8: George W. Dunlap , Samuel T. King , Sukru Cinar , Murtaza A. Basrai , Peter M. Chen, ReVirt: enabling intrusion analysis through virtual-machine logging and replay, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060309]
• 9: fuzen_op. fu.exe and msdirectx.sys. www.rootkit.com/.
• 10: T. Garfinkel and M. Rosenblum. A virtual machine introspection based architecture for intrusion detection. In Proc. Network and Distributed Systems Security Symposium, February 2003.
• 11: Mor Harchol-Balter , Allen B. Downey, Exploiting process lifetime distributions for dynamic load balancing, ACM Transactions on Computer Systems (TOCS), v.15 n.3, p.253-285, Aug. 1997 [doi>10.1145/263326.263344]
• 12: Holy Father. HackerDefender. hxdef.org.
• 13: Intel and Symantec Corp. Symantec virtual security solution and pcs with intel vpro technology. http://www.intel.com/business/casestudies/symantec\_solutions\_brief.pdf.
• 14: Intel Corporation. Vtx specification. developer.intel.com, 2005.
• 15: Stephen T. Jones , Andrea C. Arpaci-Dusseau , Remzi H. Arpaci-Dusseau, Antfarm: tracking processes in a virtual machine environment, Proceedings of the annual conference on USENIX '06 Annual Technical Conference, p.1-1, May 30-June 03, 2006, Boston, MA
• 16: Stephen T. Jones , Andrea C. Arpaci-Dusseau , Remzi H. Arpaci-Dusseau, Geiger: monitoring the buffer cache in a virtual machine environment, Proceedings of the 12th international conference on Architectural support for programming languages and operating systems, October 21-25, 2006, San Jose, California, USA
• 17: Ashlesha Joshi , Samuel T. King , George W. Dunlap , Peter M. Chen, Detecting past and present intrusions through vulnerability-specific predicates, Proceedings of the twentieth ACM symposium on Operating systems principles, October 23-26, 2005, Brighton, United Kingdom
• 18: J. Jung, V. Paxson, A.W. Berger, and H. Balakrishnan. Fast Portscan Detection Using Sequential Hypothesis Testing. In IEEE Symposium on Security and Privacy 2004, Oakland, CA, May 2004.
• 19: Paul A. Karger , Mary Ellen Zurko , Douglas W. Bonin , Andrew H. Mason , Clifford E. Kahn, A Retrospective on the VAX VMM Security Kernel, IEEE Transactions on Software Engineering, v.17 n.11, p.1147-1165, November 1991 [doi>10.1109/32.106971]
• 20: Microsoft. Windows malicious software removal tool. www.microsoft.com.
• 21: T. Miller. t0rn rootkit. www.ossec.net/rootkits/studies/t0rn.txt.
• 22: R. Naraine. Microsoft: Stealth Rootkits Are Bombarding XP SP2 Boxes. www.eweek.com/article2/0,1895,1896605,00.asp.
• 23: Nick L. Petroni, Jr. , Timothy Fraser , Jesus Molina , William A. Arbaugh, Copilot - a coprocessor-based kernel runtime integrity monitor, Proceedings of the 13th conference on USENIX Security Symposium, p.13-13, August 09-13, 2004, San Diego, CA
• 24: R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2006. ISBN 3-900051-07-0.
• 25: F.L. Ramsey and D.W. Schafer. The Statistical Sleuth: A Course in Methods of Data Analysis. Duxbury Press, Boston, MA, 2nd edition, 2002.
• 26: J. Rutkowska. klister. www.invisiblethings.org/tools/klister-0.4.zip.
• 27: SANS Institute. Subseven trojan v 1.1. www.sans.org/resources/idfaq/subseven.php.
• 28: sd and devik. Suckit. Phrack #58, article 0x07.
• 29: A. Wald. Sequential Analysis. John Wiley & Sons, Inc., New York, NY, 3rd edition, September 1952.
• 30: Doug Beck , Binh Vo , Chad Verbowski, Detecting Stealth Software with Strider GhostBuster, Proceedings of the 2005 International Conference on Dependable Systems and Networks, p.368-377, June 28-July 01, 2005 [doi>10.1109/DSN.2005.39]

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.6 Security and Protection

General Terms:
Design, Measurement, Security

Keywords:
inference, security, virtual machine