Towards experimental evaluation of code obfuscation techniques

ABSTRACT

While many obfuscation schemes proposed, none of them satisfy any strong definition of obfuscation. Furthermore secure general-purpose obfuscation algorithms have been proven to be impossible. Nevertheless, obfuscation schemes which in practice slow down malicious reverse-engineering by obstructing code comprehension for even short periods of time are considered a useful protection against malicious reverse engineering. In previous works, the difficulty of reverse engineering has been mainly estimated by means of code metrics, by the computational complexity of static analysis or by comparing the output of de-obfuscating tools. In this paper we take a different approach and assess the difficulty attackers have in understanding and modifying obfuscated code through controlled experiments involving human subjects.

REFERENCES

• 1: Bertrand Anckaert , Matias Madou , Bjorn De Sutter , Bruno De Bus , Koen De Bosschere , Bart Preneel, Program obfuscation: a quantitative approach, Proceedings of the 2007 ACM workshop on Quality of protection, October 29-29, 2007, Alexandria, Virginia, USA [doi>10.1145/1314257.1314263]
• 2: Boaz Barak , Oded Goldreich , Russell Impagliazzo , Steven Rudich , Amit Sahai , Salil P. Vadhan , Ke Yang, On the (Im)possibility of Obfuscating Programs, Proceedings of the 21st Annual International Cryptology Conference on Advances in Cryptology, p.1-18, August 19-23, 2001
• 3: V. Basili, G. Caldiera, and D. H. Rombach. The Goal Question Metric Paradigm, Encyclopedia of Software Engineering. John Wiley and Sons, 1994.
• 4: C. Collberg, C. Thomborson, and D. Low. A taxonomy of obduscating transformations. Technical Report 148, Dept. of Computer Science, The Univ. of Auckland, 1997.
• 5: Christian Collberg , Clark Thomborson , Douglas Low, Manufacturing cheap, resilient, and stealthy opaque constructs, Proceedings of the 25th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, p.184-196, January 19-21, 1998, San Diego, California, United States [doi>10.1145/268946.268962]
• 6: P. Dalgaard. Introductory Statistics with R. Springer, 2002.
• 7: J. L. Devore. Probability and Statistics for Engineering and the Sciences. Duxbury Press; 7 edition, 2007.
• 8: Hideaki Goto , Masahiro Mambo , Kenjiro Matsumura , Hiroki Shizuya, An Approach to the Objective and Quantitative Evaluation of Tamper-Resistant Software, Proceedings of the Third International Workshop on Information Security, p.82-96, December 20-21, 2000
• 9: K. Heffner and C. Collberg. The obfuscation executive. In Proceedings of the 7th International Conference on Information Security, ISC'04, volume 3255 of LNCS, pages 428--440, 2004.
• 10: A. N. Oppenheim. Questionnaire Design, Interviewing and Attitude Measurement. Pinter, London, 1992.
• 11: Shari Lawrence Pfleeger, Experimental design and analysis in software engineering, part 5: analyzing the data, ACM SIGSOFT Software Engineering Notes, v.20 n.5, p.14-17, Dec. 1995 [doi>10.1145/217030.217032]
• 12: D. Sheskin. Handbook of Parametric and Nonparametric Statistical Procedures (fourth edition). Chapman & All, 2007.
• 13: I. Sutherland, G. E. Kalb, A. Blyth, and G. Mulley. An empirical examination of the reverse engineering process for binary files. Computers & Security, 25(3):221--228, 2006.
• 14: P. Tyma. Method for renaming identifiers of a computer program. US patent 6,102,966, 2000.
• 15: Sharath K. Udupa , Saumya K. Debray , Matias Madou, Deobfuscation: Reverse Engineering Obfuscated Code, Proceedings of the 12th Working Conference on Reverse Engineering, p.45-54, November 07-11, 2005 [doi>10.1109/WCRE.2005.13]
• 16: Claes Wohlin , Per Runeson , Martin Höst , Magnus C. Ohlsson , Bjöorn Regnell , Anders Wesslén, Experimentation in software engineering: an introduction, Kluwer Academic Publishers, Norwell, MA, 2000

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics

General Terms:
Experimentation, Measurement, Security

Keywords:
empirical studies, software obfuscation