Finding bugs in dynamic web applications

ABSTRACT

Web script crashes and malformed dynamically-generated Web pages are common errors, and they seriously impact usability of Web applications. Current tools for Web-page validation cannot handle the dynamically-generated pages that are ubiquitous on today's Internet. In this work, we apply a dynamic test generation technique, based on combined concrete and symbolic execution, to the domain of dynamic Web applications. The technique generates tests automatically, uses the tests to detect failures, and minimizes the conditions on the inputs exposing each failure, so that the resulting bug reports are small and useful in finding and fixing the underlying faults. Our tool Apollo implements the technique for PHP. Apollo generates test inputs for the Web application, monitors the application for crashes, and validates that the output conforms to the HTML specification. This paper presents Apollo's algorithms and implementation, and an experimental evaluation that revealed 214 faults in 4 PHP Web applications.

REFERENCES

• 1: S. Anand, P. Godefroid, and N. Tillmann. Demand-driven compositional symbolic execution. In TACAS, 2008.
• 2: M. Benedikt, J. Freire, and P. Godefroid. VeriWeb: Automatically testing dynamic Web sites. In WWW, 2002.
• 3: Claus Braband , Anders Møller , Michael Schwartzbach, Static validation of dynamically generated HTML, Proceedings of the 2001 ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering, p.38-45, June 2001, Snowbird, Utah, United States [doi>10.1145/379605.379657]
• 4: C. Cadar and D. R. Engler. Execution generated test cases: How to make systems code crash itself. In SPIN, 2005.
• 5: C. Cadar and D. R. Engler. Execution generated test cases: How to make systems code crash itself. In SPIN, 2005.
• 6: Holger Cleve , Andreas Zeller, Locating causes of program failures, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062522]
• 7: Christoph Csallner , Nikolai Tillmann , Yannis Smaragdakis, DySy: dynamic symbolic execution for invariant inference, Proceedings of the 30th international conference on Software engineering, May 10-18, 2008, Leipzig, Germany [doi>10.1145/1368088.1368127]
• 8: David Wagner , Drew Dean, Intrusion Detection via Static Analysis, Proceedings of the 2001 IEEE Symposium on Security and Privacy, p.156, May 14-16, 2001
• 9: S. Elbaum, K.-R. Chilakamarri, M. Fisher, and G. Rothermel. Web application characterization through directed requests. In WODA, 2006.
• 10: Sebastian Elbaum , Gregg Rothermel , Srikanth Karre , Marc Fisher II, Leveraging User-Session Data to Support Web Application Testing, IEEE Transactions on Software Engineering, v.31 n.3, p.187-202, March 2005 [doi>10.1109/TSE.2005.36]
• 11: Michael Emmi , Rupak Majumdar , Koushik Sen, Dynamic test input generation for database applications, Proceedings of the 2007 international symposium on Software testing and analysis, July 09-12, 2007, London, United Kingdom [doi>10.1145/1273463.1273484]
• 12: M. Fisher, S. G. Elbaum, and G. Rothermel. Dynamic characterization of Web application interfaces. In FASE, 2007.
• 13: Patrice Godefroid, Compositional dynamic test generation, Proceedings of the 34th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages, January 17-19, 2007, Nice, France
• 14: Patrice Godefroid , Adam Kiezun , Michael Y. Levin, Grammar-based whitebox fuzzing, Proceedings of the 2008 ACM SIGPLAN conference on Programming language design and implementation, June 07-13, 2008, Tucson, AZ, USA
• 15: Patrice Godefroid , Nils Klarlund , Koushik Sen, DART: directed automated random testing, Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, June 12-15, 2005, Chicago, IL, USA
• 16: P. Godefroid, M. Y. Levin, and D. Molnar. Automated whitebox fuzz testing. In NDSS, 2008.
• 17: William G. J. Halfond , Alessandro Orso, Improving test case generation for web applications using automated interface discovery, Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, September 03-07, 2007, Dubrovnik, Croatia [doi>10.1145/1287624.1287646]
• 18: Kobi Inkumsah , Tao Xie, Evacon: a framework for integrating evolutionary and concolic testing for object-oriented programs, Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, November 05-09, 2007, Atlanta, Georgia, USA [doi>10.1145/1321631.1321700]
• 19: Martin Johns , Christian Beyerlein, SMask: preventing injection attacks in web applications by approximating automatic data/code separation, Proceedings of the 2007 ACM symposium on Applied computing, March 11-15, 2007, Seoul, Korea [doi>10.1145/1244002.1244071]
• 20: Nenad Jovanovic , Christopher Kruegel , Engin Kirda, Pixy: A Static Analysis Tool for Detecting Web Application Vulnerabilities (Short Paper), Proceedings of the 2006 IEEE Symposium on Security and Privacy, p.258-263, May 21-24, 2006 [doi>10.1109/SP.2006.29]
• 21: Rupak Majumdar , Koushik Sen, Hybrid Concolic Testing, Proceedings of the 29th international conference on Software Engineering, p.416-426, May 20-26, 2007 [doi>10.1109/ICSE.2007.41]
• 22: Rupak Majumdar , Ru-Gang Xu, Directed test generation using symbolic grammars, Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, November 05-09, 2007, Atlanta, Georgia, USA [doi>10.1145/1321631.1321653]
• 23: Yasuhiko Minamide, Static approximation of dynamically generated Web pages, Proceedings of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1060745.1060809]
• 24: Ghassan Misherghi , Zhendong Su, HDD: hierarchical delta debugging, Proceedings of the 28th international conference on Software engineering, May 20-28, 2006, Shanghai, China [doi>10.1145/1134285.1134307]
• 25: R. O'Callahan. Personal communication, 2008.
• 26: T. Pietraszek and C. V. Berghe. Defending against injection attacks through context-sensitive string evaluation. In RAID, 2005.
• 27: Filippo Ricca , Paolo Tonella, Analysis and testing of Web applications, Proceedings of the 23rd International Conference on Software Engineering, p.25-34, May 12-19, 2001, Toronto, Ontario, Canada
• 28: Koushik Sen , Darko Marinov , Gul Agha, CUTE: a concolic unit testing engine for C, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal
• 29: Sara Sprenkle , Emily Gibson , Sreedevi Sampath , Lori Pollock, Automated replay and failure detection for web applications, Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, November 07-11, 2005, Long Beach, CA, USA [doi>10.1145/1101908.1101947]
• 30: Zhendong Su , Gary Wassermann, The essence of command injection attacks in web applications, Conference record of the 33rd ACM SIGPLAN-SIGACT symposium on Principles of programming languages, p.372-382, January 11-13, 2006, Charleston, South Carolina, USA
• 31: Gary Wassermann , Zhendong Su, Sound and precise analysis of web applications for injection vulnerabilities, Proceedings of the 2007 ACM SIGPLAN conference on Programming language design and implementation, June 10-13, 2007, San Diego, California, USA
• 32: Gary Wassermann , Zhendong Su, Static detection of cross-site scripting vulnerabilities, Proceedings of the 30th international conference on Software engineering, May 10-18, 2008, Leipzig, Germany [doi>10.1145/1368088.1368112]
• 33: Yichen Xie , Alex Aiken, Static detection of security vulnerabilities in scripting languages, Proceedings of the 15th conference on USENIX Security Symposium, July 31-August 04, 2006, Vancouver, B.C., Canada
• 34: Andreas Zeller, Yesterday, my program worked. Today, it does not. Why?, Proceedings of the 7th European software engineering conference held jointly with the 7th ACM SIGSOFT international symposium on Foundations of software engineering, p.253-267, September 06-10, 1999, Toulouse, France

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

General Terms:
Reliability, Verification

Keywords:
dynamic analysis, php, software testing, web applications