QVM: an efficient runtime for detecting defects in deployed systems

ABSTRACT

Coping with software defects that occur in the post-deployment stage is a challenging problem: bugs may occur only when the system uses a specific configuration and only under certain usage scenarios. Nevertheless, halting production systems until the bug is tracked and fixed is often impossible. Thus, developers have to try to reproduce the bug in laboratory conditions. Often the reproduction of the bug consists of the lion share of the debugging effort.

In this paper we suggest an approach to address the aforementioned problem by using a specialized runtime environment (QVM, for Quality Virtual Machine). QVM efficiently detects defects by continuously monitoring the execution of the application in a production setting. QVM enables the efficient checking of violations of user-specified correctness properties, e.g., typestate safety properties, Java assertions, and heap properties pertaining to ownership.

QVM is markedly different from existing techniques for continuous monitoring by using a novel overhead manager which enforces a user-specified overhead budget for quality checks. Existing tools for error detection in the field usually disrupt the operation of the deployed system. QVM, on the other hand, provides a balanced trade off between the cost of the monitoring process and the maintenance of sufficient accuracy for detecting defects. Specifically, the overhead cost of using QVM instead of a standard JVM, is low enough to be acceptable in production environments.

We implemented QVM on top of IBM's J9 Java Virtual Machine and used it to detect and fix various errors in real-world applications.

REFERENCES

• 1: GOIM: Gamers own instant messenger. available at http://goim.us/wiki/show/GOIM.
• 2: Edward Aftandilian , Samuel Z. Guyer, GC assertions: using the garbage collector to check heap properties, Proceedings of the 2008 ACM SIGPLAN workshop on Memory systems performance and correctness: held in conjunction with the Thirteenth International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS '08), March 02-02, 2008, Seattle, Washington [doi>10.1145/1353522.1353533]
• 3: Chris Allan , Pavel Avgustinov , Aske Simon Christensen , Laurie Hendren , Sascha Kuzins , Ondřej Lhoták , Oege de Moor , Damien Sereni , Ganesh Sittampalam , Julian Tibble, Adding trace matching with free variables to AspectJ, Proceedings of the 20th annual ACM SIGPLAN conference on Object oriented programming, systems, languages, and applications, October 16-20, 2005, San Diego, CA, USA
• 4: Matthew Arnold , Barbara G. Ryder, A framework for reducing the cost of instrumented code, Proceedings of the ACM SIGPLAN 2001 conference on Programming language design and implementation, p.168-179, June 2001, Snowbird, Utah, United States
• 5: Matthew Arnold , Barbara G. Ryder, Thin Guards: A Simple and Effective Technique for Reducing the Penalty of Dynamic Class Loading, Proceedings of the 16th European Conference on Object-Oriented Programming, p.498-524, June 10-14, 2002
• 6: Auerbach, J., Bacon, D., Cheng, P., Grove, D., Biron, B., Gracie, C., McCloskey, B., Micic, A., and Sciampacone, R. Toshio suganuma and toshiaki yasue and motohiro kawahito and hideaki komatsu and toshio nakatani. In Proceedings of the International Conference on Embedded Software (New York, NY, USA, 2008), ACM.
• 7: Pavel Avgustinov , Julian Tibble , Oege de Moor, Making trace monitors feasible, Proceedings of the 22nd annual ACM SIGPLAN conference on Object oriented programming systems and applications, October 21-25, 2007, Montreal, Quebec, Canada
• 8: Azureus - Java BitTorrent client. http://azureus.sourceforge.net/.
• 9: Bodden, E., Hendren, L. J., Lam, P., Lhoták, O., and Naeem, N. A. Collaborative runtime verification with tracematches. In 7th International Workshop on Runtime Verification (RV) (2007), vol. 4839 of Lecture Notes in Computer Science, pp. 9--21.
• 10: Bodden, E., Hendren, L. J., and Lhoták, O. A staged static program analysis to improve the performance of runtime monitoring. In ECOOP (2007), pp. 525--549.
• 11: Michael D. Bond , Kathryn S. McKinley, Bell: bit-encoding online memory leak detection, ACM SIGPLAN Notices, v.41 n.11, November 2006
• 12: Feng Chen , Grigore Roşu, Mop: an efficient and generic runtime verification framework, Proceedings of the 22nd annual ACM SIGPLAN conference on Object oriented programming systems and applications, October 21-25, 2007, Montreal, Quebec, Canada
• 13: Chilimbi, T. M., and Ganapathy, V. Heapmd: identifying heap-based bugs using anomaly detection. vol. 34, ACM, pp. 219--228.
• 14: Trishul M. Chilimbi , Martin Hirzel, Dynamic hot data stream prefetching for general-purpose programs, Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation, June 17-19, 2002, Berlin, Germany
• 15: Robert DeLine , Manuel Fähndrich, Enforcing high-level protocols in low-level software, Proceedings of the ACM SIGPLAN 2001 conference on Programming language design and implementation, p.59-69, June 2001, Snowbird, Utah, United States
• 16: DeLine, R., and Fähndrich, M. Adoption and focus: Practical linear types for imperative programming. pp. 13--24.
• 17: Isil Dillig , Thomas Dillig , Eran Yahav , Satish Chandra, The CLOSER: automating resource management in java, Proceedings of the 7th international symposium on Memory management, June 07-08, 2008, Tucson, AZ, USA [doi>10.1145/1375634.1375636]
• 18: Eclipse. Standard widget toolkit (swt). http://www.eclipse.org/swt/.
• 19: Stephen Fink , Eran Yahav , Nurit Dor , G. Ramalingam , Emmanuel Geay, Effective typestate verification in the presence of aliasing, Proceedings of the 2006 international symposium on Software testing and analysis, July 17-20, 2006, Portland, Maine, USA [doi>10.1145/1146238.1146254]
• 20: Stephen J. Fink , Feng Qian, Design, implementation and evaluation of adaptive recompilation with on-stack replacement, Proceedings of the international symposium on Code generation and optimization: feedback-directed and runtime optimization, March 23-26, 2003, San Francisco, California
• 21: Foster, J. S., Terauchi, T., and Aiken, A. Flow-sensitive type qualifiers. pp. 1--12.
• 22: Matthias Hauswirth , Trishul M. Chilimbi, Low-overhead memory leak detection using adaptive statistical profiling, ACM SIGPLAN Notices, v.39 n.11, November 2004
• 23: David L. Heine , Monica S. Lam, A practical flow-sensitive and context-sensitive C and C++ memory leak detector, Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, June 09-11, 2003, San Diego, California, USA
• 24: Maria Jump , Kathryn S. McKinley, Cork: dynamic memory leak detection for garbage-collected languages, ACM SIGPLAN Notices, v.42 n.1, January 2007
• 25: Jeremy Lau , Matthew Arnold , Michael Hind , Brad Calder, Online performance auditing: using hot optimizations without getting burned, ACM SIGPLAN Notices, v.41 n.6, June 2006
• 26: Tal Lev-Ami , Shmuel Sagiv, TVLA: A System for Implementing Static Analyses, Proceedings of the 7th International Symposium on Static Analysis, p.280-301, June 29-July 01, 2000
• 27: Ben Liblit, Cooperative Bug Isolation: Winning Thesis of the 2005 ACM Doctoral Dissertation Competition (Lecture Notes in Computer Science), Springer-Verlag New York, Inc., Secaucus, NJ, 2007
• 28: Livshits, V. B. Turning Eclipse against itself: Finding bugs in Eclipse code using lightweight static analysis. Eclipsecon '05 Research Exchange, Mar. 2005.
• 29: Microsystems, S. Jvmtm tool interface, version 1.0. In http://java.sun.com/j2se/1.5.0/docs/guide/jvmti/jvmti.html.
• 30: Mitchell, N. The runtime structure of object ownership. In ECOOP (2006), D. Thomas, Ed., vol. 4067 of Lecture Notes in Computer Science, Springer, pp. 74--98.
• 31: Nick Mitchell , Gary Sevitsky, The causes of bloat, the limits of health, Proceedings of the 22nd annual ACM SIGPLAN conference on Object oriented programming systems and applications, October 21-25, 2007, Montreal, Quebec, Canada
• 32: Peter Müller , Arsenii Rudich, Ownership transfer in universe types, Proceedings of the 22nd annual ACM SIGPLAN conference on Object oriented programming systems and applications, October 21-25, 2007, Montreal, Quebec, Canada
• 33: Feng Qian , Laurie Hendren, An adaptive, region-based allocator for java, Proceedings of the 3rd international symposium on Memory management, June 20-21, 2002, Berlin, Germany
• 34: G. Ramalingam , Alex Warshavsky , John Field , Deepak Goyal , Mooly Sagiv, Deriving specialized program analyses for certifying component-client conformance, Proceedings of the ACM SIGPLAN 2002 Conference on Programming language design and implementation, June 17-19, 2002, Berlin, Germany
• 35: Shaham, R., Yahav, E., Kolodner, E., and Sagiv, M. Establishing local temporal heap safety properties with applications to compile-time memory management. In Static Analysis Symposium (2003).
• 36: R E Strom , S Yemini, Typestate: A programming language concept for enhancing software reliability, IEEE Transactions on Software Engineering, v.12 n.1, p.157-171, Jan. 1986
• 37: Toshio Suganuma , Toshiaki Yasue , Motohiro Kawahito , Hideaki Komatsu , Toshio Nakatani, A dynamic optimization framework for a Java just-in-time compiler, Proceedings of the 16th ACM SIGPLAN conference on Object oriented programming, systems, languages, and applications, p.180-195, October 14-18, 2001, Tampa Bay, FL, USA
• 38: Eran Yahav , G. Ramalingam, Verifying safety properties using separation and heterogeneous abstractions, Proceedings of the ACM SIGPLAN 2004 conference on Programming language design and implementation, June 09-11, 2004, Washington DC, USA

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Debugging aids

Additional Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.4 Processors
          Subjects: Run-time environments

General Terms:
Algorithms, Reliability

Keywords:
algorithms, reliability, virtual machines