Controlled dynamic performance analysis

ABSTRACT

We are interested in obtaining detailed performance information on-the-fly from long-running systems without adversely affecting the performance of the systems. We have developed a methodology consisting of a framework, DYPER, and a number of specialized agents called proflets each of which analyzes a different performance aspect. DYPER gathers performance information with a guaranteed maximum overhead that is dynamically settable by the programmer using priorities set by the proflets. Moreover, the type of information that the system can provide is generally only available for tools that generally have too much overhead to be usable in production or long-running systems. DYPER includes the ability to control and display performance data as the program is run.

REFERENCES

• 1: Jennifer M. Anderson , Lance M. Berc , Jeffrey Dean , Sanjay Ghemawat , Monika R. Henzinger , Shun-Tak A. Leung , Richard L. Sites , Mark T. Vandevoorde , Carl A. Waldspurger , William E. Weihl, Continuous profiling: where have all the cycles gone?, ACM Transactions on Computer Systems (TOCS), v.15 n.4, p.357-390, Nov. 1997 [doi>10.1145/265924.265925]
• 2: Ziya Aral , Ilya Gertner, Non-intrusive and interactive profiling in parasight, Proceedings of the ACM/SIGPLAN conference on Parallel programming: experience with applications, languages and systems, p.21-30, July 19-21, 1988, New Haven, Connecticut, United States
• 3: Dorian C. Arnold, Dong H. Ahn, Bronis R. de Supinski, Gregory Lee, Barton P. Miller, and Martin Schulz, "Stack trace analysis for large scale debugging," Proc. IPDPS 2007, pp. 1--10 (March 2007).
• 4: Thomas Ball , James R. Larus, Optimally profiling and tracing programs, Proceedings of the 19th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, p.59-70, January 19-22, 1992, Albuquerque, New Mexico, United States [doi>10.1145/143165.143180]
• 5: Thomas Ball , James R. Larus, Efficient path profiling, Proceedings of the 29th annual ACM/IEEE international symposium on Microarchitecture, p.46-57, December 02-04, 1996, Paris, France
• 6: Andrew R. Bernat , Barton P. Miller, Incremental call-path profiling: Research Articles, Concurrency and Computation: Practice & Experience, v.19 n.11, p.1533-1547, August 2007 [doi>10.1002/cpe.v19:11]
• 7: Walter Binder , Jane G. Hulaas , Alex Villazón, Portable resource control in Java, Proceedings of the 16th ACM SIGPLAN conference on Object oriented programming, systems, languages, and applications, p.139-155, October 14-18, 2001, Tampa Bay, FL, USA
• 8: Walter Binder and Jarle Hulaas, "Java bytecode transformations for efficient portable CPU accounting," Electronic Notes in Theoretical Computer Science Vol. 141 pp. 53--73 (2005).
• 9: Walter Binder and Jarle Hulaas, "Using bytecode instruction counting as portable CPU consumption metric," Electronic Notes in Theoretical Computer Science Vol. 153 pp. 57--77 (2006).
• 10: Walter Binder, Portable and accurate sampling profiling for Java, Software—Practice & Experience, v.36 n.6, p.615-650, May 2006 [doi>10.1002/spe.v36:6]
• 11: Harold W. Cain , Barton P. Miller , Brian J. N. Wylie, A Callgraph-Based Search Strategy for Automated Performance Diagnosis (Distinguished Paper), Proceedings from the 6th International Euro-Par Conference on Parallel Processing, p.108-122, August 29-September 01, 2000
• 12: Bryan M. Cantrill , Michael W. Shapiro , Adam H. Leventhal, Dynamic instrumentation of production systems, Proceedings of the annual conference on USENIX Annual Technical Conference, p.2-2, June 27-July 02, 2004, Boston, MA
• 13: Trishul M. Chilimbi and Matthias Hauswirth, "Low-overhead memory leak detection using adaptive statistical profiling," ASPLOS '04, pp. 156--164 (October 2004).
• 14: Mandy Chung, "Using JConsole to monitor appliocations," Sun Microsystems, http://java.sun.com/developer/techicalArticals/j2se/jconsole.html, (December 2004).
• 15: Grzegorz Czajkowski, Stephen Hahn, Glenn Skinner, Pete Soper, and Ciaran Bryce, "A resource management interface for the Java platform," Sun Microsystems TR-2003-124, (May 2003).
• 16: Markus Dahm, "Byte code engineering," Java Infromations Tage '99, pp. 267--277 (1999).
• 17: Mikhali Dmitriev, "Design of JFluid: A profiling technology and tool based on dynamic bytecode instrumentation," Sun Microsystems Report TR_2003-125, (November 2003).
• 18: The Eclipse Foundation, "Eclipse test and performance tools platform project," http://www.eclipse.org/tptp/index.php, (August 2007).
• 19: Nathan Froyd , John Mellor-Crummey , Rob Fowler, Low-overhead call path profiling of unmodified, optimized code, Proceedings of the 19th annual international conference on Supercomputing, June 20-22, 2005, Cambridge, Massachusetts [doi>10.1145/1088149.1088161]
• 20: Susan L. Graham , Peter B. Kessler , Marshall K. Mckusick, Gprof: A call graph execution profiler, ACM SIGPLAN Notices, v.17 n.6, p.120-126, June 1982
• 21: Martin Hirzel and Trishul M. Chilimbi, "Bursty tracing: a framework for low-overhead temporal profiling," 4th Workshop on Feedback-Directed and Dynamic Optimization, (December 2001).
• 22: Jeffrey K. Hollingsworth , Barton P. Miller, An Adaptive Cost System for Parallel Program Instrumentation, Proceedings of the Second International Euro-Par Conference on Parallel Processing, p.88-97, August 26-29, 1996
• 23: Karen L. Karavanic , Barton P. Miller, Improving online performance diagnosis by the use of historical performance data, Proceedings of the 1999 ACM/IEEE conference on Supercomputing (CDROM), p.42-es, November 14-19, 1999, Portland, Oregon, United States [doi>10.1145/331532.331574]
• 24: Gunter Kniesel and Pascal Costanza, "JMangler: a framework for load-time transformation of Java class files," SCAM 2001, pp. 100--110 (November 2001).
• 25: Chris Laffra, Doug Lorch, Dave Streeter, Frank Tip, and John Field, "What is Jikes Bytecode Toolkit," http://www.alphaworks.ibm.com/tech/jikesbt, (March 2000).
• 26: James R. Larus, "EEL guts: Using the EEL executable editing library," University of Wisconsin-Madison Computer Science Department (November 1996).
• 27: Ben Liblit , Alex Aiken , Alice X. Zheng , Michael I. Jordan, Bug isolation via remote program sampling, Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, June 09-11, 2003, San Diego, California, USA
• 28: Babak Mahdavi and Karel Driesen, "Heap hot spot visualization in Java," McGill U. CS Tech report SOCS-01.8, (May 2001).
• 29: Leo Meyerovich, Michael Greenberg, Gregory Cooper, Aleks Bromfield, and Shriram Krishnamurthi, "FlapJax," http://www.flapjax-lang.org, (2007).
• 30: Sun Microsystems, "JVM tool interface version 1.0," http://java.sun.com/j2se/1.5.0/docs/guide/jvmti/jvmti.html, (November 2004).
• 31: Sun Microsystems, "Dynamic tracing support in the Java HotSpot virtual machine," http://ava.sun.com/j2se/reference/whitepapers/java-dtrace-whitepaper.pdf, (2005).
• 32: Sun Microsystems, "Performance Analyzier, Sun Studio 11," Sun Microsystems Manual, (November 2005).
• 33: Barton P. Miller , Mark D. Callaghan , Jonathan M. Cargille , Jeffrey K. Hollingsworth , R. Bruce Irvin , Karen L. Karavanic , Krishna Kunchithapadam , Tia Newhall, The Paradyn Parallel Performance Measurement Tool, Computer, v.28 n.11, p.37-46, November 1995 [doi>10.1109/2.471178]
• 34: Alexander V. Mirgorodskiy and Barton P. Miller, "Crosswalk: a tool for performance profiling across the user-kernel boundary," Parallel Computing Conference, (2003).
• 35: Alessandro Orso, "Selective capture and reply of program executions," WODA 2005, (okscrpe).
• 36: Wim De Pauw , David Lorenz , John Vlissides , Mark Wegman, Execution patterns in object-oriented visualization, Proceedings of the 4th conference on USENIX Conference on Object-Oriented Technologies and Systems, p.16-16, April 27-30, 1998, Santa Fe, New Mexico
• 37: Wim De Pauw, Doug Kimelman, and John Vlissides, "Visualizing object-oriented software execution," pp. 329--346 in Software Visualization: Programming as a Multimedia Experience, ed. Blaine A. Price,MIT Press (1998).
• 38: Wim De Pauw , Gary Sevitsky, Visualizing Reference Patterns for Solving Memory Leaks in Java, Proceedings of the 13th European Conference on Object-Oriented Programming, p.116-134, June 14-18, 1999
• 39: Wim De Pauw, Nick Mitchell, Martin Robillard, Gary Sevitsky, and Harini Srinivasan, "Drive-by analysis of running programs," Proc. ICSE Workshop of Software Visualization, (May 2001).
• 40: David J. Pearce , Matthew Webster , Robert Berry , Paul H. J. Kelly, Profiling with AspectJ, Software—Practice & Experience, v.37 n.7, p.747-777, June 2007 [doi>10.1002/spe.v37:7]
• 41: Steven P. Reiss, Connecting Tools Using Message Passing in the Field Environment, IEEE Software, v.7 n.4, p.57-66, July 1990 [doi>10.1109/52.56450]
• 42: Steven P. Reiss, The Field Programming Environment: A Friendly Integrated Environment for Learning and Development, Kluwer Academic Publishers, Norwell, MA, 1995
• 43: Steven P. Reiss, "Visualization for software engineering - programming environments," in Software Visualization: Programming as a Multimedia Experience, ed. Blaine Price, MIT Press (1997).
• 44: Steven P. Reiss, JIVE: visualizing Java in action demonstration description, Proceedings of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon
• 45: Steven P. Reiss , Manos Renieris, Jove: java as it happens, Proceedings of the 2005 ACM symposium on Software visualization, May 14-15, 2005, St. Louis, Missouri [doi>10.1145/1056018.1056034]
• 46: Philip Charles Roth, "Scalable On-Line Automated Performance Diagnosis," U. Wisconsin-Madison Ph.D. Dissertation, (2005).
• 47: Philip C. Roth , Barton P. Miller, On-line automated performance diagnosis on thousands of processes, Proceedings of the eleventh ACM SIGPLAN symposium on Principles and practice of parallel programming, March 29-31, 2006, New York, New York, USA [doi>10.1145/1122971.1122984]
• 48: Titos Saridakis, Christos Nikolaou, Maria Karavassili, Evangelos Markatos, Apostolos Zarras, and Catherine Houstis, "ArrayTracer: a parallel performance analysis tool," U. Ioannina Tech. Report LYDIA/WP2/T.2.2/D4(b), (1996).
• 49: Arjan Seesing , Alessandro Orso, InsECTJ: a generic instrumentation framework for collecting dynamic information within Eclipse, Proceedings of the 2005 OOPSLA workshop on Eclipse technology eXchange, p.45-49, October 16-17, 2005, San Diego, California [doi>10.1145/1117696.1117706]
• 50: ShiftOne, "JRat: the Java runtime analysis toolkit," http://jrat.sourceforge.net, (May 2005).
• 51: Rebecca Koskela , Margaret Simmons, Parallel computer systems: performance instrumentation and visualization, ACM, New York, NY, 1990
• 52: A. Srivastava, A. Edwards, and H. Vo, "Vulcan: binary transformation in a distributed environment," Microsoft Research Tech Report MSR-TR-2001-50, (2001).
• 53: MIPS Computer Systems, Inc., RISCompiler Languages Programmer's Guide. December 1988.
• 54: Wily Technology, "Introscope and IBM Tivoli," CA Wily White Paper, (2006).
• 55: Sebastien Vauclair, Extensible Java Profiler, Diploma Thesis, EPFL (2002).
• 56: Jeffrey S. Vetter , Daniel A. Reed, Managing performance analysis with dynamic statistical projection pursuit, Proceedings of the 1999 ACM/IEEE conference on Supercomputing (CDROM), p.44-es, November 14-19, 1999, Portland, Oregon, United States [doi>10.1145/331532.331576]
• 57: D. Viswanathan , S. Liang, Java virtual machine profiler interface, IBM Systems Journal, v.39 n.1, p.82-95, January 2000
• 58: Abdul Waheed, Herman D. Hughes, and Diane T. Rover, "A resource occupancy model for evaluating instrumentation system overheads," Michigan State U. Dept. of EE Report TR-MSU-EE-SCSL-023-95, (May 1995).
• 59: John Whaley, A portable sampling-based profiler for Java virtual machines, Proceedings of the ACM 2000 conference on Java Grande, p.78-87, June 03-04, 2000, San Francisco, California, United States [doi>10.1145/337449.337483]
• 60: Brian J. N. Wylie, Bernd Mohr, and Felix Wolf, "Holistic hardware counter performance analysis of parallel programs," Proc. ParCo, (September 2005).
• 61: Benjamin Zorn and Paul Hilfinger, "A memory allocation profiler for C and Lisp programs," Proc. Summer 1988 USENIX Conference, pp. 223--237 (1998).

INDEX TERMS

Primary Classification:
  B. Hardware
  B.8 Performance and Reliability
      B.8.2 Performance Analysis and Design Aids

General Terms:
Measurement, Performance

Keywords:
instrumentation, performance analysis, profiling, statistical analysis