How java programs interact with virtual machines at the microarchitectural level

ABSTRACT

Java workloads are becoming increasingly prominent on various platforms ranging from embedded systems, over general-purpose computers to high-end servers. Understanding the implications of all the aspects involved when running Java workloads, is thus extremely important during the design of a system that will run such workloads. In other words, understanding the interaction between the Java application, its input and the virtual machine it runs on, is key to a succesful design. The goal of this paper is to study this complex interaction at the microarchitectural level, e.g., by analyzing the branch behavior, the cache behavior, etc. This is done by measuring a large number of performance characteristics using performance counters on an AMD K7 Duron microprocessor. These performance characteristics are measured for seven virtual machine configurations, and a collection of Java benchmarks with corresponding inputs coming from the SPECjvm98 benchmark suite, the SPECjbb2000 benchmark suite, the Java Grande Forum benchmark suite and an open-source raytracer, called Raja with 19 scene descriptions. This large amount of data is further analyzed using statistical data analysis techniques, namely principal components analysis and cluster analysis. These techniques provide useful insights in an understandable way.From our experiments, we conclude that (i) the behavior observed at the microarchitectural level is primarily determined by the virtual machine for small input sets, e.g., the SPECjvm98 s1 input set; (ii) the behavior can be quite different for various input sets, e.g., short-running versus long-running benchmarks; (iii) for long-running benchmarks with few hot spots, the behavior can be primarily determined by the Java program and not the virtual machine, i.e., all the virtual machines optimize the hot spots to similarly behaving native code; (iv) in general, the behavior of a Java application running on one virtual machine can be significantly different from running on another virtual machine. These conclusions warn researchers working on Java workloads to be careful when using a limited number of Java benchmarks or virtual machines since this might lead to biased conclusions.

REFERENCES

• 1: Advanced Micro Devices, Inc. AMD Athlon Processor x86 Code Optimization Guide, February 2002. http://www.amd.com
• 2: B. Alpern , C. R. Attanasio , J. J. Barton , M. G. Burke , P. Cheng , J.-D. Choi , A. Cocchi , S. J. Fink , D. Grove , M. Hind , S. F. Hummel , D. Lieber , V. Litvinov , M. F. Mergen , T. Ngo , J. R. Russell , V. Sarkar , M. J. Serrano , J. C. Shepherd , S. E. Smith , V. C. Sreedhar , H. Srinivasan , J. Whaley, The Jalapeño virtual machine, IBM Systems Journal, v.39 n.1, p.211-238, January 2000
• 3: Matthew Arnold , Stephen Fink , David Grove , Michael Hind , Peter F. Sweeney, Adaptive optimization in the Jalapeño JVM, Proceedings of the 15th ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications, p.47-65, October 2000, Minneapolis, Minnesota, United States
• 4: Todd Austin , Eric Larson , Dan Ernst, SimpleScalar: An Infrastructure for Computer System Modeling, Computer, v.35 n.2, p.59-67, February 2002 [doi>10.1109/2.982917]
• 5: S. J. Baylor , M. Devarakonda , S. J. Fink , E. Gluzberg , M. Kalantar , P. Muttineni , E. Barsness , R. Arora , R. Dimpsey , S. J. Munroe, Java server benchmarks, IBM Systems Journal, v.39 n.1, p.57-81, January 2000
• 6: BEASystems, Inc. BEA Weblogic JRockit -- The Server JVM: Increasing Server-side Performance and Manageability, August 2002. http://www.bea.com/products/weblogic/jrockit
• 7: Pradip Bose, Performance evaluation and validation of microprocessors, Proceedings of the 1999 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.226-227, May 01-04, 1999, Atlanta, Georgia, United States
• 8: K.R. Bowers and D. Kaeli. Characterizing the SPECjvm98 ben hmarks on the Java Virtual Machine. Te hnical Report ECE-CEG-98-026, Northeastern University, Department of Electrical and Computer Engineering, September 1998.
• 9: J.M. Bull, L.A. Smith, M.D. Westhead, D.S. Henty, and R.A.Davey.A benchmark suite for high performance Java. Concurrency,Practice and Experience, 12(6):375--388, May 2000.
• 10: K. Chow, A. Wright, and K. Lai. Characterization of Java workloads by principal components analysis and indirect branches. In Proceedings of the Workshop on Workload Characterization (WWC-1998), held in conjunction with he 31st Annual ACM/IEEE International Symposium on Microarchitecture (MICRO-31), pages 11--19, November 1998.
• 11: Rajagopalan Desikan , Doug Burger , Stephen W. Keckler, Measuring Experimental Error in Microprocessor Simulation, Proceedings of the 28th annual international symposium on Computer architecture, p.266-277, June 30-July 04, 2001, Göteborg, Sweden [doi>10.1145/379240.565338]
• 12: K. Diefendorff. K7 challenges Intel. Microprocessor Report, 12(14), October 1998.
• 13: Lieven Eeckhout , Hans Vandierendonck , Koen De Bosschere, Designing Computer Architecture Research Workloads, Computer, v.36 n.2, p.65-71, February 2003 [doi>10.1109/MC.2003.1178050]
• 14: L. Eeckhout, H. Vandierendonck, and K.De Bosschere. Quantifying the impact of input data sets on program behavior and its applications. Journal of Instruction-Level Parallelism, 5:1--33, February 2003. http://www.jilp.org/vol5
• 15: C.-H. A. Hsieh , M. T. Conte , T. L. Johnson , J. C. Gyllenhaal , W.-M. W. Hwu, A study of the cache and branch performance issues with running Java on current hardware platforms, Proceedings of the 42nd IEEE International Computer Conference, p.211, February 23-26, 1997
• 16: Cheng-Hsueh A. Hsieh , John C. Gyllenhaal , Wen-mei W. Hwu, Java bytecode to native code translation: the caffeine prototype and preliminary results, Proceedings of the 29th annual ACM/IEEE international symposium on Microarchitecture, p.90-99, December 02-04, 1996, Paris, France
• 17: R. A. Johnson , D. W. Wichern, Applied multivariate statistical analysis, Prentice-Hall, Inc., Upper Saddle River, NJ, 1988
• 18: Tao Li , Lizy Kurian John , Anand Sivasubramaniam , N. Vijaykrishnan , Juan Rubio, Understanding and improving operating system effects in control flow prediction, Proceedings of the 10th international conference on Architectural support for programming languages and operating systems, October 05-09, 2002, San Jose, California
• 19: Tao Li , Lizy Kurian John , Vijaykrishnan Narayanan , Anand Sivasubramaniam , Jyotsna Sabarinathan , Anupama Murthy, Using complete system simulation to characterize SPECjvm98 benchmarks, Proceedings of the 14th international conference on Supercomputing, p.22-33, May 08-11, 2000, Santa Fe, New Mexico, United States [doi>10.1145/335231.335234]
• 20: R. Radhakrishnan, N. Vijaykrishnan, L.K. John, and A. Sivasubramaniam. Architectural issues in java runtime systems. In Proceedings of the Sixth International Symposium on High Performance Computer Archicecture (HPCA-6),pages 387--398, January 2000.
• 21: Ramesh Radhakrishnan , N. Vijaykrishnan , Lizy Kurian John , Anand Sivasubramaniam , Juan Rubio , Jyotsna Sabarinathan, Java Runtime Systems: Characterization and Architectural Implications, IEEE Transactions on Computers, v.50 n.2, p.131-146, February 2001 [doi>10.1109/12.908989]
• 22: Mendel Rosenblum , Edouard Bugnion , Scott Devine , Stephen A. Herrod, Using the SimOS machine simulator to study complex computer systems, ACM Transactions on Modeling and Computer Simulation (TOMACS), v.7 n.1, p.78-103, Jan. 1997 [doi>10.1145/244804.244807]
• 23: Yefim Shuf , Mauricio J. Serrano , Manish Gupta , Jaswinder Pal Singh, Characterizing the memory behavior of Java workloads: a structured view and opportunities for optimizations, Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.194-205, June 2001, Cambridge, Massachusetts, United States
• 24: StatSoft, Inc. STATISTICA for Windows. Computer program manual. 1999. http://www.statsoft.com 1999.
• 25: T. Suganuma , T. Ogasawara , M. Takeuchi , T. Yasue , M. Kawahito , K. Ishizaki , H. Komatsu , T. Nakatani, Overview of the IBM Java just-in-time compiler, IBM Systems Journal, v.39 n.1, p.175-193, January 2000
• 26: Sun Microsystems, Inc. The Java HotSpot Virtual Machine, v1.4.1, September 2002.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Design studies

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Performance attributes; Measurement techniques

General Terms:
Experimentation, Measurement, Performance

Keywords:
Java workloads, performance analysis, statistical data analysis, virtual machine technology, workload characterization