Benchmarking Java against C and Fortran for scientific applications

ABSTRACT

Increasing interest is being shown in the use of Java for scientific applications. The Java Grande benchmark suite [4] was designed with such applications primarily in mind. The perceived lack of performance of Java still deters many potential users, despite recent advances in just-in-time (JIT) and adaptive compilers. There are however few benchmark results available comparing Java to more traditional languages such as C and Fortran. To address this issue, a subset of the Java Grande Benchmarks have been re-written in C and Fortran allowing direct performance comparisons between the three languages. The performance of a range of Java execution environments, C and Fortran compilers have been tested across a number of platforms using the suite. These demonstrate that on some platforms (notably Intel Pentium) the performance gap is now quite small.

REFERENCES

• 1: C. A. Addison , V. S. Getov , A. J. G. Hey , R. W. Hockney , I. C. Wolton, The GENESIS distributed-memory benchmarks, Computer benchmarks, Elsevier Science Publishers B. V., Amsterdam, The Netherlands, 1993
• 2: Mark Baker , Bryan Carpenter , Geoffrey Fox , Sung Hoon Ko , Sang Lim, MPIJAVA: An Object-Oriented JAVA Interface to MPI, Proceedings of the 11 IPPS/SPDP'99 Workshops Held in Conjunction with the 13th International Parallel Processing Symposium and 10th Symposium on Parallel and Distributed Processing, p.748-762, April 12-16, 1999
• 3: Bell, D. (1997) Make Java fast: Optimize!, Java World, vol. 2, no. 4, April 1997, www. j avaworld, com/j avaworld/j w-04-1997/ jw-O4-optimize, him1
• 4: Bull, J.M., Smith, L.A., Westhead, M.D., Henty, D.S. and Davey, R.A. (2000) A Benchmark Suite for High Performance Java, Concurrency, Practice and Experience, vol. 12, pp. 375-388.
• 5: J. M. Bull , L. A. Smith , M. D. Westhead , D. S. Henty , R. A. Davey, A methodology for benchmarking Java Grande applications, Proceedings of the ACM 1999 conference on Java Grande, p.81-88, June 12-14, 1999, San Francisco, California, United States [doi>10.1145/304065.304103]
• 6: Caromel, D., Doyon, F., Klauser, W. and Vayssiere, J. (1998) A distributed raytracer for benchmarking Java RMI and Serialization, www. inria, fr/sloop/C3D/
• 7: Jack J. Dongarra, Performance of Various Computers Using Standard Linear Equations Software, University of Tennessee, Knoxville, TN, 1989
• 8: Getov, V.S. The ParkBench single-processor low-level benchmarks in Java, available from perun, hscs. wmin. ac. uk/CSPE/s oftware, html
• 9: Griswold, W. and Phillips, P. (1997) UCSD Benchmarks for Java, w#w- cso. ucsd. edu/users/wgg/JavaProf/ javaprof, html
• 10: Hardwick, J. Java Mierobenehmarks, wwv. cs. cmu. edu/~ j ch/j ava/benchmarks, html
• 11: Hockney, R.W. (1992) A Framework for Benchmark Performance Analysis, Supercomputer, vol. 48, no. IX(2), pp. 9-22.
• 12: Jacob, M., Philippsen M. and Karrenbach, M. (1998) Large-scale parallel geophysical algorithms in Java: a .feasibility study, Concurrency: Practice and Experience, vol. 10, nos. 11-13, pp. 1143-1154.
• 13: Java Grande Forum, www. javagrande, org.
• 14: M. E. Kambites, J. Obdrzalek and J. M. Bull (2001) An OpenMP-like Interface for Parallel Programming in Java, to appear in Concurrency and Computation: Practice and Experience.
• 15: Lai, X., Massey J.L., and Murphy, S. (1992) Markov ciphers and di~erential cryptanalysis, in Advances in Cryptology--Eurocrypt '91, pp. 17-38, Springer-Verlag.
• 16: J. A. Mathew , P. D. Coddington , K. A. Hawick, Analysis and development of Java Grande benchmarks, Proceedings of the ACM 1999 conference on Java Grande, p.72-80, June 12-14, 1999, San Francisco, California, United States [doi>10.1145/304065.304101]
• 17: Pendragon Software Corp. Caffieine Mark 3.0, www.webfayre, com/pendragon/cm3 /
• 18: Pozo, 1%. Java SciMark benchmark for scientific computing, math. nist. gov/s cimark2
• 19: Richter, H. BenchBeans: A Benchmark for Java Applets, user. cs .tu-berlin.de/Nmondraig/ english/benchbeans, html
• 20: Roulo, M. (1998) Accelerate your Java apps/ JavaWorld, vol. 3, no. 9, Sept. 1998, available from www. j avaworld, com/j avaworld/jw-09-1998/ j w-09-speed, html
• 21: SPEC, SPEC JVM98 Benchmarks, www. spec. org/osg/jvm98/
• 22: Trom, J. The Fhourstones 2.0 Benchmark, ww-w. cwi. nl/--~tromp/c4/fhour, html
• 23: Zachmann, G. Java/C++ Benchmark, www. igd. fhg. de/--~zach/benchmarks/ java-vs-c++, html

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: Java

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Scientific databases

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.2 Installation Management
          Subjects: Benchmarks

General Terms:
Design, Experimentation, Performance

Keywords:
C, Fortran, Java, benchmarking, performance, scientific applications