A real-time benchmark for Java™

ABSTRACT

The introduction of pause time sensitive garbage collectors in various Java™ Standard Edition runtime implementations and the availability of real-time Java implementations based on Real-Time Specification for Java (RTSJ, aka JSR-1) has opened the door to using the Java programming language and runtime system for both soft and hard real-time applications. However, there are few general benchmarks to assess the real-time qualities of these implementations, leaving the developer and other product evaluators at the mercy of potentially biased and/or misleading marketing literature and propaganda.

In this paper, we propose a standardized benchmark for real-time performance of various Java implementations, ranging from Java SE implementations with low pause time oriented garbage collectors to full RTSJ implementations providing hard real-time capabilities. This benchmark is based on the SPECjbb2005 benchmark, but is modified to be more aligned with the needs of soft and hard real-time applications. In this paper, we refer to this SPECjbb2005 variant as SPECjbb2005rt, though the fully recognize that this is not an official SPEC® benchmark and might not get such a name even if it becomes a SPEC benchmark. This benchmark provides both throughput and response time metrics, providing product evaluators a mechanism to judge the real-time qualities of competing Java Virtual Machine and Real-time Operating System implementations.

We will describe this proposed benchmark and how it differs from the standard SPECjbb2005 benchmark. We will also describe how the benchmark works with standard Java threads versus realtime Java threads. We will then present some results from different product offerings to demonstrate the usefulness of the benchmark. In accordance with SPEC fair use policy^[1] the vendors and product names, the command line options, and any other identifying characteristics of specific implementations are not revealed here as the purpose of this paper is to demonstrate the usefulness of the benchmark and not competitive positioning of the product offerings.

REFERENCES

• 1: http://www.spec.org/jbb2005

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.8 Performance

Additional Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.5 Object-oriented Programming

General Terms:
Design, Measurement, Performance, Reliability

Keywords:
Java, RTSJ, SPECjbb2005, benchmarks, real-time