Java support for data-intensive systems: experiences building the telegraph dataflow system

ABSTRACT

Database system designers have traditionally had trouble with the default services and interfaces provided by operating systems. In recent years, developers and enthusiasts have increasingly promoted Java as a serious platform for building data-intensive servers. Java provides a number of very helpful language features, as well as a full run-time environment reminiscent of a traditional operating system. This combination of features and community support raises the question of whether Java is better or worse at supporting data-intensive server software than a traditional operating system coupled with a weakly-typed language such as C or C++.In this paper, we summarize and discuss our experience building the Telegraph dataflow system in Java. We highlight some of the pleasures of coding with Java, and some of the pains of coding around Java in order to obtain good performance in a data-intensive server. For those issues that were painful, we present concrete suggestions for evolving Java's interfaces to better suit serious software systems development. We believe these experiences can provide insight for other designers to avoid pitfalls we encountered and to decide if Java is a suitable platform for their system.

REFERENCES

• 1: IBM alphaWorks Home Page. http://www.alphaworks.ibm.com/, 2001.
• 2: Java 2 Platform API Specification, JDK 1.4. http://java.sun.com/j2se/1.4/docs, 2001.
• 3: OptimizeIt Home Page. http://www.optimizeit.com/, 2001.
• 4: PostgreSQL. http://www.postgresql.com/, 2001.
• 5: VTune Tool Home Page. http://developer.intel.com/vtune/analyzer/, 2001.
• 6: Ron Avnur , Joseph M. Hellerstein, Eddies: continuously adaptive query processing, Proceedings of the 2000 ACM SIGMOD international conference on Management of data, p.261-272, May 15-18, 2000, Dallas, Texas, United States
• 7: M. K. Bergman. The Deep Web: Surfacing Hidden Value, White Paper. http://www.brightplanet.com/deepcontent/index.asp/, September 2001.
• 8: B. N. Bershad , S. Savage , P. Pardyak , E. G. Sirer , M. E. Fiuczynski , D. Becker , C. Chambers , S. Eggers, Extensibility safety and performance in the SPIN operating system, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.267-283, December 03-06, 1995, Copper Mountain, Colorado, United States
• 9: G. Czajkowski, T. Mayr, P. Seshadri, and T. von Eicken. Resource Control for Java Database Extensions. 5th USENIX Conference on Object-Oriented Technologies and Systems, May 1999.
• 10: O. Doederlein. The Java Performance Report - Part III. http://www.javalobby.org/fr/-html/frm/javalobby/features/-jpr/part3.html, September 2000.
• 11: Michael Godfrey , Tobias Mayr , Praveen Seshadri , Thorsten von Eicken, Secure and portable database extensibility, Proceedings of the 1998 ACM SIGMOD international conference on Management of data, p.390-401, June 01-04, 1998, Seattle, Washington, United States
• 12: Jim Gray , Andreas Reuter, Transaction Processing: Concepts and Techniques, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1992
• 13: Hamilton, James. Personal Communication. Feb 2001.
• 14: Allan Heydon , Marc Najork, Performance limitations of the Java core libraries, Proceedings of the ACM 1999 conference on Java Grande, p.35-41, June 12-14, 1999, San Francisco, California, United States [doi>10.1145/304065.304092]
• 15: Hong, Wei. Personal Communication. Feb 2001.
• 16: IBM DB2. IBM DB2 Reference Manual. Version 6.1.
• 17: M. Frans Kaashoek , Dawson R. Engler , Gregory R. Ganger , Hector M. Briceño , Russell Hunt , David Mazières , Thomas Pinckney , Robert Grimm , John Jannotti , Kenneth Mackenzie, Application performance and flexibility on exokernel systems, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.52-65, October 05-08, 1997, Saint Malo, France
• 18: D. Kegel. The C10K Problem. http://www.kegel.com/c10k.html, 2001.
• 19: Butler W. Lampson , David D. Redell, Experience with processes and monitors in Mesa, Communications of the ACM, v.23 n.2, p.105-117, Feb. 1980 [doi>10.1145/358818.358824]
• 20: C. Mohan , Don Haderle , Bruce Lindsay , Hamid Pirahesh , Peter Schwarz, ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging, ACM Transactions on Database Systems (TODS), v.17 n.1, p.94-162, March 1992 [doi>10.1145/128765.128770]
• 21: C. Mohan , Frank Levine, ARIES/IM: an efficient and high concurrency index management method using write-ahead logging, Proceedings of the 1992 ACM SIGMOD international conference on Management of data, p.371-380, June 02-05, 1992, San Diego, California, United States
• 22: D. C. Schmidt. Reactor: An Object Behavioral Pattern for Demultiplexing and Dispatching Handles for Synchronous Events. Pattern Languages of Programs Conference, August 1994.
• 23: Michael Stonebraker, Operating system support for database management, Communications of the ACM, v.24 n.7, p.412-418, July 1981 [doi>10.1145/358699.358703]
• 24: M. Welsh. Non-blocking I/O for Java. http://www.cs.berkeley.edu/ mdw/proj/java-nbio, September 2001.
• 25: M. Welsh and D. Culler. Jaguar: Enabling Efficient Communication and I/O in Java. Concurrency Practice and Experience, Dec 1999.
• 26: Matt Welsh , David Culler , Eric Brewer, SEDA: an architecture for well-conditioned, scalable internet services, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 27: N. Wyatt and A. Carlson. Cloudscape 3.6, A Technical Overview. A Cloudscape White paper, March 2001. see http://www.cloudscape.com/products/whitepapers.jsp.