How scalable is J2EE technology?

ABSTRACT

ECperf, the widely recognized industry standard J2EE benchmark, has attracted a large number of results submissions and their subsequent publication. However, ECperf places little restriction on the hardware platform, operating systems and databases utilized in the benchmarking process. This, combined with the existence of only two primary metrics, makes it difficult to answer critical questions such as "Is there a limit to J2EE scalability?" and "Is scale-up or scale-out more effective?". By mining the full-disclosure archives for trends and correlations we have discovered that J2EE technology is very scalable, both in a scale-up and scale-out manner. Other observed trends include, a linear correlation between middle-tier total processing power and throughput, as well as between J2EE Application Server license costs and throughput. However, the results clearly indicate that there is an increasing cost per user with increasing capacity systems, and scale-up is proportionately more expensive than scale-out.

REFERENCES

• 1: ECperf Specification and Kit, <u>http://java.sun.com/j2ee/ecperf/</u>
• 2: ECperf results: <u>www.theserverside.com/ecperf</u>
• 3: JSR 4 -- ECperf Benchmark Specification, <u>http://jcp.org/en/jsr/detail?id=004</u>
• 4: J2EE compatibility, <u>http://java.sun.com/j2ee/compatibility.html</u>
• 5: JBoss ECperf kit from CSIRO, <u>http://www.cmis.csiro.au/adsat/jbossecperf.htm</u>
• 6: Paul Brebner et al, Evaluating J2EE Application Servers, CSIRO MTE Report, CSIRO Publishing and Cutter Consortium, 2002: <u>http://www.cmis.csiro.au/adsat/j2eev2.htm</u>
• 7: Nile benchmark, <u>http://www.gotdotnet.com/team/compare/Nile%20Benchmark%20 Results.doc</u>
• 8: NASDAQ transaction rate, <u>http://www.nasdaqnews.com/news/pr2002/ne_section02_165.html</u>
• 9: TPC-C analysis, <u>http://www.benchmarkresources.com/handbook/chapter12.pdf</u>
• 10: Gunther, N, Commerical Clusters and Scalability, at: <u>http://www.teamquest.com/html/gunther/scalability.shtml</u>
• 11: Brebner, P, "Is your AppServer being Crippled by the JVM?", in proceedings of: BorCon2002, 5th Annual Asia Pacific Borland Conference, Australia, Sydney.
• 12: SPEC ™ Information: <u>http://www.spec.org/spec/trademarks.html</u>
• 13: SPECjAppServer2001: <u>http://www.spec.org/osg/jAppServer2001/</u>
• 14: Samuel D. Kounev and Alejandro P. Buchmann, Implementing and Optimising Sun's ECperf Benchmark with WebLogic Server, BEA Technology Conference, 14 November 2001, Frankfurt: <u>http://de.bea.com/events/techkonf2001/presentations/pdf/hik_2c.pdf</u>
• 15: Samuel D. Kounev, Eliminating ECperf Persistence Bottlenecks when using RDBMS with Pessimistic Concurrency Control, <u>http://www.dvs1.informatik.tu-darmstadt.de/DVS1/staff/skounev/pub/ecperf1.0-opt-proposal.pdf</u>
• 16: Samuel D. Kounev and Alejandro P. Buchmann, Performance Issues in E-Business Systems, in: SSGRR 2002W (International Conference on Advances in Infrastructure for e-Business, e-Education, e-Science, e-Medicine, and Mobile Technologies on the Internet): <u>http://www.ssgrr.it/en/ssgrr2002w/papers/1.pdf</u>
• 17: Samuel D. Kounev and Alejandro P. Buchmann, Improving Data Access of J2EE Applications by Exploiting Asynchronous Messaging and Caching Services, in: VLDB 2002: <u>http://www.cs.ust.hk/vldb2002/VLDB2002-papers/S16P03.pdf</u>
• 18: Samuel D. Kounev, Eliminating ECperf Persistence Bottlenecks when using RDBMS with Pessimistic Concurrency Control, <u>http://www.dvs1.informatik.tu-darmstadt.de/DVS1/staff/skounev/pub/ecperf1.0-opt-proposal.pdf</u>
• 19: Ian Gorton , Sean Baker, Enterprise Transaction Processing Systems: Putting the Cobra Ots, Encina++ and Orbixotm to Work, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2000
• 20: Paul Brebner , Shuping Ran, Entity Bean A, B, C's: Enterprise Java Beans Commit Options and Caching, Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg, p.36-55, November 12-16, 2001
• 21: SPEC: <u>http://www.benchmarkresources.com/handbook/chapter9.pdf</u>
• 22: ECperf: <u>http://www.jinspired.com/products/jdbinsight/ecperfuncovered.pdf</u>
• 23: Shanti Subramanyam, Ramesh Ramachandran and Akara Sucharitakul, Performance Analysis and Tuning of the Java™ 2 Platform, Enterprise Edition (J2EE™), Java One, 2001 and 2002, San Francisco.
• 24: Ricardo Morin, Hardeep Reehal, and Kumar Shiv, Java™ 2 Platform, Enterprise Edition (J2EE) Performance Evaluation Workloads--Analysis and Lessons Learned, JavaOne 2002, San Francisco
• 25: Yan Zhang , Anna Liu , Wei Qu, Comparing industry benchmarks for J2EE application server: IBM's trade2 vs Sun's ECperf, Proceedings of the twenty-sixth Australasian conference on Computer science: research and practice in information technology, p.199-206, February 01, 2003, Adelaide, Australia
• 26: EJB Benchmarking: <u>http://nenya.ms.mff.cuni.cz/~prochazk/papers/TREJBComp.pdf</u>
• 27: 2002 WebSphere eBU (Trade2, ECperf and other benchmarks): <u>http://www-900.ibm.com/developerWorks/cn/wsdd/download/pdf/ws-ebu/scene4-perfscal-ch.pdf</u>
• 28: Martin Karlsson, Kevin E. Moore, Erik Hagersten, and David A. Wood, Memory Characterization of the ECperf Benchmark, <u>http://www.cs.wisc.edu/multifacet/papers/WMPI02_ECperfSEQ1 2.pdf</u>
• 29: Shuping Ran , Paul Brebner , Ian Gorton, The Rigorous Evaluation of Enterprise Java Bean Technology, Proceedings of the The 15th International Conference on Information Networking, p.93, January 31-February 02, 2001
• 30: Ian Gorton , Anna Liu , Paul Brebner, Rigorous Evaluation of COTS Middleware Technology, Computer, v.36 n.3, p.50-55, March 2003 [doi>10.1109/MC.2003.1185217]

INDEX TERMS

Keywords:
Enterprise Java Beans (EJB), J2EE, scalability, scale-out, scale-up