Synthesizing client load models for performance engineering via web crawling

ABSTRACT

Accurate web application performance testing relies on the use of loading tests based on a realistic client behaviour load model. Unfortunately developing such load models and associated test plans and scripts is tedious and error-prone with most existing web performance testing tools providing limited client load modelling capabilities. We describe a new approach and toolset that we have developed, MaramaMTE+, which improves the ability to model realistic web client load behaviour, automatically generates complex web application testing plans and scripts, and integrates load behaviour modelling with a generic performance engineering tool. MaramaMTE+ uses a stochastic form chart as its client loading model. A 3^rd party web crawler application extracts structural information from a target web site, aggregating the collected data into a crawler database that is then used for form chart model generation. The performance engineer then augments this synthesized form chart with client loading probabilities. Realistic web loading tests for a 3^rd party web load testing tool are then automatically generated from this resultant stochastic form chart client load model. We describe the development of our MaramaMTE+ environment, example usage of the tool, and compare and contrast the results obtained from our generated performance load tests against hand-built 3^rd party tool load tests

REFERENCES

• 1: Apache JMeter: http://jakarta.apache.org/jmeter/index.html
• 2: Paul Barford , Mark Crovella, Generating representative Web workloads for network and server performance evaluation, Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems, p.151-160, June 22-26, 1998, Madison, Wisconsin, United States
• 3: Beyer, D. CCVisu - A Tool for General Force-Directed Graph Layout and Co-Change Visualization, See http://directory.fsf.org/ccvisu.html
• 4: Yuhong Cai , John Grundy , John Hosking, Experiences Integrating and Scaling a Performance Test Bed Generator with an Open Source CASE Tool, Proceedings of the 19th IEEE international conference on Automated software engineering, p.36-45, September 20-24, 2004 [doi>10.1109/ASE.2004.36]
• 5: Dai, X., Grundy, J. and Lo, B.: Comparing and contrasting micro-payment models for Ecommerce systems, International Conferences of Info-tech and Info-net (ICII), China (2001).
• 6: Giovanni Denaro , Andrea Polini , Wolfgang Emmerich, Early performance testing of distributed software applications, Proceedings of the 4th international workshop on Software and performance, January 14-16, 2004, Redwood Shores, California
• 7: Draheim, D. and Weber, G., Modeling Submit/Response Style Systems with Form Charts and Dialogue Constraints, LNCS Volume 2889/2003, Springer.
• 8: Dirk Draheim , John Grundy , John Hosking , Christof Lutteroth , Gerald Weber, Realistic Load Testing ofWeb Applications, Proceedings of the Conference on Software Maintenance and Reengineering, p.57-70, March 22-24, 2006
• 9: Sebastian Elbaum , Srikanth Karre , Gregg Rothermel, Improving web application testing with user session data, Proceedings of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon
• 10: Foy, C. Fitnesse Selenium Wrapper, Saturday, September 16, 2006, http://www.cornetdesign.com/2006/09/fitnesse-selenium-wrapper.html
• 11: John Grundy , John Hosking , Lei Li , Na Liu, Performance engineering of service compositions, Proceedings of the 2006 international workshop on Service-oriented software engineering, May 27-28, 2006, Shanghai, China [doi>10.1145/1138486.1138493]
• 12: John Grundy , John Hosking , Nianping Zhu , Na Liu, Generating Domain-Specific Visual Language Editors from High-level Tool Specifications, Proceedings of the 21st IEEE/ACM International Conference on Automated Software Engineering, p.25-36, September 18-22, 2006 [doi>10.1109/ASE.2006.39]
• 13: Hellsten, C., Automate acceptance tests with Selenium, IBM Developerworks, 20 Dec 2005, See: http://www-128.ibm.com/developerworks/web/library/wa-selenium-ajax/#N100A2
• 14: Java.net, Java Pet Store 2.0 Reference Application, https://blueprints.dev.java.net/petstore/
• 15: Yan Liu , Alan Fekete , Ian Gorton, Design-Level Performance Prediction of Component-Based Applications, IEEE Transactions on Software Engineering, v.31 n.11, p.928-941, November 2005 [doi>10.1109/TSE.2005.127]
• 16: Akhil Mehra , John Grundy , John Hosking, A generic approach to supporting diagram differencing and merging for collaborative design, Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, November 07-11, 2005, Long Beach, CA, USA [doi>10.1145/1101908.1101940]
• 17: Microsoft Download Center, Web Application Stress Tool, See:http://www.microsoft.com/downloads/details.aspx?familyid =e2c0585a-062a-439ea67d75a89aa36495&displaylang=en
• 18: Robert C. Miller , Krishna Bharat, SPHINX: a framework for creating personal, site-specific Web crawlers, Proceedings of the seventh international conference on World Wide Web 7, p.119-130, April 1998, Brisbane, Australia
• 19: Daniel A. Menascé, Load Testing of Web Sites, IEEE Internet Computing, v.6 n.4, p.70-74, July 2002 [doi>10.1109/MIC.2002.1020328]
• 20: Connie U. Smith , Catalina M. Lladó , Vittorio Cortellessa , Antinisca Di Marco , Lloyd G. Williams, From UML models to software performance results: an SPE process based on XML interchange formats, Proceedings of the 5th international workshop on Software and performance, p.87-98, July 12-14, 2005, Palma, Illes Balears, Spain [doi>10.1145/1071021.1071030]
• 21: Sara Sprenkle , Emily Gibson , Sreedevi Sampath , Lori Pollock, Automated replay and failure detection for web applications, Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, November 07-11, 2005, Long Beach, CA, USA [doi>10.1145/1101908.1101947]
• 22: B. M. Subraya , S. V. Subrahmanya, Object Driven Performance Testing in Web Applications, Proceedings of the The First Asia-Pacific Conference on Quality Software (APAQS'00), p.17, October 30-31, 2000

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Performance measures

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: Computer-aided software engineering (CASE)

General Terms:
Management, Measurement, Performance

Keywords:
form chart, web load testing, web user behaviour modelling