Traffic-aware stress testing of distributed systems based on UML models

ABSTRACT

A stress test methodology aimed at increasing chances of discovering faults related to network traffic in distributed systems is presented. The technique uses the UML 2.0 model of the distributed system under test, augmented with timing information, and is based on an analysis of the control flow in sequence diagrams. It yields stress test requirements that are made of specific control flow paths along with time values indicating when to trigger them. Different variants of our stress testing technique already exist (they stress different aspects of a distributed system) and we focus here on one variant that is designed to identify and to stress test the system at the instant when data traffic on a network is maximal. Using a real-world distributed system specification, we design and implement a prototype distributed system and describe, for that particular system, how the stress test cases are derived and executed using our methodology. The stress test results indicate that the technique is significantly more effective at detecting network traffic-related faults when compared to test cases based on an operational profile.

REFERENCES

• 1: Alberto Avritzer , Elaine J. Weyuker, The Automatic Generation of Load Test Suites and the Assessment of the Resulting Software, IEEE Transactions on Software Engineering, v.21 n.9, p.705-716, September 1995 [doi>10.1109/32.464549]
• 2: Borland Corporation, "Borland Delphi," www.borland.com/delphi, 2005.
• 3: L. Briand and Y. Labiche, "A UML-based Approach to System Testing," Journal of Software and Systems Modeling, vol. 1, no. 1, pp. 10--42, 2002.
• 4: Lionel C. Briand , Yvan Labiche , Marwa Shousha, Stress testing real-time systems with genetic algorithms, Proceedings of the 2005 conference on Genetic and evolutionary computation, June 25-29, 2005, Washington DC, USA [doi>10.1145/1068009.1068183]
• 5: T. Brown, A. Pasetti, W. Pree, T. A. Henzinger, and C. M. Kirsch, "A Reusable and Platform-independent Framework for Distributed Control Systems," Proc. of Digital Avionics Systems Conference, pp. 1--11, 2001.
• 6: R. J. A. Buhr, Use Case Maps as Architectural Entities for Complex Systems, IEEE Transactions on Software Engineering, v.24 n.12, p.1131-1155, December 1998 [doi>10.1109/32.738343]
• 7: BWI Co., "ElipseSCADA," http://www.bwi.com/proot/2775, 2004.
• 8: Derek Coleman , Patrick Arnold , Stephanie Bodoff , Chris Dollin , Helena Gilchrist , Fiona Hayes , Paul Jeremaes, Object-oriented development: the fusion method, Prentice-Hall, Inc., Upper Saddle River, NJ, 1994
• 9: Z. Constantinescu, P. Petrovic, A. Pedersen, D. Federici, and J. Campos, "QADPZ (Quite Advanced Distributed Parallel Zystem)," in http://qadpz.sourceforge.net, 2003.
• 10: A. Daneels and W. Salter, "What is SCADA?," Proc. of Int. Conf. on Accelerator and Large Experimental Physics Control Systems, pp. 339--343, 1999.
• 11: European Information Society Technologies, "Component Based Open Source Architecture for Distributed Telecom Applications," in http://coach.objectweb.org, 2003.
• 12: R. Fletcher, Practical methods of optimization; (2nd ed.), Wiley-Interscience, New York, NY, 1987
• 13: V. Garousi, L. Briand, and Y. Labiche, "Control Flow Analysis of UML 2.0 Sequence Diagrams," Proceedings of the European Conference on Model Driven Architecture-Foundations and Applications, LNCS 3748, pp. 160--174, 2005.
• 14: V. Garousi, L. Briand, and Y. Labiche, "Traffic-aware Model-based Stress Testing of Distributed Systems," Technical Report SCE-05-13, Carleton University, http://www.sce.carleton.ca/squall/pubs/tech_report/TR_SCE-05-13.pdf, 2005.
• 15: Hassan Gomaa, Designing Concurrent, Distributed, and Real-Time Applications with Uml, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2000
• 16: D. Richard Kuhn, Sources of Failure in the Public Switched Telephone Network, Computer, v.30 n.4, p.31-36, April 1997 [doi>10.1109/2.585151]
• 17: S. Mackay, E. Wright, and J. Park, Practical Data Communications for Instrumentation and Control: Newnes, June, 2003.
• 18: Steven S. Muchnick, Advanced compiler design and implementation, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 19: J. D. Musa, "The Operational Profile in Software Reliability Engineering: An Overview," Proc. of Int. Symp. on Software Reliability Engineering, 1992.
• 20: Clémentine Nebut , Franck Fleurey , Yves Le Traon , Jean-Marc Jézéquel, Requirements by Contracts allow Automated System Testing, Proceedings of the 14th International Symposium on Software Reliability Engineering, p.85, November 17-21, 2003
• 21: Object Management Group (OMG), "OCL 2.0 Specification," 2005.
• 22: Object Management Group (OMG), "UML 2.0 Superstructure Specification," 2005.
• 23: Object Management Group (OMG), "UML 2.0 Testing Profile Specification," 2003.
• 24: Object Management Group (OMG), "UML Profile for Schedulability, Performance, and Time (v1.0)," 2003.
• 25: Tom Pender, UML Bible, John Wiley & Sons, Inc., New York, NY, 2003
• 26: B. Stojkovic and I. Vujosevic, "A compact SCADA system for a smaller size electric power system control-a fast, object-oriented and cost-effective approach," IEEE Power Engineering Society Winter Meeting, pp. 695--700, 2002.
• 27: N. Toshida, M. Uesugi, Y. Nakata, M. Nomoto, and T. Uchida, "Open Distributed EMS/SCADA System," Hitachi Review, vol. 47, no. 5, pp. 208--213, 1998.
• 28: Jeffrey J. P. Tsai , Yaodong Bi , Steve J. H. Yang , Ross A. W. Smith, Distributed real-time systems: monitoring, visualization, debugging, and analysis, John Wiley & Sons, Inc., New York, NY, 1996
• 29: Ryuji Wakizono , Toshikazu Kawamura , Takehiko Tsuchiya , Takahiro Hatanaka , Tatsuji Tanaka, Object-oriented database management system for process control systems—development and evaluation, Proceedings of the 1999 ACM symposium on Applied computing, p.204-209, February 28-March 02, 1999, San Antonio, Texas, United States [doi>10.1145/298151.298245]
• 30: Elaine J. Weyuker , Filippos I. Vokolos, Experience with Performance Testing of Software Systems: Issues, an Approach, and Case Study, IEEE Transactions on Software Engineering, v.26 n.12, p.1147-1156, December 2000 [doi>10.1109/32.888628]
• 31: C. S. D. Yang, "Identifying Potentially Load Sensitive Code Regions for Stress Testing," Proc. of Mid-Atlantic Student Workshop on Programming Languages and Systems, 1996.
• 32: Jian Zhang , S. C. Cheung, Automated test case generation for the stress testing of multimedia systems, Software—Practice & Experience, v.32 n.15, p.1411-1435, December 2002 [doi>10.1002/spe.487]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems

General Terms:
Algorithms, Reliability, Verification

Keywords:
UML, distributed systems, model-based testing, network traffic, stress testing