Test case prioritization for regression testing of service-oriented business applications

ABSTRACT

Regression testing assures the quality of modified service-oriented business applications against unintended changes. However, a typical regression test suite is large in size. Earlier execution of those test cases that may detect failures is attractive. Many existing prioritization techniques order test cases according to their respective coverage of program statements in a previous version of the application. On the other hand, industrial service-oriented business applications are typically written in orchestration languages such as WS-BPEL and integrated with workflow steps and web services via XPath and WSDL. Faults in these artifacts may cause the application to extract wrong data from messages, leading to failures in service compositions. Surprisingly, current regression testing research hardly considers these artifacts. We propose a multilevel coverage model to capture the business process, XPath, and WSDL from the perspective of regression testing. We develop a family of test case prioritization techniques atop the model. Empirical results show that our techniques can achieve significantly higher rates of fault detection than existing techniques.

REFERENCES

• 1: BPWS4J: a Platform for Creating and Executing BPEL4WS Processes, Version 2.1. IBM, 2002. Available at http://www.alphaworks.ibm.com/tech/bpws4j.
• 2: W. K. Chan, S. C. Cheung, and K. R. P. H. Leung. A metamorphic testing approach for online testing of service-oriented software applications. International Journal of Web Services Research, 4 (2): 60--80, 2007.
• 3: Huo Yan Chen , T. H. Tse , T. Y. Chen, TACCLE: a methodology for object-oriented software testing at the class and cluster levels, ACM Transactions on Software Engineering and Methodology (TOSEM), v.10 n.1, p.56-109, Jan. 2001 [doi>10.1145/366378.366380]
• 4: Hyunsook Do , Gregg Rothermel , Alex Kinneer, Empirical Studies of Test Case Prioritization in a JUnit Testing Environment, Proceedings of the 15th International Symposium on Software Reliability Engineering, p.113-124, November 02-05, 2004 [doi>10.1109/ISSRE.2004.18]
• 5: Sebastian Elbaum , Alexey G. Malishevsky , Gregg Rothermel, Test Case Prioritization: A Family of Empirical Studies, IEEE Transactions on Software Engineering, v.28 n.2, p.159-182, February 2002 [doi>10.1109/32.988497]
• 6: S.-S. Hou, L. Zhang, T. Xie, and J.-S. Sun. Quota-constrained test-case prioritization for regression testing of service-centric systems. In Proceedings of the IEEE International Conference on Software Maintenance (ICSM 2008), pages 257--266. IEEE Computer Society Press, Los Alamitos, CA, 2008.
• 7: Monica Hutchins , Herb Foster , Tarak Goradia , Thomas Ostrand, Experiments of the effectiveness of dataflow- and controlflow-based test adequacy criteria, Proceedings of the 16th international conference on Software engineering, p.191-200, May 16-21, 1994, Sorrento, Italy
• 8: H. K. N. Leung and L. J. White. Insights into regression testing. In Proceedings of the IEEE International Conference on Software Maintenance (ICSM 1989), pages 60--69. IEEE Computer Society Press, Los Alamitos, CA, 1989.
• 9: Zheng Li , Mark Harman , Robert M. Hierons, Search Algorithms for Regression Test Case Prioritization, IEEE Transactions on Software Engineering, v.33 n.4, p.225-237, April 2007 [doi>10.1109/TSE.2007.38]
• 10: E. Martin, S. Basu, and T. Xie. Automated testing and response analysis of Web services. In Proceedings of the IEEE International Conference on Web Services (ICWS 2007), pages 647--654. IEEE Computer Society Press, Los Alamitos, CA, 2007.
• 11: Lijun Mei , W. K. Chan , T. H. Tse, An Adaptive Service Selection Approach to Service Composition, Proceedings of the 2008 IEEE International Conference on Web Services, p.70-77, September 23-26, 2008 [doi>10.1109/ICWS.2008.22]
• 12: Lijun Mei , W.K. Chan , T.H. Tse, Data flow testing of service-oriented workflow applications, Proceedings of the 30th international conference on Software engineering, May 10-18, 2008, Leipzig, Germany [doi>10.1145/1368088.1368139]
• 13: Gerome Miklau , Dan Suciu, Containment and equivalence for a fragment of XPath, Journal of the ACM (JACM), v.51 n.1, p.2-45, January 2004 [doi>10.1145/962446.962448]
• 14: Oliver Moser , Florian Rosenberg , Schahram Dustdar, Non-intrusive monitoring and service adaptation for WS-BPEL, Proceeding of the 17th international conference on World Wide Web, April 21-25, 2008, Beijing, China [doi>10.1145/1367497.1367607]
• 15: Akira K. Onoma , Wei-Tek Tsai , Mustafa Poonawala , Hiroshi Suganuma, Regression testing in an industrial environment, Communications of the ACM, v.41 n.5, p.81-86, May 1998 [doi>10.1145/274946.274960]
• 16: Oracle BPEL Process Manager. Oracle Technology Network. Available at http://www.oracle.com/technology/products/ias/bpel/. Last access February 8, 2009.
• 17: Sandra Rapps , Elaine J. Weyuker, Selecting Software Test Data Using Data Flow Information, IEEE Transactions on Software Engineering, v.11 n.4, p.367-375, April 1985 [doi>10.1109/TSE.1985.232226]
• 18: Gregg Rothermel , Mary Jean Harrold, Analyzing Regression Test Selection Techniques, IEEE Transactions on Software Engineering, v.22 n.8, p.529-551, August 1996 [doi>10.1109/32.536955]
• 19: Gregg Rothermel , Roland J. Untch , Chengyun Chu, Prioritizing Test Cases For Regression Testing, IEEE Transactions on Software Engineering, v.27 n.10, p.929-948, October 2001 [doi>10.1109/32.962562]
• 20: Michael E. Ruth , Shengru Tu, Towards automating regression test selection for web services, Proceedings of the 16th international conference on World Wide Web, May 08-12, 2007, Banff, Alberta, Canada [doi>10.1145/1242572.1242799]
• 21: Travel handling. BPEL Repository. IBM, 2006 Available at http://www.alphaworks.ibm.com/tech/bpelrepository.
• 22: Web Services Business Process Execution Language Version 2.0. OSAIS. Available at http://www.oasis-open.org/committees/wsbpel/. Last access February 8, 2009.
• 23: Web Services Description Language (WSDL) 1.1. W3C, 2001. Available at http://www.w3.org/TR/wsdl.
• 24: Web Services Invocation Framework: DSL Provider Sample Application. Apache Software Foundation, 2006. Available at http://ws.apache.org/wsif/wsif_samples/index.html.
• 25: XML Path Language (XPath) Recommendation. W3C, 2007. Available at http://www.w3.org/TR/xpath20/.
• 26: Chunyang Ye , S. C. Cheung , W. K. Chan, Publishing and composition of atomicity-equivalent services for B2B collaboration, Proceedings of the 28th international conference on Software engineering, May 20-28, 2006, Shanghai, China [doi>10.1145/1134285.1134335]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Product metrics

General Terms:
Measurement, Reliability, Verification

Keywords:
WSDL, XPath, service orientation, test case prioritization