Search-based testing of service level agreements

ABSTRACT

The diffusion of service oriented architectures introduces the need for novel testing approaches. On the one side, testing must be able to identify failures in the functionality provided by service. On the other side, it needs to identify cases in which the Service Level Agreement (SLA) negotiated between the service provider and the service consumer is not met. This would allow the developer to improve service performances, where needed, and the provider to avoid promising Quality of Service (QoS) levels that cannot be guaranteed. This paper proposes the use of Genetic Algorithms to generate inputs and configurations for service-oriented systems that cause SLA violations. The approach has been implemented in a tool and applied to an audio processing workflow and to a service for chart generation. In both cases, the approach was able to produce test data able to violate some QoS constraints.

REFERENCES

• 1: A. Andrieux, K. Czajkowski, A. Dan, K. Keahey, H. Ludwig, J. Pruyne, J. Rofrano, S. Tuecke, and M. Xu. Web Services Agreement Specification (WS-Agreement). http://www.gridforum.org/Meetings/GGF11/Documents/draft-ggf-graap-agreement. pdf last accessed: March 9th 2007.
• 2: Xiaoying Bai , Wenli Dong , Wei-Tek Tsai , Yinong Chen, WSDL-Based Automatic Test Case Generation for Web Services Testing, Proceedings of the IEEE International Workshop, p.215-220, October 20-21, 2005 [doi>10.1109/SOSE.2005.43]
• 3: Luciano Baresi , Carlo Ghezzi , Sam Guinea, Smart monitors for composed services, Proceedings of the 2nd international conference on Service oriented computing, November 15-19, 2004, New York, NY, USA [doi>10.1145/1035167.1035195]
• 4: Antonia Bertolino , Andrea Polini, The Audition Framework for TestingWeb Services Interoperability, Proceedings of the 31st EUROMICRO Conference on Software Engineering and Advanced Applications, p.134-142, August 30-September 03, 2005 [doi>10.1109/EUROMICRO.2005.55]
• 5: Lionel C. Briand , Yvan Labiche , Marwa Shousha, Stress testing real-time systems with genetic algorithms, Proceedings of the 2005 conference on Genetic and evolutionary computation, June 25-29, 2005, Washington DC, USA [doi>10.1145/1068009.1068183]
• 6: M. Bruno, G. Canfora, M. DiPenta, G. Esposito, and V. Mazza. Using test cases as contract to ensure service compliance across releases. In Service-Oriented Computing - ICSOC 2005, Third International Conference, Amsterdam, The Netherlands, December 12-15, 2005, Proceedings pages 87--100, 2005.
• 7: Gerardo Canfora , Massimiliano Di Penta, Testing Services and Service-Centric Systems: Challenges and Opportunities, IT Professional, v.8 n.2, p.10-17, March 2006 [doi>10.1109/MITP.2006.51]
• 8: G. Canfora, M. DiPenta, R. Esposito, F. Perfetto, and M. L. Villani. Service composition (re) binding driven by application-specific qos. In Service-Oriented Computing - ICSOC 2006, 4th International Conference, Chicago, IL, USA, December 4-7, 2006, Proceedings pages 141--152, 2006.
• 9: Gerardo Canfora , Massimiliano Di Penta , Raffaele Esposito , Maria Luisa Villani, An approach for QoS-aware service composition based on genetic algorithms, Proceedings of the 2005 conference on Genetic and evolutionary computation, June 25-29, 2005, Washington DC, USA [doi>10.1145/1068009.1068189]
• 10: J. Cardoso. Quality of Service and Semantic Composition of Workflows PhDthesis, Univ. of Georgia, 2002.
• 11: Chen Fu , Barbara G. Ryder , Ana Milanova , David Wonnacott, Testing of java web services for robustness, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 12: Vahid Garousi , Lionel C. Briand , Yvan Labiche, Traffic-aware stress testing of distributed systems based on UML models, Proceeding of the 28th international conference on Software engineering, May 20-28, 2006, Shanghai, China [doi>10.1145/1134285.1134340]
• 13: Mark Harman , Lin Hu , Rob Hierons , Joachim Wegener , Harmen Sthamer , André Baresel , Marc Roper, Testability Transformation, IEEE Transactions on Software Engineering, v.30 n.1, p.3-16, January 2004 [doi>10.1109/TSE.2004.1265732]
• 14: E. Martin, S. Basu, and T. Xie. Automated robustness testing of web services. In Proceedings of the 4th International Workshop on SOA And Web Services Best Practices (SOAWS 2006) October 2006.
• 15: Phil McMinn, Search-based software test data generation: a survey: Research Articles, Software Testing, Verification & Reliability, v.14 n.2, p.105-156, June 2004 [doi>10.1002/stvr.v14:2]
• 16: Jeff Offutt , Wuzhi Xu, Generating test cases for web services using data perturbation, ACM SIGSOFT Software Engineering Notes, v.29 n.5, September 2004 [doi>10.1145/1022494.1022529]
• 17: Massimo Paolucci , Takahiro Kawamura , Terry R. Payne , Katia P. Sycara, Semantic Matching of Web Services Capabilities, Proceedings of the First International Semantic Web Conference on The Semantic Web, p.333-347, June 09-12, 2002
• 18: W. T. Tsai , Ray Paul , Yamin Wang , Chun Fan , Dong Wang, Extending WSDL to Facilitate Web Services Testing, Proceedings of the 7th IEEE International Symposium on High Assurance Systems Engineering (HASE'02), p.171, October 23-25, 2002
• 19: M. Turner, D. Budgen, and P. Brereton. Turning software into aservice. IEEE Computer 36(10):38--44, 2003.
• 20: J. Wegener, A. Baresel, and H. Sthamer. Evolutionary test environment for automatic structural testing. Information & Software Technology 43(14):841--854, 2001.
• 21: J. Wegener and M. Grochtmann. Testing temporal correctness of real-time systems by means of genetic algorithms. In Quality Week 1997.
• 22: Liangzhao Zeng , Boualem Benatallah , Anne H.H. Ngu , Marlon Dumas , Jayant Kalagnanam , Henry Chang, QoS-Aware Middleware for Web Services Composition, IEEE Transactions on Software Engineering, v.30 n.5, p.311-327, May 2004 [doi>10.1109/TSE.2004.11]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.0 General

General Terms:
Performance, Reliability

Keywords:
quality of service, search-based testing, service level agreement