A framework and a tool for robustness testing of communicating software

ABSTRACT

Robustness testing aims at verifying the acceptable behavior of a system under unexpected conditions. In this paper we propose a framework and a tool for robustness test cases generation. Our framework consists of two phases: (1) Construction of an increased specication by integrating hazards in the nominal specification model written in SDL. The rule of the increased specification is to specify the acceptable behavior in presence of hazards. (2) A specific method to generate robustness test cases (in TTCN-3) from the increased specification and a robustness test purpose. We also give some experimental results on the TCP protocol.

REFERENCES

• 1: J. H. Barton , E. W. Czeck , Z. Z. Segall , D. P. Siewiorek, Fault Injection Experiments Using FIAT, IEEE Transactions on Computers, v.39 n.4, p.575-582, April 1990 [doi>10.1109/12.54853]
• 2: G.-J. Carrette. CRASHME: Random input testing. http://people.delphiforums.com/gjc/crashme.html, 1996.
• 3: R. CASTANET and H. WAESELYNK. Techniques avancées de test de systèmes complexes: Test de robustesse. Technical report, Action spécifique 23 du CNRS, 11 2003.
• 4: J. DeVale, P. Koopman, and D. Guttendorf. The ballista software robustness testing service. In Testing Computer Software Conference (TCSC99), June 1999.
• 5: D. Estrin, S. Gupta, and A. Helmy. STRESS: Systematic testing of robustness by evaluation of synthesized scenarios. http://netweb.usc.edu/stress/, 1998.
• 6: IEEE. Information technology - Open Systems Interconnection - Conformance testing methodology and framework - Part 3: The Tree and Tabular Combined Notation (TTCN), number 9646 in 2, 2003.
• 7: ITU-T. Specification and description language (sdl). ITU-T Recommendation no Z.105, International Telecommunication Union. Genève, 1999.
• 8: T. Jéron, C. Jard, C. Viho, B. Caillaud, H. Kahlouche, P. Morel, J.-C. Fernandez, A. Kerbrat, and M. Bozga. Génération automatique de tests pour les protocoles: l'exemple de l'approche formelle de TGV. Revue de l'Electricité et de l'Electronique (REE), 3, March 1999.
• 9: I. S. G. of Software Engineering Terminology 610.12-1990. Customer and terminology standards. In IEEE Standards Software Engineering, IEEE Press, 1, 1999.
• 10: J. Postel. Transmission control protocol. IETF, RFC793, September 1981.
• 11: J. Röning, M. Laakso, and A. Takanen. PROTOS - systematic approach to eliminate software vulnerabilities. http://www.ee.oulu.fi/research/ouspg, May 2002. 2002.
• 12: F. Saad-khorchef, I. Berrada, A. Rollet, and R. Castanet. Automated robustness testing for reactive systems: Application to communicating protocols. In 6th International Workshop on Innovative Internet Community Systems (12CS 2006), Neuchâtel, Switzerland, June 26--28 2006.
• 13: Jan Tretmans, Test Generation with Inputs, Outputs, and Quiescence, Proceedings of the Second International Workshop on Tools and Algorithms for Construction and Analysis of Systems, p.127-146, March 27-29, 1996
• 14: J. Tretmans and E. Brinksma. TorX: Automated model based testing. In First European Conference on Model-Driven Software Engineering, 2003.
• 15: I.-T. R. Z.140--142. The testing and test control notation", version 3 (ttcn-3), rec. z.140: Ttcn-3 core language, rec. z.141: Tabular presentation format for ttcn-3 (tft), rec. z.142: Graphical presentation format for ttcn-3 (gft). ITU-T, Geneva (Switzerland), 2002.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

Keywords:
IOLTS, TCP protocol, formal testing, robustness testing