Structural specification-based testing with ADL

ABSTRACT

This paper describes a specification-based black-box technique for testing program units. The main contribution is the method that we have developed to derive test conditions, which are descriptions of test cases, from the formal specification of each program unit. The derived test conditions are used to guide test selection and to measure comprehensiveness of existing test suites. Our technique complements traditional code-based techniques such as statement coverage and branch coverage. It allows the tester to quickly develop a black-box test suite.In particular, this paper presents techniques for deriving test conditions from specifications written in the Assertion Definition Language (ADL) [SH94], a predicate logic-based language that is used to describe the relationships between inputs and outputs of a program unit. Our technique is fully automatable, and we are currently implementing a tool based on the techniques presented in this paper.

REFERENCES

• CHR82: L.A. Clarke, J. Hassell, and D. J. Richardson, "A close look at domain testing." IEEE Trans. Software Eng., vol. SE-8, no. 4, pp. 380-390, July 1982.
• CR83: L.A. Clarke and D. J. Richardson, "A rigorous approach to error-sensitive testing." In Proceedings of the Sixteenth Hawaii International Conference on System Sciences, pp. 197-206, January 1983.
• CSR95: J. Chang, S. Sankar, and D. J. Richardson, "Automated test selection from ADL specifications." In Proceedings of the California Software Symposium (CSS), Irvine, California, March 1995.
• Car81: Robert Cartwright, Formal program testing, Proceedings of the 8th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, p.125-132, January 26-28, 1981, Williamsburg, Virginia [doi>10.1145/567532.567546]
• DS95: R. Doong and S. Sankar, "Specification-based coverage criteria for ADL." Submitted for publication, 1995.
• Fos80: K.A. Foster, "Error sensitive test case analysis (ESTCA)." IEEE Trans. Software Eng., vol. SE-6, pp. 258-264, May 1980.
• Fos84: K. A. Foster, Sensitive test data for logic expressions, ACM SIGSOFT Software Engineering Notes, v.9 n.2, p.120-125, April 1984 [doi>10.1145/1010925.1010935]
• GG76: J.B. Goodenough and S. L. Gerhart, "Toward a theory of test data selection." IEEE Trans. Software Eng., vol. SE-1, no. 2, pp. 156-173, June 1975.
• Gou83: J.S. Gourlay, "A mathematical framework for the investigation of testing." IEEE Trans. Software Eng., vol. SE-9, no. 6, pp. 686-709, November 1983.
• Mye79: Glenford J. Myers, Art of Software Testing, John Wiley & Sons, Inc., New York, NY, 1979
• OB88: T. J. Ostrand , M. J. Balcer, The category-partition method for specifying and generating fuctional tests, Communications of the ACM, v.31 n.6, p.676-686, June 1988 [doi>10.1145/62959.62964]
• RC85: D.J. Richardson and L. A. Clarke, "Partition analysis: a method combining testing and verification." IEEE Trans. Software Eng., vol. SE-11, no. 12, pp. 1477- 1490, December 1985.
• SC93: P. Stocks , D. Carrington, Test template framework: a specification-based testing case study, Proceedings of the 1993 ACM SIGSOFT international symposium on Software testing and analysis, p.11-18, June 28-30, 1993, Cambridge, Massachusetts, United States
• ROT89: D. Richardson , O. O'Malley , C. Tittle, Approaches to specification-based testing, Proceedings of the ACM SIGSOFT '89 third symposium on Software testing, analysis, and verification, p.86-96, December 13-15, 1989, Key West, Florida, United States
• SH94: Roger Hayes , Sriram Sankar, Specifying and Testing Software Components using ADL, Sun Microsystems, Inc., Mountain View, CA, 1994
• WGS94: E. Weyuker , T. Goradia , A. Singh, Automatically Generating Test Data from a Boolean Specification, IEEE Transactions on Software Engineering, v.20 n.5, p.353-363, May 1994 [doi>10.1109/32.286420]
• WO80: E.J. Weyuker and T. J. Ostrand, "Theories of program testing and the application of revealing subdomains." IEEE Trans. Software Eng., vol. SE-6, no. 3, pp. 236- 246, May 1980.
• WC80: L.J. White and E. I. Cohen, "A domain strategy for computer program testing" IEEE Trans. Software Eng., vol. SE-6, no. 3, pp. 247-257, May 1980.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

  F. Theory of Computation
  F.3 LOGICS AND MEANINGS OF PROGRAMS
      F.3.1 Specifying and Verifying and Reasoning about Programs
          Subjects: Specification techniques

General Terms:
Design, Verification