Approaches to specification-based testing

ABSTRACT

Current software testing practices focus, almost exclusively, on the implementation, despite widely acknowledged benefits of testing based on software specifications. We propose approaches to specification-based testing by extending a wide variety of implementation-based testing techniques to be applicable to formal specification languages. We demonstrate these approaches for the Anna and Larch specification languages.

REFERENCES

• BCFG86: L Bougé , N Choquet , L Fribourg , M C Gaudel, Test sets generation from algebraic specifications using logic programming, Journal of Systems and Software, v.6 n.4, p.343-360, Nov. 1986 [doi>10.1016/0164-1212(86)90004-X]
• CPRZ89: L. A. Clarke , A. Podgurski , D. J. Richardson , S. J. Zeil, A Formal Evaluation of Data Flow Path Selection Criteria, IEEE Transactions on Software Engineering, v.15 n.11, p.1318-1332, November 1989 [doi>10.1109/32.41326]
• CR83: Lori A. Clarke and Debra J. Richardson. "A Rigorous Approach to Error-Sensitive Testing". In Proceedings of the Sixteenth Hawaii International Conference on System Sciences, pages 197-206, January 1983.
• CRZ88: Lori A. Clarke , Debra J. Richardson , Steven J. Zeil, TEAM: a support environment for testing, evaluation, and analysis, Proceedings of the third ACM SIGSOFT/SIGPLAN software engineering symposium on Practical software development environments, p.153-162, November 28-30, 1988, Boston, Massachusetts, United States
• DLS78: Richard DeMillo, R.J. Lipton, and F.G. Sayward. Hints on test data selection: help for the practicing programmer. Computer, 4(11), April 1978.
• GB83: Ajei Gopal and Tim Budd. "Program Testing by Specification Mutation". Technical Report TR 83-17, University of Arizona, November 1983.
• GG75: J.B. Goodenough and S.L. Gerhart. "Toward a Theory of Test Data Selection". IEEE Transactions on Software Engineering, SE-1(2):156-173, June 1975.
• GHM87: J. D. Gannon , R. G. Hamlet , H. D. Mills, Theory of modules, IEEE Transactions on Software Engineering, v.13 n.7, p.820-829, July 1987 [doi>10.1109/TSE.1987.233493]
• GHW85: John Guttag, James Horning, and Jeannette Wing. "The Larch Family of Specification Languages". IEEE Transactions on Software Engineering, pages 24-36, September 1985.
• GM88: M.-C. Gaudel and B. Marre. "Algebraic Specifications and Software Testing: Theory and Application". In Rapport LRI #407, 1988.
• GMH81: John Gannon , Paul McMullin , Richard Hamlet, Data Abstraction, Implementation, Specification, and Testing, ACM Transactions on Programming Languages and Systems (TOPLAS), v.3 n.3, p.211-223, July 1981 [doi>10.1145/357139.357140]
• Gou83: John S. Gourlay. "A Mathematical Framework for the Investigation of Testing". IEEE Transactions on Software Engineering, SE-9(6):686-709, November 1983.
• GT79: Joseph A. Goguen and Joseph J. Tardo. "An Introduction to OBJ: A Language for Writing and Testing Formal Algebraic Program Specifications". In IEEE Conference on Specification of Reliable Software, pages 170-188, 1979.
• Ham77: Richard G. Hamlet. Testing programs with the aid of a compiler. IEEE %ansactions on Software Engineering, SE-3(4):279-290, July 1977.
• How78: William E. Howden. "Introduction to the Theory of Testing". In Edward Miller and William E. Howden, editors, ntorial: Software Testing and Validation Techniques, pages 16-19. IEEE, New York, 1978.
• How86: W E Howden, A functional approach to program testing and analysis, IEEE Transactions on Software Engineering, v.12 n.10, p.997-1005, Oct. 1986
• Kemm85: Richard A. Kemmerer. "Testing Formal Specifications to Detect Design Errors'. IEEE Transactions on Software Engineering, SEll(l), January 1985.
• L+84: David C. Luckham , Friedrich W. Henke , Bernd Krieg-Brueckner , Olaf Owe, ANNA: a language for annotating ADA programs, Stanford University, Stanford, CA, 1984
• Las88: Janusz Laski. "Testing in Top-Down Program Development". In Second Workshop on Software Testing, Verification, and Analysis, pages 72-79, July 1988.
• LvH85: David C. Luckham and EYiedrich W. van Henke. "An Overview of Anna, a Specification Langage for Ada". IEEE Software, pages 9-22, March 1985.
• Mye79: Glenford J. Myers, Art of Software Testing, John Wiley & Sons, Inc., New York, NY, 1979
• OB88: T. J. Ostrand , M. J. Balcer, The category-partition method for specifying and generating fuctional tests, Communications of the ACM, v.31 n.6, p.676-686, June 1988 [doi>10.1145/62959.62964]
• OSW86: Thomas J. Ostrand, Ron Sigal, and Elaine J. Weyuker. "Design for a Tool to Manage Specification-Based Testing". IEEE Transactions on Software Engineering, SE 12:41-50, 1986.
• RC81: Debra J. Richardson , Lori A. Clarke, A partition analysis method to increase program reliability, Proceedings of the 5th international conference on Software engineering, p.244-253, March 09-12, 1981, San Diego, California, United States
• RC85a: Debra J. Richardson and Lori A. Clarke. "Partition Analysis: A Method Combining Testing and Verification". IEEE Transactions on Software Engineering, SE-11(12):1477- 1490, December 1985.
• RC85b: D J Richardson , L A Clarke, Testing techniques based on symbolic evaluation, Proceedings of Centre for Software Reliability Workshop on Software: requirements, specification and testing, p.93-110, June 1986, Univ. of East Anglia, Norwich, United Kingdom
• Rea89: Reasoning Systems, Palo Alto, California. Refine User's Guide, 1989.
• RT86: Debra J. Richardson , Margaret C. Thompson, A New Model for Error Detection, University of Massachusetts, Amherst, MA, 1986
• RT88: Debra Richardson and Margaret Thompson. "The RE- LAY Model of Error Detection". In Second Workshop on Software Testing, Verijication and Analysis. IEEE CS Software Engineering Technical Committee, July 1988.
• Vel87: F. R. D. Velasco, A method for test data selection, Journal of Systems and Software, v.7 n.2, p.89-97, June 1, 1987 [doi>10.1016/0164-1212(87)90013-6]
• WC80: L.J. White and E.I. Cohen. "A Domain Strategy for Computer Program Testing". IEEE Transactions on Software Engineering, SE6(3):247-257, May 1980.
• Win83: Jeannette Wing. "A Two-Tiered Approach to Specifying Programs". PhD thesis, Massachussetts Institute of Technology, 1983.
• WO80: E.J. Wevuker and T.J. Ostrand. "Theories of Program Testing and the Application of Revealing Subdomains". IEEE Transactions on Software Engineering, SE-6(3):236-246, May 1980.

INDEX TERMS

Primary Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: Larch

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.1 Requirements/Specifications
          Subjects: Languages
          Nouns: ANNA
      D.2.4 Software/Program Verification
          Subjects: Validation

General Terms:
Languages, Reliability, Verification