The evaluation of program-based software test data adequacy criteria

ABSTRACT

In earlier work, a preliminary set of axioms for software test data adequacy was introduced in order to formalize properties which should be satisfied by any good program-based adequacy criterion. Here, we extend this work by augmenting the set with additional axioms which substantially strengthen the set. In doing so, we rule out several types of unsuitable notions of adequacy.

REFERENCES

• 1: Budd, T.A. "Mutation Analysis: Ideas, Examples. Problems and Prospects.'' In Computer Program Testing, ChandrasekararL and Radicchi, 129-148. North-Holland, New York, 1981.
• 2: Martin D. Davis , Ron Sigal , Elaine J. Weyuker, Computability, complexity, and languages (2nd ed.): fundamentals of theoretical computer science, Academic Press Professional, Inc., San Diego, CA, 1994
• 3: Davis, M.D. and Weyuker, E.J. A formal notion of program-based test data adequacy. Inf. and Control, 56, 1-2 (Jan.-Feb. lc,83), 52-71.
• 4: Martin Davis , Elaine Weyuker, Metric space-based test-base adequacy criteria, The Computer Journal, v.31 n.1, p.17-24, Feb., 1988 [doi>10.1093/comjnl/31.1.17]
• 5: DeMillo, R.A., Lipton, R.}. and Sayward, F.G. Hints on test data selection: Help for the practicing programmer. Computer, 11, 4 (Apr. I978}, 34-41.
• 6: Phyllis G. Frankl, The use of data flow information for the selection and evaluation of software test data, 1987
• 7: FrankI, P.G. and Weyuker, E.J. Data flow testing in the presence of unexecutable paths. In Proc. Workshop on Software Testing (Banff, Alberta, Canada, July 15-17, 1986), 4-13.
• 8: Hamlet, R.G. Testing programs with the aid of a computer. IEEE Trans. Software Eng., SE-3, 4 {July 1977), 279-290.
• 9: Hamlet. R.G. Reliability theory of program testing. Acta Informatica, /6, (19811, 31-43.
• 10: C. A. R. Hoare , I. J. Hayes , He Jifeng , C. C. Morgan , A. W. Roscoe , J. W. Sanders , I. H. Sorensen , J. M. Spivey , B. A. Sufrin, Laws of programming, Communications of the ACM, v.30 n.8, p.672-686, Aug. 1987 [doi>10.1145/27651.27653]
• 11: Iannino, A., Musa, J.D., Okumoto, K. and Littlewood, B. Criteria for software reliability model comparisons. IEEE Trans. Software Eng., SE-IO, 6 (Nov. 1984), 687-691.
• 12: Glenford J. Myers, Softwear Reliability, John Wiley & Sons, Inc., New York, NY, 1976
• 13: Sandra Rapps , Elaine J. Weyuker, Selecting software test data using data flow information, IEEE Transactions on Software Engineering, v.11 n.4, p.367-375, April 1985 [doi>10.1109/TSE.1985.232226]
• 14: Weyuker, E.J. The applicability of program schema results to programs. Int. J. Computer and Information Sci., 8, 5 (Nov. 1979), 387-403.
• 15: Weyuker, E.J. and Ostrand, T.J. Theories of program testing and the application of revealing subdomains. IEEE Trans. Software Eng., SE-6, 3 (May 1980), 236-246.
• 16: E J Weyuker, Axiomatizing software test data adequacy, IEEE Transactions on Software Engineering, v.12 n.12, p.1128-1138, Dec.1986
• 17: E. J. Weyuker, Evaluating Software Complexity Measures, IEEE Transactions on Software Engineering, v.14 n.9, p.1357-1365, September 1988 [doi>10.1109/32.6178]
• 18: Weyuker, E.J. An empirical study of the complexity of data flow testing, In Proc. Second Workshop Testing, Verification and Analysis (Banff. Alberta, Canada, July 19-21, 1988). to appear.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

Additional Classification:
  F. Theory of Computation
  F.3 LOGICS AND MEANINGS OF PROGRAMS

General Terms:
Algorithms, Design, Performance