Investigations of the software testing coupling effect

ABSTRACT

Fault-based testing strategies test software by focusing on specific, common types of faults. The coupling effect hypothesizes that test data sets that detect simple types of faults are sensitive enough to detect more complex types of faults. This paper describes empirical investigations into the coupling effect over a specific class of software faults. All of the results from this investigation support the validity of the coupling effect. The major conclusion from this investigation is the fact that by explicitly testing for simple faults, we are also implicitly testing for more complicated faults, giving us confidence that fault-based testing is an effective way to test software.

REFERENCES

• 1: ~ACREE, A, T., BUDD, T. A., DEMILLO, R. A., LIPTON, R. J., AND SAYWARD, F. G. Mutation ~analysis Tech. Rep. GIT-ICS-79/08, School of Information and Computer Science, Georgia ~Inst. of Technology, At}anta, Ga., Sept. 1979.
• 2: Timothy Alan Budd, Mutation analysis of program test data, 1980
• 3: ~BUDD, T. A., AND ANGLUIN, D. Two notions of correctness and their relation to testing. ~Acta I,f 18, 1 (Nov. 1982), 31-45.
• 4: ~cRAFT, W.M. Detecting equivalent mutants using compiler optimization techniqnes. M.S. ~thesis, Dept. of Computer Science, Clemson Univ., Clemson, S.C , 1989.
• 5: ~DEMILLO, R. A., GUINI)I, D. S., KING, K. l~., MCCRACKEN, W. M., AND OFFUTT, A.J. An ~extended overview of the Mothra software testing environment. In Proceedings of the Second ~Workshop on Software Testing, Verification, and Analysis (Banff, Alberta, July 1988), IEEE ~Computer Society Press, pp. 142-151.
• 6: ~DEMILLO, R. A., LIPTON, R. J., AND SAYWARD, F.G. Hints on test data selection: Help for ~the practicing programmer. IEEE Computer 11, 4 (Apr. 1978), 34-41.
• 7: Richard A. DeMillo , A. Jefferson Offutt, Constraint-Based Automatic Test Data Generation, IEEE Transactions on Software Engineering, v.17 n.9, p.900-910, September 1991 [doi>10.1109/32.92910]
• 8: ~GmGIS, M. R., AND WOODWARD, M.R. An experimental comparison of the error exposing ~ability of program testing criteria. In Proceedings of the Workshop on Software Testing (July ~1986), IEEE Computer Society Press, 64-73.
• 9: C. A. R. Hoare, Proof of a program: FIND, Communications of the ACM, v.14 n.1, p.39-45, Jan. 1971 [doi>10.1145/362452.362489]
• 10: ~HOWDEN, W.E. Reliability of the path analysis testing strategy. IEEE Trans. Softw. Eng. ~2, 3 (Sept 1976), 208-215
• 11: ~HOWDEN, W.E. Weak mutation testing and completeness of test sets. IEEE Trans. Softw. ~Eng. 8, 4 (July 1982), 371-379.
• 12: William E Howden, Functional program testing and analysis, McGraw-Hill, Inc., New York, NY, 1986
• 13: K. N. King , A. Jefferson Offutt, A Fortran language system for mutation-based software testing, Software—Practice & Experience, v.21 n.7, p.685-718, July 1991 [doi>10.1002/spe.4380210704]
• 14: ~LIPTON, R. J, AND SAYWARD, F. G. The status of research on program mutation. In Digest ~for the Workshop on Software Testing and Test Documentation (Dec. 1978), pp. 355-373.
• 15: ~MARICK, B. Two experiments in software testing. Tech. Rep. UIUCDCS-R-90-1644, Dept. of ~Computer Science, Univ. of Illinois at Urbana-Champaign, Nov. 1990.
• 16: Larry Joe Morell, A theory of error-based testing, 1983
• 17: ~MORELL, L. J. Theoretical insights into fault-based testing. In Proceedings of the Second ~Workshop on Software Testing, Verificatwn, and Analysis (Banff, Alberta, July 1988), IEEE ~Computer Society Press, pp. 45-62.
• 18: A. Offutt, The coupling effect: fact or fiction, Proceedings of the ACM SIGSOFT '89 third symposium on Software testing, analysis, and verification, p.131-140, December 13-15, 1989, Key West, Florida, United States
• 19: ~OFFUTT, A.J. An integrated automatic test data generation system. J. Syst. Integration 1, ~3(1991), 397-409.
• 20: ~RICHARDSON, I). J., AND THOMSPON, M C. The relay model for error detection and its ~apphcation. In Proceec!mgs of the Second Workshop on Software Testing, Verification, and ~Analysis (Banff, Alberta, July 1988) IEEE Computer Society Press, pp. 223-230.
• 21: Elaine J. Weyuker, Assessing Test Data Adequacy through Program Inference, ACM Transactions on Programming Languages and Systems (TOPLAS), v.5 n.4, p.641-655, Oct. 1983 [doi>10.1145/69575.357231]
• 22: ~WEYUKER, E. J., AND OSTRAND, T. J. Theories of program testing and the application of ~revealing subdomams. IEEE Trans. Softw, Eng. 6, 3 (May 1980), 236-246

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Reliability

General Terms:
Experimentation, Reliability, Verification

Keywords:
fault-based testing, mutation, software testing, unit testing