A semantic model of program faults

ABSTRACT

Program faults are artifacts that are widely studied, but there are many aspects of faults that we still do not understand. In addition to the simple fact that one important goal during testing is to cause failures and thereby detect faults, a full understanding of the characteristics of faults is crucial to several research areas in testing. These include fault-based testing, testability, mutation testing, and the comparative evaluation of testing strategies. In this workshop paper, we explore the fundamental nature of faults by looking at the differences between a syntactic and semantic characterization of faults. We offer definitions of these characteristics and explore the differentiation. Specifically, we discuss the concept of "size" of program faults --- the measurement of size provides interesting and useful distinctions between the syntactic and semantic characterization of faults. We use the fault size observations to make several predictions about testing and present preliminary data that supports this model. We also use the model to offer explanations about several questions that have intrigued testing researchers.

REFERENCES

• 1: IEEE Standard Glossary of So/tware Engineering Terminology. ANSI/IEEE Std 729-1983, 1983.
• 2: T. A. Budd and D. Angluin. Two notions of correctness and their relation to testing. Acta informatica, 18(1):31-45, November 1982.
• 3: R. A. DeMillo, D. S. Guindi, K. N. King, W. M. Mc- Cracken, and A. J. Offutt. An extended overview of the Mothra software testing environment. In Proceedings o/ the Second Workshop on Software Testing, Verification, and Analysis, pages 142-151, Banff Alberta, July 1988. IEEE Computer Society Press.
• 4: R. A. DeMillo, R. J. Lipton, and F. G. Sayward. Hints on test data selection: Help for the practicing programmer. IEEE Computer, 11(4):34-41, April 1978.
• 5: Tarak Goradia, Dynamic impact analysis: a cost-effective technique to enforce error-propagation, Proceedings of the 1993 ACM SIGSOFT international symposium on Software testing and analysis, p.171-181, June 28-30, 1993, Cambridge, Massachusetts, United States
• 6: R. Hamlet, Theoretical comparison of testing methods, Proceedings of the ACM SIGSOFT '89 third symposium on Software testing, analysis, and verification, p.28-37, December 13-15, 1989, Key West, Florida, United States
• 7: R. G. Hamlet. Testing programs with the aid of a compiler. IEEE Transactions on Software Engineering, 3(4), July 1977.
• 8: John C. Knight , Paul E. Ammann, An experimental evaluation of simple methods for seeding program errors, Proceedings of the 8th international conference on Software engineering, p.337-342, August 28-30, 1985, London, England
• 9: Scott Meyers, Effective C++: 50 specific ways to improve your programs and designs, Addison Wesley Longman Publishing Co., Inc., Redwood City, CA, 1992
• 10: H. D. Mills. On the statistical validation of computer programs. Technical report FSC 72-6015, IBM, 1972.
• 11: L. J. Morell, A Theory of Fault-Based Testing, IEEE Transactions on Software Engineering, v.16 n.8, p.844-857, August 1990 [doi>10.1109/32.57623]
• 12: A. Jefferson Offutt, Investigations of the software testing coupling effect, ACM Transactions on Software Engineering and Methodology (TOSEM), v.1 n.1, p.5-20, Jan. 1992 [doi>10.1145/125489.125473]
• 13: A. J. Offutt and A. Irvine. Testing object-oriented software using the category-partition method. In Seventeenth International Conj'erence on Technology o/ Object. Oriented Languages and Systems (TOOLS USA '95), Santa Barbara, CA, August 1995.
• 14: Jeff Offutt , Ammei Lee , Gregg Rothermel , Roland Untch , Author:, An Experimental Determination of Sufficient Mutation Operators, George Mason University, Fairfax, VA, 1994
• 15: A. Jefferson Offutt , Gregg Rothermel , Christian Zapf, An experimental evaluation of selective mutation, Proceedings of the 15th international conference on Software Engineering, p.100-107, May 17-21, 1993, Baltimore, Maryland, United States
• 16: K. Tewary and M. J. Harrold. Fault modeling using the program dependence graph. In Proceedings of the Fifth International Symposium on Software Reliability Engineering, pages 126-135, Monterey CA, November 1994.
• 17: Jeffrey Voas , Larry Morrel , Keith Miller, Predicting Where Faults Can Hide from Testing, IEEE Software, v.8 n.2, p.41-48, March 1991 [doi>10.1109/52.73748]
• 18: K. S. H. T. Wah. Fault coupling in finite bijective functions. The Journal of Software Testing, Verification, and Reliability, 5(1):3-47, March 1995.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.0 General
          Subjects: Standards

General Terms:
Design, Theory, Verification