Experimental comparison of three system test strategies preliminary report

ABSTRACT

This paper reports on some experiments which compare three system test strategies which were applied to artificial software systems. The artificial systems were seeded with known defects and then tested according to the three strategies which involved random system level input, random system level input with defect removal, and modified module coverage. The results suggests that of the three strategies considered here, the modified module coverage exposes a higher number of defects, especially for large systems.

REFERENCES

• 1: Boris Beizer, Software system testing and quality assurance, Van Nostrand Reinhold Co., New York, NY, 1984
• 2: Frederick P. Brooks, Jr., The mythical man-month (anniversary ed.), Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1995
• 3: G. Collins and G. Blay, Structured Systems Development Techniques: Strategic Planning to System Testing, Wiley, New York, 1982.
• 4: P A Currit , M Dyer , H D Mills, Certifying the reliability of software, IEEE Transactions on Software Engineering, v.12 n.1, p.3-11, Jan. 1986
• 5: J. W. Duran and S. C. Ntafos, An evaluation of random testing, IEEE Trans. Software Eng. SE- 10, 1984, pp.438444.
• 6: D. Hamlet, Testing for probable correctness, Proc. Workshop on Software Testing, Banff, Canada, July, 1986, IEEE Computer Society Press, Washington D.C., 1986, pp.92-97.
• 7: Bill Hetzel, The complete guide to software testing (2nd ed.), QED Information Sciences, Inc., Wellesley, MA, 1988
• 8: CORPORATE IEEE, Software engineering standards, IEEE Computer Society Press, Los Alamitos, CA, 1987
• 9: H. D. Mills, Structured programming: retrospect and prospect, IEEE Software, 3 (no. 6), 1986, pp.58-66.
• 10: Glenford J. Myers, Art of Software Testing, John Wiley & Sons, Inc., New York, NY, 1979
• 11: R. M. Poston and M. W. Bruen, Counting down to zero software failures, IEEE Software, 4 (no. 5), 1987, pp. 54-61.
• 12: J. H. Rowland, Artificial systems for software engineering studies, Proc. Workshop on Software Testing, Verification, and Analysis, Banff, Canada, July 1988, IEEE Computer Society Press, Washington, D. C. , 1988, pp. 80-88.
• 13: J. H. Rowland and N. A. Sendek, Software Testing: Introduction to System Level-Complez Application Testing, IBM Corporation, Austin, TX 1986.
• 14: Richard W. Selby , Victor R. Basili , F. Terry Baker, Cleanroom software development: an empirical evaluation, IEEE Transactions on Software Engineering, v.13 n.9, p.1027-1037, September 1, 1987 [doi>10.1109/TSE.1987.233525]
• 15: M. A. Vouk, D. F. McAllister, and K. C. Tai, An experimental evaluation of the effectiveness of random testing of fault-tolerant software, Proc. Workshop on Software Testing, Banff, Canada, July 1986, IEEE Computer Society Press, Washington, D. C., 1986, pp. 74-81.
• 16: E. Yourdon, Managing the structured techniques: 4th edition, Prentice-Hall, Inc., Upper Saddle River, NJ, 1988

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Debugging aids

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS
      B.4.5 Reliability, Testing, and Fault-Tolerance**
          Subjects: Hardware reliability**

  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Validation

General Terms:
Experimentation, Reliability, Verification