Predicting the cost-effectiveness of regression testing strategies

ABSTRACT

Selective regression testing strategies aim at choosing an appropriate subset of test cases from among a previously run test suite for a software system, based on information about the changes made to the system to create new versions. Although there has been a significant amount of research in recent years on the design of such strategies, there has been significantly less investigation of their cost-effectiveness. In this paper some computationally efficient predictors of the cost-effectiveness of the two main classes of selective regression testing approaches are presented. A case study is described in which these predictors are used to assess the appropriateness of using a particular regression testing strategy to test multiple versions of a widely-used software system.

REFERENCES

• 1: P. Benedusi, A. Cimitile, and U. DeCarlini. Postmaintenance Testing Based on Path Change Analysis. In Proc. Conf. Software Maintenance 1988, Phoenix, Oct. 1988, pp. 352-361.
• 2: Morris, I. Bolsky , David G. Korn, The KornShell command and programming language, Prentice-Hall, Inc., Upper Saddle River, NJ, 1989
• 3: Yih-Farn Chen , David S. Rosenblum , Kiem-Phong Vo, TestTube: a system for selective regression testing, Proceedings of the 16th international conference on Software engineering, p.211-220, May 16-21, 1994, Sorrento, Italy
• 4: K.F. Fischer. A Test Case Selection Method for the Validation of Software Maintenance Modifications. In Proc. COMPSA C '77, Nov. 1977, pp. 421-426.
• 5: M.J. Harrold and M.L. Soffa. An Incremental Approach to Unit Testing during Maintenance. In Proc. Conf. Software Maintenance 1988, Phoenix, Oct. 1988, pp. 362- 367.
• 6: Jean Hartmann , David J. Robson, Techniques for Selective Revalidation, IEEE Software, v.7 n.1, p.31-36, January 1990 [doi>10.1109/52.43047]
• 7: D. G. Kern and K.-P. Vo. SFIO: Safe/fast string/file IO. In Proc. Summer 1991 Useniz Conf., pages 235- 256. USENIX Association, June 1991.
• 8: H.K.N. Leung and L. White. Insights into Regression Testing. In ProcTOC. Conf. Software Maintenance 1989, Miami, oct. 1989, pp. 60-69.
• 9: H. K. N. Leung , L. White, Insights into testing and regression testing global variables, Journal of Software Maintenance: Research and Practice, v.2 n.4, p.209-222, Dec. 1990 [doi>10.1002/smr.4360020403]
• 10: H.K.N. Leung and L. White. A Cost Model to Compare Regression Test Strategies. In Proc. Conf. Software Maintenance 1991, Sorrento, Italy, Oct. 1991, pp. 201- 208.
• 11: T.J. Ostrand and E.J. Weyuker. Using Data Flow Analysis for Regression Testing. In Proc. Sixth Annual Pacific Northwest Software Quality Conf., Portland, Or, Sep. 1988, pp. 233-247.
• 12: Gregg Rothermel , Mary Jean Harrold, A framework for evaluating regression test selection techniques, Proceedings of the 16th international conference on Software engineering, p.201-210, May 16-21, 1994, Sorrento, Italy
• 13: A. Taha, S.M. Thebaut, and S-S. Liu. An Approach to Software Fault Localization and Revalidation Based on Incremental Data Flow Analysis. In Proc. COMPSA C89, 1989, psp. 527-534.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.4 Software/Program Verification
          Subjects: Validation

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement
          Subjects: Corrections**

General Terms:
Algorithms, Design, Reliability, Verification

Keywords:
cost estimation, regression testing, software analysis, test coverage