Automated test data generation using an iterative relaxation method

ABSTRACT

An important problem that arises in path oriented testing is the generation of test data that causes a program to follow a given path. In this paper, we present a novel program execution based approach using an iterative relaxation method to address the above problem. In this method, test data generation is initiated with an arbitrarily chosen input from a given domain. This input is then iteratively refined to obtain an input on which all the branch predicates on the given path evaluate to the desired outcome. In each iteration the program statements relevant to the evaluation of each branch predicate on the path are executed, and a set of linear constraints is derived. The constraints are then solved to obtain the increments for the input. These increments are added to the current input to obtain the input for the next iteration. The relaxation technique used in deriving the constraints provides feedback on the amount by which each input variable should be adjusted for the branches on the path to evaluate to the desired outcome.When the branch conditions on a path are linear functions of input variables, our technique either finds a solution for such paths in one iteration or it guarantees that the path is infeasible. In contrast, existing execution based approaches may require an unacceptably large number of iterations for relatively long paths because they consider only one input variable and one branch predicate at a time and use backtracking. When the branch conditions on a path are nonlinear functions of input variables, though it may take more then one iteration to derive a desired input, the set of constraints to be solved in each iteration is linear and is solved using Gaussian elimination. This makes our technique practical and suitable for automation.

REFERENCES

• 1: Jonathan A. Bauer , Alan B. Finger, Test plan generation using formal grammars, Proceedings of the 4th international conference on Software engineering, p.425-432, September 17-19, 1979, Munich, Germany
• 2: David L Bird , Carlos Urias Munoz, Automatic generation of random self-checking test cases, IBM Systems Journal, v.22 n.3, p.229-245, 1983
• 3: C. H. Bischof , L. Roh , A. J. Mauer-Oats, ADIC: an extensible automatic differentiation tool for ANSI-C, Software—Practice & Experience, v.27 n.12, p.1427-1456, Dec. 1997 [doi>10.1002/(SICI)1097-024X(199712)27:12<1427::AID-SPE138>3.3.CO;2-H]
• 4: L.A. Clarke, "A System to Generate Test Data and SymboIicaIIy Execute Programs," IEEE Transactions on Software Engineering, Vol. SE2, No. 3, pages 215-222, September 1976.
• 5: W. H. Deason , D. B. Brown , K. -H. Chang , J. H. Cross II, A Rule-Based Software Test Data Generator, IEEE Transactions on Knowledge and Data Engineering, v.3 n.1, p.108-117, March 1991 [doi>10.1109/69.75894]
• 6: Richard A. DeMillo , A. Jefferson Offutt, Constraint-Based Automatic Test Data Generation, IEEE Transactions on Software Engineering, v.17 n.9, p.900-910, September 1991 [doi>10.1109/32.92910]
• 7: Matthew J. Gallagher , V. Lakshmi Narasimhan, ADTEST: A Test Data Generation Suite for Ada Software Systems, IEEE Transactions on Software Engineering, v.23 n.8, p.473-484, August 1997 [doi>10.1109/32.624304]
• 8: Arnaud Gotlieb , Bernard Botella , Michel Rueher, Automatic test data generation using constraint solving techniques, Proceedings of the 1998 ACM SIGSOFT international symposium on Software testing and analysis, p.53-62, March 02-04, 1998, Clearwater Beach, Florida, United States
• 9: W. H. Jessop , J. R. Kane , S. Roy , J. M. Scanlon, ATLAS-An Automated Software Testing System, Proceedings of the 2nd international conference on Software engineering, p.629-635, October 13-15, 1976, San Francisco, California, United States
• 10: B. Korel, Automated Software Test Data Generation, IEEE Transactions on Software Engineering, v.16 n.8, p.870-879, August 1990 [doi>10.1109/32.57624]
• 11: B. Korel, A Dynamic Approach of Test Data Generation. In Conference on Software Maintenance, pages 311-317, San Diego, CA, November 1990.
• 12: Norman R. Lyons, An automatic data generating system for data base simulation and testing, ACM SIGMIS Database, v.8 n.4, p.10-13, Spring 1977 [doi>10.1145/1017601.1017603]
• 13: E. F. Miller, Jr. , R. A. Melton, Automated generation of testcase datasets, ACM SIGPLAN Notices, v.10 n.6, p.51-58, June 1975
• 14: W. MiIIer and D.L. Spooner, "Automatic Generation of Floating-Point Test Data," IEEE Transactions on Software Engineering, Vol. SE2, No. 3, pages 223-226, September 1976.
• 15: F. Scheid, "Numerical Analysis," Schaum' s Outline Series, McGraw-Hill Book Company, 1968.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

Additional Classification:
  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.3 Numerical Linear Algebra
          Subjects: Linear systems (direct and iterative methods)
  G.4 MATHEMATICAL SOFTWARE
      Subjects: Algorithm design and analysis

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management
          Subjects: Software development

General Terms:
Algorithms, Design, Measurement, Performance, Reliability, Theory

Keywords:
dynamic test data generation, input dependency set, path testing, predicate residuals, predicate sliccs, relaxation methods