Efficient unit test case minimization

ABSTRACT

Randomized unit test cases can be very effective in detecting defects. In practice, however, failing test cases often comprise long sequences of method calls that are tiresome to reproduce and debug. We present a combination of static slicing and delta debugging that automatically minimizes the sequence of failure-inducing method calls. In a case study on the EiffelBase library, the strategy minimizes failing unit test cases on average by 96%.

This approach improves on the state of the art by being far more efficient: in contrast to the approach of Lei and Andrews, who use delta debugging alone, our case study found slicing to be 50 times faster, while providing comparable results. The combination of slicing and delta debugging gives the best results and is 11 times faster.

REFERENCES

• 1: Hiralal Agrawal , Joseph R. Horgan, Dynamic program slicing, ACM SIGPLAN Notices, v.25 n.6, p.246-256, Jun. 1990
• 2: Hiralal Agrawal , Joseph R. Horgan , Edward W. Krauser , Saul London, Incremental Regression Testing, Proceedings of the Conference on Software Maintenance, p.348-357, September 01, 1993
• 3: Neelam Gupta , Haifeng He , Xiangyu Zhang , Rajiv Gupta, Locating faulty code using failure-inducing chops, Proceedings of the 20th IEEE/ACM international Conference on Automated software engineering, November 07-11, 2005, Long Beach, CA, USA [doi>10.1145/1101908.1101948]
• 4: GUPTA, R., HARROLD, M. J., AND SOFFA, M. L. Program slicing-based regression testing techniques. Softw. Test, Verif. Reliab 6, 2 (1996), 83--111.
• 5: Bogdan Korel , Janusz Laski, Dynamic slicing of computer programs, Journal of Systems and Software, v.13 n.3, p.187-195, Nov. 1990 [doi>10.1016/0164-1212(90)90094-3]
• 6: Yong Lei , James H. Andrews, Minimization of Randomized Unit Test Cases, Proceedings of the 16th IEEE International Symposium on Software Reliability Engineering, p.267-276, November 08-11, 2005 [doi>10.1109/ISSRE.2005.28]
• 7: MEYER, B., CIUPA, I., LEITNER, A., AND LIU, L. L. Automatic testing of object-oriented software. In Proceedings of SOFSEM 2007 (Current Trends in Theory and Practice of Computer Science) (2007), J. van Leeuwen, Ed., Lecture Notes in Computer Science, Springer-Verlag.
• 8: Alessandro Orso , Shrinivas Joshi , Martin Burger , Andreas Zeller, Isolating relevant component interactions with JINSI, Proceedings of the 2006 international workshop on Dynamic systems analysis, May 23-23, 2006, Shanghai, China [doi>10.1145/1138912.1138915]
• 9: Mark Weiser, Programmers use slices when debugging, Communications of the ACM, v.25 n.7, p.446-452, July 1982 [doi>10.1145/358557.358577]
• 10: Andreas Zeller , Ralf Hildebrandt, Simplifying and Isolating Failure-Inducing Input, IEEE Transactions on Software Engineering, v.28 n.2, p.183-200, February 2002 [doi>10.1109/32.988498]
• 11: Xiangyu Zhang , Haifeng He , Neelam Gupta , Rajiv Gupta, Experimental evaluation of using dynamic slices for fault location, Proceedings of the sixth international symposium on Automated analysis-driven debugging, p.33-42, September 19-21, 2005, Monterey, California, USA [doi>10.1145/1085130.1085135]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

General Terms:
Experimentation, Verification

Keywords:
minimization, slicing, unit testing