Visualization of test information to assist fault localization

ABSTRACT

One of the most expensive and time-consuming components of the debugging process is locating the errors or faults. To locate faults, developers must identify statements involved in failures and select suspicious statements that might contain faults. This paper presents a new technique that uses visualization to assist with these tasks. The technique uses color to visually map the participation of each program statement in the outcome of the execution of the program with a test suite, consisting of both passed and failed test cases. Based on this visual mapping, a user can inspect the statements in the program, identify statements involved in failures, and locate potentially faulty statements. The paper also describes a prototype tool that implements our technique along with a set of empirical studies that use the tool for evaluation of the technique. The empirical studies show that, for the subject we studied, the technique can be effective in helping a user locate faults in a program.

REFERENCES

• 1: xSlice: A tool for program debugging. http://xsuds.argreenhouse.com/html-man/coverpage.html.
• 2: H. Agrawal, J. Horgan, S. London, and W. Wong. Fault localization using execution slices and dataflow tests. In Proceedings of IEEE Software Reliability Engineering, pages 143-151, 1995.
• 3: Thomas Ball , Stephen G. Eick, Software Visualization in the Large, Computer, v.29 n.4, p.33-43, April 1996 [doi>10.1109/2.488299]
• 4: James S. Collofello , Scott N. Woodfield, Evaluating the effectiveness of reliability-assurance techniques, Journal of Systems and Software, v.9 n.3, p.191-195, March 1989 [doi>10.1016/0164-1212(89)90039-3]
• 5: James Eagan , Mary Jean Harrold , James A. Jones , John Stasko, Technical Note: Visually Encoding Program Test Information to Find Faults in Software, Proceedings of the IEEE Symposium on Information Visualization 2001 (INFOVIS'01), p.33, October 22-23, 2001
• 6: Stephen G. Eick , Joseph L. Steffen , Eric E. Sumner, Jr., Seesoft-A Tool for Visualizing Line Oriented Software Statistics, IEEE Transactions on Software Engineering, v.18 n.11, p.957-968, November 1992 [doi>10.1109/32.177365]
• 7: Sebastian Elbaum , Alexey G. Malishevsky , Gregg Rothermel, Prioritizing test cases for regression testing, Proceedings of the 2000 ACM SIGSOFT international symposium on Software testing and analysis, p.102-112, August 21-24, 2000, Portland, Oregon, United States
• 8: Mary Jean Harrold , James A. Jones , Tongyu Li , Donglin Liang , Alessandro Orso , Maikel Pennings , Saurabh Sinha , S. Alexander Spoon , Ashish Gujarathi, Regression test selection for Java software, Proceedings of the 16th ACM SIGPLAN conference on Object oriented programming, systems, languages, and applications, p.312-326, October 14-18, 2001, Tampa Bay, FL, USA
• 9: Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage, Proceedings of the IEEE International Conference on Software Maintenance (ICSM'01), p.92, November 07-09, 2001
• 10: Richard A. DeMillo , Hsin Pan , Eugene H. Spafford, Failure and Fault Analysis for Software Debugging, Proceedings of the 21st International Computer Software and Applications Conference, p.515-521, August 11-15, 1997
• 11: Gregg Rothermel , Mary Jean Harrold, A safe, efficient regression test selection technique, ACM Transactions on Software Engineering and Methodology (TOSEM), v.6 n.2, p.173-210, April 1997 [doi>10.1145/248233.248262]
• 12: Gregg Rothermel , Mary Jean Harrold , Jeffery Ostrin , Christie Hong, An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites, Proceedings of the International Conference on Software Maintenance, p.34, March 16-19, 1998
• 13: Gregg Rothermel , Roland J. Untch , Chengyun Chu, Prioritizing Test Cases For Regression Testing, IEEE Transactions on Software Engineering, v.27 n.10, p.929-948, October 2001 [doi>10.1109/32.962562]
• 14: John T. Stasko , Marc H. Brown , Blaine A. Price, Software Visualization, MIT Press, Cambridge, MA, 1997
• 15: Telcordia Technologies, Inc. xATAC: A tool for improving testing effectiveness. http://xsuds.argreenhouse.com/html-man/coverpage.html.
• 16: I. Vessey. Expertise in debugging computer programs. International Journal of Man-Machine Studies: A process analysis, 23(5):459-494, 1985.
• 17: F. I. Vokolos , P. G. Frankl, Empirical Evaluation of the Textual Differencing Regression Testing Technique, Proceedings of the International Conference on Software Maintenance, p.44, March 16-19, 1998

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Debugging aids

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Tracing

  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.8 Types of Simulation
          Subjects: Visual

General Terms:
Algorithms, Design, Experimentation, Performance