AFID: an automated fault identification tool

ABSTRACT

We present the Automatic Fault IDentification Tool (AFID). AFID automatically constructs repositories of real software faults by monitoring the software development process. AFID records both a fault revealing test case and a faulty version of the source code for any crashing faults that the developer discovers and a fault correcting source code change for any crashing faults that the developer corrects. The test cases are a significant contribution, because they enable new research that explores the dynamic behaviors of the software faults.

AFID uses a ptrace-based monitoring mechanism to monitor both the compilation and execution of the application. The ptrace-based technique makes it straightforward for AFID to support a wide range of programming languages and compilers. Our benchmark results indicate that the monitoring overhead will be acceptable for most developers. We performed a short case study to evaluate how effectively the AFID tool records software faults. In our case study, AFID recorded 12 software faults from the 8 participants.

REFERENCES

• 1: Jong-Deok Choi , Harini Srinivasan, Deterministic replay of Java multithreaded applications, Proceedings of the SIGMETRICS symposium on Parallel and distributed tools, p.48-59, August 03-04, 1998, Welches, Oregon, United States [doi>10.1145/281035.281041]
• 2: Ben Collins-Sussman, The subversion project: buiding a better CVS, Linux Journal, v.2002 n.94, p.3, February 2002
• 3: Valentin Dallmeier , Thomas Zimmermann, Extraction of bug localization benchmarks from history, Proceedings of the twenty-second IEEE/ACM international conference on Automated software engineering, November 05-09, 2007, Atlanta, Georgia, USA [doi>10.1145/1321631.1321702]
• 4: Hyunsook Do , Sebastian Elbaum , Gregg Rothermel, Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact, Empirical Software Engineering, v.10 n.4, p.405-435, October 2005 [doi>10.1007/s10664-005-3861-2]
• 5: M. Haardt and M. Coleman. Ptrace(2). Linux programmer's manual, Section 2, November 1999.
• 6: T. J. LeBlanc , J. M. Mellor-Crummey, Debugging Parallel Programs with Instant Replay, IEEE Transactions on Computers, v.36 n.4, p.471-482, April 1987 [doi>10.1109/TC.1987.1676929]
• 7: Benjamin Livshits , Thomas Zimmermann, DynaMine: finding common error patterns by mining software revision histories, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal
• 8: Nachiappan Nagappan , Thomas Ball , Andreas Zeller, Mining metrics to predict component failures, Proceedings of the 28th international conference on Software engineering, May 20-28, 2006, Shanghai, China [doi>10.1145/1134285.1134349]
• 9: Stephan Neuhaus , Thomas Zimmermann , Christian Holler , Andreas Zeller, Predicting vulnerable software components, Proceedings of the 14th ACM conference on Computer and communications security, October 28-31, 2007, Alexandria, Virginia, USA [doi>10.1145/1315245.1315311]
• 10: R. Sekar, V. N. Venkatakrishnan, S. Basu, S. Bhatkar, and D. C. DuVarney. Model-carrying code: practical approach for safe execution of untrusted applications, October 2003.
• 12: Raphael R. Some , Won S. Kim , Garen Khanoyan , Leslie Callum , Anil Agrawal , John J. Beahan, Software-Implemented Fault Injection Methodology for Design and Validation of System Fault Tolerance, Proceedings of the 2001 International Conference on Dependable Systems and Networks (formerly: FTCS), p.501-506, July 01-04, 2001
• 13: Jaime Spacco , Jaymie Strecker , David Hovemeyer , William Pugh, Software repository mining with Marmoset: an automated programming project snapshot and testing system, ACM SIGSOFT Software Engineering Notes, v.30 n.4, July 2005
• 14: Richard P. Spillane , Charles P. Wright , Gopalan Sivathanu , Erez Zadok, Rapid file system development using ptrace, Proceedings of the 2007 workshop on Experimental computer science, p.22-es, June 13-14, 2007, San Diego, California [doi>10.1145/1281700.1281722]
• 15: John Steven , Pravir Chandra , Bob Fleck , Andy Podgurski, jRapture: A Capture/Replay tool for observation-based testing, Proceedings of the 2000 ACM SIGSOFT international symposium on Software testing and analysis, p.158-167, August 21-24, 2000, Portland, Oregon, United States
• 16: Ahmed E. Hassan , Richard C. Holt , Audris Mockus, MSR 2004: International Workshop on Mining Software Repositories, Proceedings of the 26th International Conference on Software Engineering, p.770-771, May 23-28, 2004
• 17: Annie T. T. Ying , Gail C. Murphy , Raymond Ng , Mark C. Chu-Carroll, Predicting Source Code Changes by Mining Change History, IEEE Transactions on Software Engineering, v.30 n.9, p.574-586, September 2004 [doi>10.1109/TSE.2004.52]
• 37: Jacek Śliwerski , Thomas Zimmermann , Andreas Zeller, When do changes induce fixes?, Proceedings of the 2005 international workshop on Mining software repositories, p.1-5, May 17-17, 2005, St. Louis, Missouri

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Debugging aids

General Terms:
Documentation, Measurement

Keywords:
fault collection