Evaluating advantages of test driven development: a controlled experiment with professionals

ABSTRACT

Test driven development (TDD) is gaining interest among practitioners and researchers: it promises to increase the quality of the code. Even if TDD is considered a development practice, it relies on the use of unit testing. For this reason, it could be an alternative to the testing after coding (TAC), which is the usual approach to run and execute unit tests after having written the code. We wondered which are the differences between the two practices, from the standpoint of quality and productivity. In order to answer our research question, we carried out an experiment in a Spanish Software House. The results suggest that TDD improves the unit testing but slows down the overall process.

REFERENCES

• 1: Kent Beck, Extreme programming explained: embrace change, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 2: David P. Darcy , Chris F. Kemerer, OO Metrics in Practice, IEEE Software, v.22 n.6, p.17-19, November 2005 [doi>10.1109/MS.2005.160]
• 3: Edwards, S. Using test-driven development in the classroom: Providing students with automatic, concrete feedback on performance. In Proc. of the Int'l Conference on Education and Information Systems: Technologies and Applications (EISTA'03), (Orlando, Florida, USA, 2003).
• 4: Hakan Erdogmus , Maurizio Morisio , Marco Torchiano, On the Effectiveness of the Test-First Approach to Programming, IEEE Transactions on Software Engineering, v.31 n.3, p.226-237, March 2005 [doi>10.1109/TSE.2005.37]
• 5: George, B. and Williams, L. A structured experiment of testdriven development. Information and Software Technology 46 (May 2004), pp.337--342.
• 6: A. Geras , M. Smith , J. Miller, A Prototype Empirical Evaluation of Test Driven Development, Proceedings of the Software Metrics, 10th International Symposium on (METRICS'04), p.405-416, September 11-17, 2004 [doi>10.1109/METRICS.2004.2]
• 7: Ross Grable , Jacquelyn Jernigan , Casey Pogue , Dale Divis, Metrics for Small Projects: Experiences at the SED, IEEE Software, v.16 n.2, p.21-29, March 1999 [doi>10.1109/52.754048]
• 8: Barbara Kitchenham , Emilia Mendes, Software Productivity Measurement Using Multiple Size Measures, IEEE Transactions on Software Engineering, v.30 n.12, p.1023-1035, December 2004 [doi>10.1109/TSE.2004.104]
• 9: Muller, M., and Hagner, O. Experiment about Test-first programming. In Proc. of Empirical Assessment in Software Engineering (EASE'02), (Keele, UK, 2002).
• 10: Pankur M., Ciglaric M., Trampus M. and Vidmar T. Towards empirical evaluation of test-driven development in a university environment. In EUROCON 2003. Computer as a Tool. The IEEE Region 8 , Volume: 2, (Ljublijana, Slovenia, 2003), IEEE CS Press, pp.83, 86.
• 11: Rahul Premraj , Martin Shepperd , Barbara Kitchenham , Pekka Forselius, An Empirical Analysis of Software Productivity over Time, Proceedings of the 11th IEEE International Software Metrics Symposium (METRICS'05), p.37, September 19-22, 2005 [doi>10.1109/METRICS.2005.8]
• 12: Dag I. K. Sjberg , Bente Anda , Erik Arisholm , Tore Dybå , Magne Jrgensen , Amela Karahasanovic , Espen F. Koren , Marek Vokác, Conducting Realistic Experiments in Software Engineering, Proceedings of the 2002 International Symposium on Empirical Software Engineering, p.17, October 03-04, 2002
• 13: Laurie Williams , E. Michael Maximilien , Mladen Vouk, Test-Driven Development as a Defect-Reduction Practice, Proceedings of the 14th International Symposium on Software Reliability Engineering, p.34, November 17-21, 2003
• 14: The ECLIPSE IDE. Available in http://www.eclipse.org/
• 15: The JUnit Testing Framework. Available in http://www.junit.org.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Performance measures

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
      D.2.8 Metrics
          Subjects: Process metrics; Product metrics

Keywords:
empirical software engineering, process quality, test driven development