Contract driven development = test driven development - writing test cases

ABSTRACT

Although unit tests are recognized as an important tool in software development, programmers prefer to write code, rather than unit tests. Despite the emergence of tools like JUnit which automate part of the process, unit testing remains a time-consuming, resource-intensive, and not particularly appealing activity.This paper introduces a new development method, called Contract Driven Development. This development method is based on a novel mechanism that extracts test cases from failure-producing runs that the programmers trigger. It exploits actions that developers perform anyway as part of their normal process of writing code. Thus, it takes the task of writing unit tests off the developers' shoulders, while still taking advantage of their knowledge of the intended semantics and structure of the code. The approach is based on the presence of contracts in code, which act as the oracle of the test cases. The test cases are extracted completely automatically, are run in the background, and can easily be maintained over versions. The tool implementing this methodology is called Cdd and is available both in binary and in source form.

REFERENCES

• 1: Jtest. Parasoft Corporation. http://www.parasoft.com/.
• 2: ARSLAN, V., EUGSTER, P., NIENALTOWSKI, P., AND VAUCOULEUR, S. SCOOP - concurrency made easy. Dependable Systems: Software, Computing, Networks (2006).
• 3: M. Balcer , W. Hasling , T. Ostrand, Automatic generation of test scripts from formal test specifications, Proceedings of the ACM SIGSOFT '89 third symposium on Software testing, analysis, and verification, p.210-218, December 13-15, 1989, Key West, Florida, United States
• 4: Marat Boshernitsan , Roongko Doong , Alberto Savoia, From daikon to agitator: lessons and challenges in building a commercial tool for developer testing, Proceedings of the 2006 international symposium on Software testing and analysis, July 17-20, 2006, Portland, Maine, USA [doi>10.1145/1146238.1146258]
• 5: Chandrasekhar Boyapati , Sarfraz Khurshid , Darko Marinov, Korat: automated testing based on Java predicates, Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis, July 22-24, 2002, Roma, Italy
• 6: Christoph Csallner , Yannis Smaragdakis, DSD-Crasher: a hybrid analysis tool for bug finding, Proceedings of the 2006 international symposium on Software testing and analysis, July 17-20, 2006, Portland, Maine, USA [doi>10.1145/1146238.1146267]
• 7: FELDMAN, Y. A. Extreme design by contract. In Fourth Int'l Conf. Extreme Programming and Agile Processes in Software Engineering (XP 2003) (2003), Springer Verlag, pp. 261--270.
• 8: GARGANTINI, A. Using model checking to generate fault detecting tests. In Tests and Proofs (TAP 2007). Proceedings (2007), Y. Gurevich and B. Meyer, Eds., Lecture Notes in Computer Science, Springer.
• 9: Patrice Godefroid , Nils Klarlund , Koushik Sen, DART: directed automated random testing, Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, June 12-15, 2005, Chicago, IL, USA
• 10: David Hovemeyer , William Pugh, Finding bugs is easy, ACM SIGPLAN Notices, v.39 n.12, December 2004 [doi>10.1145/1052883.1052895]
• 11: Andreas Leitner , Ilinca Ciupa , Bertrand Meyer , Mark Howard, Reconciling Manual and Automated Testing: The AutoTest Experience, Proceedings of the 40th Annual Hawaii International Conference on System Sciences, p.261a, January 03-06, 2007 [doi>10.1109/HICSS.2007.462]
• 12: Darko Marinov , Sarfraz Khurshid, TestEra: A Novel Framework for Automated Testing of Java Programs, Proceedings of the 16th IEEE international conference on Automated software engineering, p.22, November 26-29, 2001
• 13: MEYER, B., CIUPA, I., LEITNER, A., AND LIU, L. L. Automatic testing of object-oriented software. In Proceedings of SOFSEM 2007 (Current Trends in Theory and Practice of Computer Science) (2007), J. van Leeuwen, Ed., Lecture Notes in Computer Science, Springer-Verlag.
• 14: A Jefferson Offutt , Yiwei Xiong , Shaoying Liu, Criteria for Generating Specification-Based Tests, Proceedings of the 5th International Conference on Engineering of Complex Computer Systems, p.119, October 18-22, 1999
• 15: ORIAT, C. Jartege: a tool for random generation of unit tests for Java classes. Tech. Rep. RR-1069-I, Centre National de la Recherche Scientifique, Institut National Polytechnique de Grenoble, Universitáté Joseph Fourier Grenoble I, June 2004.
• 16: Alessandro Orso , Shrinivas Joshi , Martin Burger , Andreas Zeller, Isolating relevant component interactions with JINSI, Proceedings of the 2006 international workshop on Dynamic systems analysis, May 23-23, 2006, Shanghai, China [doi>10.1145/1138912.1138915]
• 17: Alessandro Orso , Bryan Kennedy, Selective capture and replay of program executions, Proceedings of the third international workshop on Dynamic analysis, p.1-7, May 17-17, 2005, St. Louis, Missouri
• 18: T. J. Ostrand , M. J. Balcer, The category-partition method for specifying and generating fuctional tests, Communications of the ACM, v.31 n.6, p.676-686, June 1988 [doi>10.1145/62959.62964]
• 19: OSTROFF, J. S., MAKALSKY, D., AND PAIGE, R. F. Agile specification-driven development. In XP (2004), J. Eckstein and H. Baumeister, Eds., vol. 3092 of Lecture Notes in Computer Science, Springer, pp. 104--112.
• 20: PACHECO, C., AND ERNST, M. D. Eclat: Automatic generation and classification of test inputs. In ECOOP 2005 - Object-Oriented Programming, 19th European Conference (Glasgow, Scotland, July 25-29, 2005).
• 21: David Saff , Michael D. Ernst, An experimental evaluation of continuous testing during development, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 22: Nikolai Tillmann , Wolfram Schulte, Parameterized unit tests with unit meister, ACM SIGSOFT Software Engineering Notes, v.30 n.5, September 2005
• 23: Willem Visser , Corina S. Pǎsǎreanu , Sarfraz Khurshid, Test input generation with java PathFinder, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 24: XIE, T., MARINOV, D., SCHULTE, W., AND NOTKIN, D. Symstra: A framework for generating object-oriented unit tests using symbolic execution. In Proceedings of the 11th International Conference on Tools and Algorithms for the Construction and Analysis of Systems (TACAS 05) (April 2005), pp. 365--381.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging
          Subjects: Testing tools (e.g., data generators, coverage testing)

General Terms:
Verification

Keywords:
unit testing