Database-aware test coverage monitoring

ABSTRACT

Unlike traditional programs, a database-centric application interacts with a database that has a complex state and structure. Even though the database is an important component of modern software, there are few tools to support the testing of database-centric applications. This paper presents a test coverage monitoring technique that tracks a program's definition and use of database entities during test suite execution. The paper also describes instrumentation probes that construct a coverage tree that records how the program and the tests cover the database. We conducted experiments to measure the costs that are associated with (i) instrumenting the program and the tests and (ii) monitoring coverage. For all of the applications, the experiments demonstrate that the instrumentation mechanism incurs an acceptable time overhead. While the use of statically inserted probes may increase the size of an application, this approach enables database-aware coverage monitoring that increases testing time from 13% to no more than 54%

REFERENCES

• 1: B. Alpern , S. Augart , S. M. Blackburn , M. Butrico , A. Cocchi , P. Cheng , J. Dolby , S. Fink , D. Grove , M. Hind , K. S. McKinley , M. Mergen , J. E. B. Moss , T. Ngo , V. Sarkar, The Jikes research virtual machine project: building an open-source research community, IBM Systems Journal, v.44 n.2, p.399-417, January 2005
• 2: Glenn Ammons , Thomas Ball , James R. Larus, Exploiting hardware performance counters with flow and context sensitive profiling, Proceedings of the ACM SIGPLAN 1997 conference on Programming language design and implementation, p.85-96, June 16-18, 1997, Las Vegas, Nevada, United States
• 3: R. Bloom. Debugging JDBC with a logging driver. Java Developer's Journal, 2006.
• 4: Sudarshan S. Chawathe , Hector Garcia-Molina, Meaningful change detection in structured data, Proceedings of the 1997 ACM SIGMOD international conference on Management of data, p.26-37, May 11-15, 1997, Tucson, Arizona, United States
• 5: Sudarshan S. Chawathe , Anand Rajaraman , Hector Garcia-Molina , Jennifer Widom, Change detection in hierarchically structured information, Proceedings of the 1996 ACM SIGMOD international conference on Management of data, p.493-504, June 04-06, 1996, Montreal, Quebec, Canada
• 6: David Chays , Yuetang Deng , Phyllis G. Frankl , Saikat Dan , Filippos I. Vokolos , Elaine J. Weyuker, An AGENDA for testing relational database applications: Research Articles, Software Testing, Verification & Reliability, v.14 n.1, p.17-44, March 2004 [doi>10.1002/stvr.v14:1]
• 7: A.S. Christensen, A. Møller, and M.I. Schwartzbach. Precise analysis of string expressions. In Proc of 10th SAS, volume 2694 of LNCS, pages 1--18, June 2003.
• 8: Hyunsook Do , Sebastian Elbaum , Gregg Rothermel, Supporting Controlled Experimentation with Testing Techniques: An Infrastructure and its Potential Impact, Empirical Software Engineering, v.10 n.4, p.405-435, October 2005 [doi>10.1007/s10664-005-3861-2]
• 9: Michael Engel , Bernd Freisleben, Supporting autonomic computing functionality via dynamic operating system kernel aspects, Proceedings of the 4th international conference on Aspect-oriented software development, p.51-62, March 14-18, 2005, Chicago, Illinois [doi>10.1145/1052898.1052903]
• 10: William G. J. Halfond , Alessandro Orso, Command-Form Coverage for Testing Database Applications, Proceedings of the 21st IEEE/ACM International Conference on Automated Software Engineering, p.69-80, September 18-22, 2006 [doi>10.1109/ASE.2006.27]
• 11: Erik Hilsdale , Jim Hugunin, Advice weaving in AspectJ, Proceedings of the 3rd international conference on Aspect-oriented software development, p.26-35, March 22-24, 2004, Lancaster, UK [doi>10.1145/976270.976276]
• 12: G.M. Kapfhammer. A Comprehensive Framework for Testing Database-Centric Applications. PhD thesis, University of Pittsburgh, Pittsburgh, Pennsylvania, 2007.
• 13: Gregory M. Kapfhammer , Mary Lou Soffa, A family of test adequacy criteria for database-driven applications, Proceedings of the 9th European software engineering conference held jointly with 11th ACM SIGSOFT international symposium on Foundations of software engineering, September 01-05, 2003, Helsinki, Finland
• 14: Mehdi Kessis , Yves Ledru , Gérard Vandome, Experiences in coverage testing of a Java middleware, Proceedings of the 5th international workshop on Software engineering and middleware, September 05-06, 2005, Lisbon, Portugal [doi>10.1145/1108473.1108483]
• 15: Gregor Kiczales , Erik Hilsdale , Jim Hugunin , Mik Kersten , Jeffrey Palm , William Griswold, Getting started with ASPECTJ, Communications of the ACM, v.44 n.10, p.59-65, Oct. 2001 [doi>10.1145/383845.383858]
• 16: Wilburt Labio , Hector Garcia-Molina, Efficient Snapshot Differential Algorithms for Data Warehousing, Proceedings of the 22th International Conference on Very Large Data Bases, p.63-74, September 03-06, 1996
• 17: Scott McMaster , Atif M. Memon, Call Stack Coverage for Test Suite Reduction, Proceedings of the 21st IEEE International Conference on Software Maintenance, p.539-548, September 25-30, 2005 [doi>10.1109/ICSM.2005.29]
• 18: S. Microsystems. JDBC data access API. 2007. http://developers.sun.com/product/jdbc/drivers/.
• 19: Jonathan Misurda , James A. Clause , Juliya L. Reed , Bruce R. Childers , Mary Lou Soffa, Demand-driven structural testing with dynamic instrumentation, Proceedings of the 27th international conference on Software engineering, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062496]
• 20: B. Monjian. PostgreSQL. Addison-Wesley, 2000.
• 21: E.W. Myers. An O(ND) difference algorithm and its variations. Algorithmica, 1(2):251--266, 1986.
• 22: Christina Pavlopoulou , Michal Young, Residual test coverage monitoring, Proceedings of the 21st international conference on Software engineering, p.277-284, May 16-22, 1999, Los Angeles, California, United States [doi>10.1145/302405.302637]
• 23: David J. Pearce , Matthew Webster , Robert Berry , Paul H. J. Kelly, Profiling with AspectJ, Software—Practice & Experience, v.37 n.7, p.747-777, June 2007 [doi>10.1002/spe.v37:7]
• 24: Andrei Popovici , Thomas Gross , Gustavo Alonso, Dynamic weaving for aspect-oriented programming, Proceedings of the 1st international conference on Aspect-oriented software development, April 22-26, 2002, Enschede, The Netherlands [doi>10.1145/508386.508404]
• 25: William Pugh, Compressing Java class files, Proceedings of the ACM SIGPLAN 1999 conference on Programming language design and implementation, p.247-258, May 01-04, 1999, Atlanta, Georgia, United States
• 26: Gregg Rothermel , Roland J. Untch , Chengyun Chu, Prioritizing Test Cases For Regression Testing, IEEE Transactions on Software Engineering, v.27 n.10, p.929-948, October 2001 [doi>10.1109/32.962562]
• 27: V. Roubtsov. Emma: a free {Java code coverage tool. http://emma.sourceforge.net/index.html, 2005.
• 28: Abraham Silberschatz , Henry F. Korth , S. Sudershan, Database System Concepts, McGraw-Hill, Inc., New York, NY, 1998
• 29: María José Suárez-Cabal , Javier Tuya, Using an SQL coverage measurement for testing database applications, Proceedings of the 12th ACM SIGSOFT twelfth international symposium on Foundations of software engineering, October 31-November 06, 2004, Newport Beach, CA, USA
• 30: Mustafa M. Tikir , Jeffrey K. Hollingsworth, Efficient instrumentation for code coverage testing, Proceedings of the 2002 ACM SIGSOFT international symposium on Software testing and analysis, July 22-24, 2002, Roma, Italy
• 31: Paolo Tonella, Evolutionary testing of classes, Proceedings of the 2004 ACM SIGSOFT international symposium on Software testing and analysis, July 11-14, 2004, Boston, Massachusetts, USA
• 32: Kristen R. Walcott , Mary Lou Soffa , Gregory M. Kapfhammer , Robert S. Roos, TimeAware test suite prioritization, Proceedings of the 2006 international symposium on Software testing and analysis, July 17-20, 2006, Portland, Maine, USA [doi>10.1145/1146238.1146240]
• 33: Randy Jay Yarger , George Reese , Tim King , Andy Oram, MySQL and mSQL, O'Reilly & Associates, Inc., Sebastopol, CA, 1999

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

General Terms:
Experimentation, Verification

Keywords:
database application, test coverage monitoring