Finding min-repros in database software

ABSTRACT

Testing and debugging database system applications is often challenging and time consuming. A database tester (or DB tester for short) has to detect a problem, determine why it happened, set up an environment to reproduce it, and then create a fix to resolve the problem. In many cases, problems appear in complex scenarios, and thus the setup to reproduce a problem may be large and difficult to understand. This makes the task of finding the root cause of a problem very challenging. As a consequence, a very arduous task for a DB tester is finding a min-repro -- a process of weeding out irrelevant inputs and finding the simplest way to reproduce a problem. Currently, a great deal of searching for minrepros is carried out manually using non-database-specific tools, which is both slow and error-prone. In this paper, we present a system designed to ease and speed up searches for minrepros in database-related products. The min-repro system employs several effective tools, including: the novel simplification transformations, the high-level language for creating custom scripts and automation, the record-and-replay functionality, and the visualization of results and the search space. In addition to the standard application mode, the min-repro system can be interacted with in the game mode. The latter can provide an intrinsically motivating environment for developing successful search strategies, which can be data-mined and generalized into search patterns and used as recommendations for DB testers.

REFERENCES

• 1: Boris Beizer, Software Testing Techniques, John Wiley & Sons, Inc., New York, NY, 1990
• 2: http://www.w3.org/TR/xexpr/.
• 3: T. J. Ostrand , M. J. Balcer, The category-partition method for specifying and generating fuctional tests, Communications of the ACM, v.31 n.6, p.676-686, June 1988 [doi>10.1145/62959.62964]
• 4: Stuart Reid, The Art of Software Testing, Second edition. Glenford J. Myers. Revised and updated by Tom Badgett and Todd M. Thomas, with Corey Sandler. John Wiley and Sons, New Jersey, U.S.A., 2004. ISBN: 0-471-46912-2, pp 234: Book Reviews, Software Testing, Verification & Reliability, v.15 n.2, p.136-137, June 2005 [doi>10.1002/stvr.v15:2]
• 5: Luis von Ahn , Laura Dabbish, Designing games with a purpose, Communications of the ACM, v.51 n.8, August 2008 [doi>10.1145/1378704.1378719]
• 6: Andreas Zeller , Ralf Hildebrandt, Simplifying and Isolating Failure-Inducing Input, IEEE Transactions on Software Engineering, v.28 n.2, p.183-200, February 2002 [doi>10.1109/32.988498]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.7 Database Administration

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.5 Testing and Debugging

General Terms:
Design, Reliability