A critical look at the TAB benchmark for physical design tools

ABSTRACT

There has recently been considerable research on physical design tuning algorithms. At the same time, there is only one published methodology to evaluate the quality of different, competing approaches: the TAB benchmark. In this paper we describe our experiences with TAB. We first report an experimental evaluation of TAB on our latest prototype for physical design tuning. We then identify certain weakness in the benchmark and briefly comment on alternatives to improve its usefulness.

REFERENCES

• 1: TPC Benchmark H. Available at http://www.tpc.org.
• 2: Sanjay Agrawal , Surajit Chaudhuri , Vivek R. Narasayya, Automated Selection of Materialized Views and Indexes in SQL Databases, Proceedings of the 26th International Conference on Very Large Data Bases, p.496-505, September 10-14, 2000
• 3: S. Agrawal et al. Database Tuning Advisor for Microsoft SQL Server 2005. In Proceedings of VLDB, 2004.
• 4: Sanjay Agrawal , Vivek Narasayya , Beverly Yang, Integrating vertical and horizontal partitioning into automated physical database design, Proceedings of the 2004 ACM SIGMOD international conference on Management of data, June 13-18, 2004, Paris, France [doi>10.1145/1007568.1007609]
• 5: Nicolas Bruno , Surajit Chaudhuri, Automatic physical database tuning: a relaxation-based approach, Proceedings of the 2005 ACM SIGMOD international conference on Management of data, June 14-16, 2005, Baltimore, Maryland [doi>10.1145/1066157.1066184]
• 6: Nicolas Bruno , Surajit Chaudhuri, To tune or not to tune?: a lightweight physical design alerter, Proceedings of the 32nd international conference on Very large data bases, September 12-15, 2006, Seoul, Korea
• 7: Surajit Chaudhuri , Mayur Datar , Vivek Narasayya, Index Selection for Databases: A Hardness Study and a Principled Heuristic Solution, IEEE Transactions on Knowledge and Data Engineering, v.16 n.11, p.1313-1323, November 2004 [doi>10.1109/TKDE.2004.75]
• 8: Surajit Chaudhuri , Vivek R. Narasayya, An Efficient Cost-Driven Index Selection Tool for Microsoft SQL Server, Proceedings of the 23rd International Conference on Very Large Data Bases, p.146-155, August 25-29, 1997
• 9: Mariano P. Consens , Denilson Barbosa , Adrian Teisanu , Laurent Mignet, Goals and benchmarks for autonomic configuration recommenders, Proceedings of the 2005 ACM SIGMOD international conference on Management of data, June 14-16, 2005, Baltimore, Maryland [doi>10.1145/1066157.1066185]
• 10: Benoit Dageville , Dinesh Das , Karl Dias , Khaled Yagoub , Mohamed Zait , Mohamed Ziauddin, Automatic SQL tuning in oracle 10g, Proceedings of the Thirtieth international conference on Very large data bases, p.1098-1109, August 31-September 03, 2004, Toronto, Canada
• 11: S. Papadomanolakis and A. Ailamaki. An integer linear programming approach to database design. In Workshop on Self-Managing Database Systems, 2007.
• 12: Alan Skelley, DB2 Advisor: An Optimizer Smart Enough to Recommend its own Indexes, Proceedings of the 16th International Conference on Data Engineering, p.101, February 28-March 03, 2000
• 13: C. Wu et al. The protein information resource: an integrated public resource of functional annotation of proteins. In Nucleic Acids Research, 2002.
• 14: Daniel C. Zilio , Jun Rao , Sam Lightstone , Guy Lohman , Adam Storm , Christian Garcia-Arellano , Scott Fadden, DB2 design advisor: integrated automatic physical database design, Proceedings of the Thirtieth international conference on Very large data bases, p.1087-1097, August 31-September 03, 2004, Toronto, Canada
• 15: Daniel C. Zilio , Calisto Zuzarte , Guy M. Lohman , Hamid Pirahesh , Jarek Gryz , Eric Alton , Dongming Liang , Gary Valentin, Recommending Materialized Views and Indexes with IBM DB2 Design Advisor, Proceedings of the First International Conference on Autonomic Computing, p.180-188, May 17-18, 2004

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.2 Physical Design

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.2 Installation Management
          Subjects: Benchmarks

General Terms:
Design, Management, Performance, Standardization