DBmbench: fast and accurate database workload representation on modern microarchitecture

ABSTRACT

With the proliferation of database workloads on servers, much recent research on server architecture has focused on database system benchmarks. The TPC benchmarks for the two most common server workloads, OLTP and DSS, have been used extensively in the database community to evaluate the database system functionality and performance. Unfortunately, these benchmarks fall short of being effective in microarchitecture and memory system research due to several key shortcomings. First, setting up the experimental environment and tuning these benchmarks to match the workload behavior of interest involves extremely complex procedures. Second, the benchmarks themselves are complex and preclude accurate correlation of microarchitecture-and memory-level bottlenecks to dominant workload characteristics. Finally, industrial-grade configurations of such benchmarks are too large and preclude their use in detailed but slow microarchitectural simulation studies of future servers. In this paper, we first present an analysis of the dominant behavior in DSS and OLTP workloads, and highlight their key processor and memory performance characteristics. We then introduce a systematic scaling framework to scale down the TPC benchmarks. Finally, we propose the DBmbench, consisting of two substantially scaled-down benchmarks: µTPC-H and µTPC-C that accurately (> 95%) capture the processor and memory performance behavior of DSS and OLTP workloads.

REFERENCES

• 1: Anastassia Ailamaki , David J. DeWitt , Mark D. Hill , David A. Wood, DBMSs on a Modern Processor: Where Does Time Go?, Proceedings of the 25th International Conference on Very Large Data Bases, p.266-277, September 07-10, 1999
• 2: Branch Behavior of a Commercial OLTP Workload on Intel IA32 Processors, Proceedings of the 2002 IEEE International Conference on Computer Design: VLSI in Computers and Processors (ICCD'02), p.242, September 16-18, 2002
• 3: Luiz André Barroso , Kourosh Gharachorloo , Edouard Bugnion, Memory system characterization of commercial workloads, Proceedings of the 25th annual international symposium on Computer architecture, p.3-14, June 27-July 02, 1998, Barcelona, Spain
• 4: {4} L. A. Barroso, K. Gharachorloo, A. Nowatzyk, and B. Verghese. Impact of chip-level integration on performance of OLTP workloads. In Proceedings of International Symposium on High-Performance Computer Architecture, January 2000.
• 5: Bob Knighten, Detailed Characterization of a Quad Pentium Pro Server Running TPC-D, Proceedings of the 1999 IEEE International Conference on Computer Design, p.108, October 10-13, 1999
• 6: E. F. Codd, A relational model of data for large shared data banks, Communications of the ACM, v.13 n.6, p.377-387, June 1970 [doi>10.1145/362384.362685]
• 7: {7} T. Diep, M. Annavaram, B. Hirano, and J. P. Shen. Analyzing performance characteristics of OLTP cached workloads by linear interpolation. In Workshop on Computer Architecture Evaluation using Commercial Workloads , September 2002.
• 8: Jim Gray, Benchmark Handbook: For Database and Transaction Processing Systems, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1992
• 9: Richard Hankins , Trung Diep , Murali Annavaram , Brian Hirano , Harald Eri , Hubert Nueckel , John P. Shen, Scaling and Charact rizing Database Workloads: Bridging the Gap between Research and Practice, Proceedings of the 36th annual IEEE/ACM International Symposium on Microarchitecture, p.151, December 03-05, 2003
• 10: {10} IBM Corporation. IBM DB2 Universal Database Administration Guide: Implementation , 2000.
• 11: {11} K. Keeton and D. A. Patterson. Towards a simplified database workload for computer architecture evaluations, chapter 4. Kluwer Academic Publishers, 2000.
• 12: Kimberly Keeton , David A. Patterson , Yong Qiang He , Roger C. Raphael , Walter E. Baker, Performance characterization of a Quad Pentium Pro SMP using OLTP workloads, Proceedings of the 25th annual international symposium on Computer architecture, p.15-26, June 27-July 02, 1998, Barcelona, Spain
• 13: Jack L. Lo , Luiz André Barroso , Susan J. Eggers , Kourosh Gharachorloo , Henry M. Levy , Sujay S. Parekh, An analysis of database workload performance on simultaneous multithreaded processors, Proceedings of the 25th annual international symposium on Computer architecture, p.39-50, June 27-July 02, 1998, Barcelona, Spain
• 14: Raghu Ramakrishnan , Johannes Gehrke, Database Management Systems, McGraw-Hill Science/Engineering/Math, 2002
• 15: Parthasarathy Ranganathan , Kourosh Gharachorloo , Sarita V. Adve , Luiz André Barroso, Performance of database workloads on shared-memory systems with out-of-order processors, Proceedings of the eighth international conference on Architectural support for programming languages and operating systems, p.307-318, October 02-07, 1998, San Jose, California, United States
• 16: Timothy Sherwood , Erez Perelman , Greg Hamerly , Brad Calder, Automatically characterizing large scale program behavior, Proceedings of the 10th international conference on Architectural support for programming languages and operating systems, October 05-09, 2002, San Jose, California
• 17: {17} SimpleScalar tool set. SimpleScalar LLC. http://www.simplescalar.com.
• 18: {18} The Standard Performance Evaluation Corporation. SPEC CPU Benchmark. http://- www.specbench.org.
• 19: Shreekant S. Thakkar , Mark Sweiger, Performance of an OLTP application on symmetry multiprocessor system, Proceedings of the 17th annual international symposium on Computer Architecture, p.228-238, May 28-31, 1990, Seattle, Washington, United States
• 20: Pedro Trancoso , Josep-L. Larriba-Pey , Zheng Zhang , Josep Torrellas, The Memory Performance of DSS Commercial Workloads in Shared-Memory Multiprocessors, Proceedings of the 3rd IEEE Symposium on High-Performance Computer Architecture, p.250, February 01-05, 1997
• 21: {21} Transaction Processing Performance Countil. TPC benchmarks. http://www.tpc.org.
• 22: Roland E. Wunderlich , Thomas F. Wenisch , Babak Falsafi , James C. Hoe, SMARTS: accelerating microarchitecture simulation via rigorous statistical sampling, Proceedings of the 30th annual international symposium on Computer architecture, June 09-11, 2003, San Diego, California

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications

General Terms:
Design, Experimentation, Measurement, Performance