FlashLogging: exploiting flash devices for synchronous logging performance

ABSTRACT

Synchronous transactional logging is the central mechanism for ensuring data persistency and recoverability in database systems. Unfortunately, magnetic disks are ill-suited for the small sequential write pattern of synchronous logging. Alternative solutions (e.g., backup servers or sophisticated battery-backed write caches in high-end disk arrays) are either expensive or complicated.

In this paper, we exploit flash devices for synchronous logging based on the observation that flash devices support small sequential writes well. Comparing a wide variety of flash devices, we find that USB flash drives are a good match for this task because of its unique characteristics: widely available USB ports, hot-plug capability useful for coping with flash wear, and low price so that multiple drives are affordable. We propose FlashLogging, a logging solution that exploits multiple (USB) flash drives for synchronous logging. We identify and address four challenges: (i) efficiently exploiting multiple flash drives for logging; (ii) coping with the large variance of write latencies because of device erasure operations; (iii) efficient recovery processing; and (iv) combining flash drives and disks for better logging and recovery performance. We implemented our solution within MySQL-InnoDB. Our real machine experiments running online transaction processing workloads (TPCC) show that FlashLogging achieves up to 5.7X improvements over magnetic-disk-based logging, and obtains up to 98.6% of the ideal performance. We further compare our design with one that uses Solid-State Drives (SSDs), and find that although SSDs improve logging performance, multiple USB flash drives can achieve comparable or better performance with much lower price.

REFERENCES

• 1: Todd Anderson , Yuri Breitbart , Henry F. Korth , Avishai Wool, Replication, consistency, and practicality: are these mutually exclusive?, Proceedings of the 1998 ACM SIGMOD international conference on Management of data, p.484-495, June 01-04, 1998, Seattle, Washington, United States
• 2: Roger Barga , Shimin Chen , David Lomet, Improving Logging and Recovery Performance in Phoenix/App, Proceedings of the 20th International Conference on Data Engineering, p.486, March 30-April 02, 2004
• 3: L. Bouganim, B. T. Jónsson, and P. Bonnet. uFLIP: Understanding ash io patterns. In CIDR, 2009.
• 4: Yuri Breitbart , Raghavan Komondoor , Rajeev Rastogi , S. Seshadri , Avi Silberschatz, Update propagation protocols for replicated databates, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.97-108, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 5: Adrian M. Caulfield , Laura M. Grupp , Steven Swanson, Gordon: using flash memory to build fast, power-efficient clusters for data-intensive applications, Proceeding of the 14th international conference on Architectural support for programming languages and operating systems, March 07-11, 2009, Washington, DC, USA
• 6: Chris Mellor. Faster usb 3.0 is coming. http://www.pcworld.com/article/137551/.
• 7: Khuzaima Daudjee , Kenneth Salem, Lazy database replication with snapshot isolation, Proceedings of the 32nd international conference on Very large data bases, September 12-15, 2006, Seoul, Korea
• 8: E. N. (Mootaz) Elnozahy , Lorenzo Alvisi , Yi-Min Wang , David B. Johnson, A survey of rollback-recovery protocols in message-passing systems, ACM Computing Surveys (CSUR), v.34 n.3, p.375-408, September 2002 [doi>10.1145/568522.568525]
• 9: Eran Gal , Sivan Toledo, Algorithms and data structures for flash memories, ACM Computing Surveys (CSUR), v.37 n.2, p.138-163, June 2005 [doi>10.1145/1089733.1089735]
• 10: Goetz Graefe, The five-minute rule twenty years later, and how flash memory changes the rules, Proceedings of the 3rd international workshop on Data management on new hardware, June 15-15, 2007, Beijing, China [doi>10.1145/1363189.1363198]
• 11: Jim Gray , Pat Helland , Patrick O'Neil , Dennis Shasha, The dangers of replication and a solution, Proceedings of the 1996 ACM SIGMOD international conference on Management of data, p.173-182, June 04-06, 1996, Montreal, Quebec, Canada
• 12: Jim Gray , Andreas Reuter, Transaction Processing: Concepts and Techniques, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1992
• 13: Stavros Harizopoulos , Daniel J. Abadi , Samuel Madden , Michael Stonebraker, OLTP through the looking glass, and what we found there, Proceedings of the 2008 ACM SIGMOD international conference on Management of data, June 09-12, 2008, Vancouver, Canada [doi>10.1145/1376616.1376713]
• 14: John L. Hennessy , David A. Patterson, Computer Architecture, Fourth Edition: A Quantitative Approach, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2006
• 15: Intel X25-M 80GB SATA Solid State Drive, Intel Ups The Ante. http://hothardware.com/Articles/Intel-X25M-80GB-SATA-Solid-State-Drive-Intel-Ups-The-Ante/?page=2.
• 16: Intel X25-M and X18-M Mainsteam SATA Solid State Drives. http://download.intel.com/design/ ash/nand/mainstream/mainstream-sata-ssd-product-brief.pdf.
• 17: D. Kline and N. Pilaka. An overview of transactional logging in db2 universal database. http://www.ibm.com/developerworks/db2/library/techarticle/0301kline/0301kline.html.
• 18: Ioannis Koltsidas , Stratis D. Viglas, Flashing up the storage layer, Proceedings of the VLDB Endowment, v.1 n.1, August 2008 [doi>10.1145/1453856.1453913]
• 19: Sang-Won Lee , Bongki Moon, Design of flash-based DBMS: an in-page logging approach, Proceedings of the 2007 ACM SIGMOD international conference on Management of data, June 11-14, 2007, Beijing, China [doi>10.1145/1247480.1247488]
• 20: Sang-Won Lee , Bongki Moon , Chanik Park , Jae-Myung Kim , Sang-Woo Kim, A case for flash memory ssd in enterprise database applications, Proceedings of the 2008 ACM SIGMOD international conference on Management of data, June 09-12, 2008, Vancouver, Canada [doi>10.1145/1376616.1376723]
• 21: Yi Lin , Bettina Kemme , Marta Patiño-Martínez , Ricardo Jiménez-Peris, Middleware based data replication providing snapshot isolation, Proceedings of the 2005 ACM SIGMOD international conference on Management of data, June 14-16, 2005, Baltimore, Maryland [doi>10.1145/1066157.1066205]
• 22: Microsoft Corp. Sql server 7.0, sql server 2000, and sql server 2005 logging and data storage algorithms. http://support.microsoft.com/kb/230785.
• 23: Sang Lyul Min , Eyee Hyun Nam, Current trends in flash memory technology: invited paper, Proceedings of the 2006 Asia and South Pacific Design Automation Conference, January 24-27, 2006, Yokohama, Japan [doi>10.1145/1118299.1118382]
• 24: Suman Nath , Phillip B. Gibbons, Online maintenance of very large random samples on flash storage, Proceedings of the VLDB Endowment, v.1 n.1, August 2008 [doi>10.1145/1453856.1453961]
• 25: Suman Nath , Aman Kansal, FlashDB: dynamic self-tuning database for NAND flash, Proceedings of the 6th international conference on Information processing in sensor networks, April 25-27, 2007, Cambridge, Massachusetts, USA [doi>10.1145/1236360.1236412]
• 26: Edmund B. Nightingale , Kaushik Veeraraghavan , Peter M. Chen , Jason Flinn, Rethink the sync, Proceedings of the 7th USENIX Symposium on Operating Systems Design and Implementation, p.1-1, November 06-08, 2006, Seattle, WA
• 27: Oracle Corp. Oracle database administrator's guide 10g release 1 (chapter 6. managing the redo log). Part Number B10739-01.
• 28: OSDL DBT2. http://sourceforge.net/projects/osdldbt.
• 29: V. Prabhakaran, T. L. Rodeheffer, and L. Zhou. Transactional ash. In OSDI, 2008.
• 30: Kenneth A. Ross, Modeling the performance of algorithms on flash memory devices, Proceedings of the 4th international workshop on Data management on new hardware, June 13-13, 2008, Vancouver, Canada [doi>10.1145/1457150.1457153]
• 31: Samsung Electronics. Onenand specification. http://www.samsung.com.
• 32: Mehul A. Shah , Stavros Harizopoulos , Janet L. Wiener , Goetz Graefe, Fast scans and joins using flash drives, Proceedings of the 4th international workshop on Data management on new hardware, June 13-13, 2008, Vancouver, Canada [doi>10.1145/1457150.1457154]
• 33: Michael Stonebraker , Samuel Madden , Daniel J. Abadi , Stavros Harizopoulos , Nabil Hachem , Pat Helland, The end of an architectural era: (it's time for a complete rewrite), Proceedings of the 33rd international conference on Very large data bases, September 23-27, 2007, Vienna, Austria
• 34: Transaction Processing Performance Council. TPC-C benchmark. http://www.tpc.org/tpcc/.
• 35: V. Vasudevan, J. Franklin, D. Andersen, A. Phanishayee, L. Tan, M. Kaminsky, and I. Moraru. Fawndamentally power-efficient clusters. In HotOS, 2009.
• 36: Rui Wang , Betty Salzberg , David Lomet, Log-based recovery for middleware servers, Proceedings of the 2007 ACM SIGMOD international conference on Management of data, June 11-14, 2007, Beijing, China [doi>10.1145/1247480.1247528]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.2 Physical Design

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Transaction processing
      H.2.7 Database Administration
          Subjects: Logging and recovery

General Terms:
Algorithms, Design, Performance, Reliability

Keywords:
flash devices, flashlogging, near-zero-delay archival disk, online transaction processing, outlier detection and hiding, recovery processing, synchronous logging, unconventional array organization