Write off-loading: Practical power management for enterprise storage

ABSTRACT

In enterprise data centers power usage is a problem impacting server density and the total cost of ownership. Storage uses a significant fraction of the power budget and there are no widely deployed power-saving solutions for enterprise storage systems. The traditional view is that enterprise workloads make spinning disks down ineffective because idle periods are too short. We analyzed block-level traces from 36 volumes in an enterprise data center for one week and concluded that significant idle periods exist, and that they can be further increased by modifying the read/write patterns using write off-loading. Write off-loading allows write requests on spun-down disks to be temporarily redirected to persistent storage elsewhere in the data center.

The key challenge is doing this transparently and efficiently at the block level, without sacrificing consistency or failure resilience. We describe our write off-loading design and implementation that achieves these goals. We evaluate it by replaying portions of our traces on a rack-based testbed. Results show that just spinning disks down when idle saves 28--36% of energy, and write off-loading further increases the savings to 45--60%.

REFERENCES

• 1: T. E. Anderson , M. D. Dahlin , J. M. Neefe , D. A. Patterson , D. S. Roselli , R. Y. Wang, Serverless network file systems, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.109-126, December 03-06, 1995, Copper Mountain, Colorado, United States
• 2: Akshat Aranya , Charles P. Wright , Erez Zadok, Tracefs: A File System to Trace Them All, Proceedings of the 3rd USENIX Conference on File and Storage Technologies, March 31-31, 2004, San Francisco, CA
• 3: Mary Baker , Satoshi Asami , Etienne Deprit , John Ouseterhout , Margo Seltzer, Non-volatile memory for fast, reliable file systems, Proceedings of the fifth international conference on Architectural support for programming languages and operating systems, p.10-22, October 12-15, 1992, Boston, Massachusetts, United States
• 4: Mary G. Baker , John H. Hartman , Michael D. Kupfer , Ken W. Shirriff , John K. Ousterhout, Measurements of a distributed file system, Proceedings of the thirteenth ACM symposium on Operating systems principles, p.198-212, October 13-16, 1991, Pacific Grove, California, United States
• 5: Enrique V. Carrera , Eduardo Pinheiro , Ricardo Bianchini, Conserving disk energy in network servers, Proceedings of the 17th annual international conference on Supercomputing, June 23-26, 2003, San Francisco, CA, USA [doi>10.1145/782814.782829]
• 6: Jeffrey S. Chase , Darrell C. Anderson , Prachi N. Thakar , Amin M. Vahdat , Ronald P. Doyle, Managing energy and server resources in hosting centers, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 7: Dennis Colarelli , Dirk Grunwald, Massive arrays of idle disks for storage archives, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-11, November 16, 2002, Baltimore, Maryland
• 8: Michael D. Dahlin , Randolph Y. Wang , Thomas E. Anderson , David A. Patterson, Cooperative caching: using remote client memory to improve file system performance, Proceedings of the 1st USENIX conference on Operating Systems Design and Implementation, p.19-es, November 14-17, 1994, Monterey, California
• 9: Fred Douglis , P. Krishnan , Brian Marsh, Thwarting the power-hungry disk, Proceedings of the USENIX Winter 1994 Technical Conference on USENIX Winter 1994 Technical Conference, p.23-23, January 17-21, 1994, San Francisco, California
• 10: Daniel Ellard , Jonathan Ledlie , Pia Malkani , Margo Seltzer, Passive NFS Tracing of Email and Research Workloads, Proceedings of the 2nd USENIX Conference on File and Storage Technologies, March 31-31, 2003, San Francisco, CA
• 11: Lakshmi Ganesh , Hakim Weatherspoon , Mahesh Balakrishnan , Ken Birman, Optimizing power consumption in large scale storage systems, Proceedings of the 11th USENIX workshop on Hot topics in operating systems, p.1-6, May 07-09, 2007, San Diego, CA
• 12: Sudhanva Gurumurthi , Anand Sivasubramaniam , Mahmut Kandemir , Hubertus Franke, DRPM: dynamic speed control for power management in server class disks, Proceedings of the 30th annual international symposium on Computer architecture, June 09-11, 2003, San Diego, California
• 13: S. Gurumurthi , Jianyong Zhang , A. Sivasubramaniam , M. Kandemir , H. Franke , N. Vijaykrishnan , M. J. Irwin, Interplay of energy and performance for disk arrays running transaction processing workloads, Proceedings of the 2003 IEEE International Symposium on Performance Analysis of Systems and Software, p.123-132, March 06-08, 2003
• 14: Intel Corporation. 2006. Dual-Core Intel® Xeon® Processor 5100 Series Datasheet. Intel Corporation, Reference Number: 313355-002.
• 15: Nikolai Joukov , Timothy Wong , Erez Zadok, Accurate and efficient replaying of file system traces, Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies, p.25-25, December 13-16, 2005, San Francisco, CA
• 16: Dong Li , Jun Wang, EERAID: energy efficient redundant and inexpensive disk array, Proceedings of the 11th workshop on ACM SIGOPS European workshop, p.29-es, September 19-22, 2004, Leuven, Belgium [doi>10.1145/1133572.1133577]
• 17: Microsoft. 2002. Event tracing. http://msdn.microsoft.com/library/. Platform SDK: Performance Monitoring, Event Tracing.
• 18: Edmund B. Nightingale , Jason Flinn, Energy-efficiency and storage flexibility in the blue file system, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.25-25, December 06-08, 2004, San Francisco, CA
• 19: Eduardo Pinheiro , Ricardo Bianchini, Energy conservation techniques for disk array-based servers, Proceedings of the 18th annual international conference on Supercomputing, June 26-July 01, 2004, Malo, France [doi>10.1145/1006209.1006220]
• 20: Eduardo Pinheiro , Wolf-Dietrich Weber , Luiz André Barroso, Failure trends in a large disk drive population, Proceedings of the 5th USENIX conference on File and Storage Technologies, p.2-2, February 13-16, 2007, San Jose, CA
• 21: Mendel Rosenblum , John K. Ousterhout, The design and implementation of a log-structured file system, Proceedings of the thirteenth ACM symposium on Operating systems principles, p.1-15, October 13-16, 1991, Pacific Grove, California, United States
• 22: Ruemmler, C. and Wilkes, J. 1993. UNIX disk access patterns. In Proceedings of the USENIX Winter Technical Conference, San Diego, CA.
• 23: Samsung. 2007. NAND flash-based solid state disk product data sheet.
• 24: SanDisk. 2007. SSD UATA 5000 1.8” data sheet. Document No. 80-11-00001.
• 25: Bianca Schroeder , Garth A. Gibson, Disk failures in the real world: what does an MTTF of 1,000,000 hours mean to you?, Proceedings of the 5th USENIX conference on File and Storage Technologies, p.1-es, February 13-16, 2007, San Jose, CA
• 26: Seagate Technology LLC. 2005. Cheetah 15K.4 SCSI Product Manual, Rev. D ed. Seagate Technology LLC, 920 Disc Drive, Scotts Valley, CA. Publication number: 100220456.
• 27: Margo Seltzer , Keith Bostic , Marshall Kirk Mckusick , Carl Staelin, An implementation of a log-structured file system for UNIX, Proceedings of the USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference Proceedings, p.3-3, January 25-29, 1993, San Diego, California
• 28: Charles Weddle , Mathew Oldham , Jin Qian , An-I Andy Wang , Peter Reiher , Geoff Kuenning, PARAID: a gear-shifting power-aware RAID, Proceedings of the 5th USENIX conference on File and Storage Technologies, p.30-30, February 13-16, 2007, San Jose, CA
• 29: Xiaoyu Yao , Jun Wang, RIMAC: a novel redundancy-based hierarchical cache architecture for energy efficient, high performance storage systems, Proceedings of the 1st ACM SIGOPS/EuroSys European Conference on Computer Systems 2006, April 18-21, 2006, Leuven, Belgium
• 30: John Zedlewski , Sumeet Sobti , Nitin Garg , Fengzhou Zheng , Arvind Krishnamurthy , Randolph Wang, Modeling Hard-Disk Power Consumption, Proceedings of the 2nd USENIX Conference on File and Storage Technologies, March 31-31, 2003, San Francisco, CA
• 31: Ningning Zhu , Jiawu Chen , Tzi-Cker Chiueh, TBBT: scalable and accurate trace replay for file server evaluation, Proceedings of the 4th conference on USENIX Conference on File and Storage Technologies, p.24-24, December 13-16, 2005, San Francisco, CA
• 32: Qingbo Zhu , Zhifeng Chen , Lin Tan , Yuanyuan Zhou , Kimberly Keeton , John Wilkes, Hibernator: helping disk arrays sleep through the winter, Proceedings of the twentieth ACM symposium on Operating systems principles, October 23-26, 2005, Brighton, United Kingdom
• 33: Qingbo Zhu , Yuanyuan Zhou, Power-Aware Storage Cache Management, IEEE Transactions on Computers, v.54 n.5, p.587-602, May 2005 [doi>10.1109/TC.2005.82]

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.2 Storage Management

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems

General Terms:
Design, Management, Performance, Reliability

Keywords:
DiskEnergy, disk spin-down, energy, enterprise storage, power, write off-loading