Efficient management of idleness in storage systems

ABSTRACT

Various activities that intend to enhance performance, reliability, and availability of storage systems are scheduled with low priority and served during idle times. Under such conditions, idleness becomes a valuable “resource” that needs to be efficiently managed. A common approach in system design is to be nonwork conserving by “idle waiting”, that is, delay the scheduling of background jobs to avoid slowing down upcoming foreground tasks.

In this article, we complement “idle waiting” with the “estimation” of background work to be served in every idle interval to effectively manage the trade-off between the performance of foreground and background tasks. As a result, the storage system is better utilized without compromising foreground performance. Our analysis shows that if idle times have low variability, then idle waiting is not necessary. Only if idle times are highly variable does idle waiting become necessary to minimize the impact of background activity on foreground performance. We further show that if there is burstiness in idle intervals, then it is possible to predict accurately the length of incoming idle intervals and use this information to serve more background jobs without affecting foreground performance.

REFERENCES

• 1: Michael Abd-El-Malek , Gregory R. Ganger , Michael K. Reiter , Jay J. Wylie , Garth R. Goodson, Lazy Verification in Fault-Tolerant Distributed Storage Systems, Proceedings of the 24th IEEE Symposium on Reliable Distributed Systems, p.179-190, October 26-28, 2005 [doi>10.1109/RELDIS.2005.20]
• 2: Eitan Bachmat , Jiri Schindler, Analysis of methods for scheduling low priority disk drive tasks, Proceedings of the 2002 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 15-19, 2002, Marina Del Rey, California
• 3: Lakshmi N. Bairavasundaram , Garth R. Goodson , Shankar Pasupathy , Jiri Schindler, An analysis of latent sector errors in disk drives, Proceedings of the 2007 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 12-16, 2007, San Diego, California, USA
• 4: Dennis Colarelli , Dirk Grunwald, Massive arrays of idle disks for storage archives, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-11, November 16, 2002, Baltimore, Maryland
• 5: John R. Douceur , William J. Bolosky, Progress-based regulation of low-importance processes, Proceedings of the seventeenth ACM symposium on Operating systems principles, p.247-260, December 12-15, 1999, Charleston, South Carolina, United States
• 6: Fred Douglis , P. Krishnan , Brian N. Bershad, Adaptive Disk Spin-down Policies for Mobile Computers, Proceedings of the 2nd Symposium on Mobile and Location-Independent Computing, p.121-137, April 10-11, 1995
• 7: Lars Eggert , Joseph D. Touch, Idletime scheduling with preemption intervals, Proceedings of the twentieth ACM symposium on Operating systems principles, October 23-26, 2005, Brighton, United Kingdom
• 8: Richard Golding , Peter Bosch , Carl Staelin , Tim Sullivan , John Wilkes, Idleness is not sloth, Proceedings of the USENIX 1995 Technical Conference Proceedings on USENIX 1995 Technical Conference Proceedings, p.17-17, January 16-20, 1995, New Orleans, Louisiana
• 9: David P. Helmbold , Darrell D. E. Long , Tracey L. Sconyers , Bruce Sherrod, Adaptive disk spin—down for mobile computers, Mobile Networks and Applications, v.5 n.4, p.285-297, Dec. 2000 [doi>10.1023/A:1019129116852]
• 10: Hai Huang , Wanda Hung , Kang G. Shin, FS2: dynamic data replication in free disk space for improving disk performance and energy consumption, Proceedings of the twentieth ACM symposium on Operating systems principles, October 23-26, 2005, Brighton, United Kingdom
• 11: Ilias Iliadis , Robert Haas , Xiao-Yu Hu , Evangelos Eleftheriou, Disk scrubbing versus intra-disk redundancy for high-reliability raid storage systems, Proceedings of the 2008 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 02-06, 2008, Annapolis, MD, USA
• 12: Litzkow, M. J., Livny, M., and Mutka, M. W. 1988. Condor - A hunter of idle workstations. In Proceedings of the IEEE International Conference on Distributed Computing Systems (ICDCS). 104--111.
• 13: Lo, V. M., Zappala, D., Zhou, D., Liu, Y., and Zhao, S. 2004. Cluster computing on the fly: P2P scheduling of idle cycles in the Internet. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS). 227--236.
• 14: Arif Merchant , Philip S. Yu, An analytical model of reconstruction time in mirrored disks, Performance Evaluation, v.20 n.1-3, p.115-129, May 1994
• 15: Mi, N., Riska, A., Smirni, E., and Riedel, E. 2008. Enhancing data availability in disk drives through background activities. In Proceedings of the Symposium on the Dependability of Systems and Networks (DSN). 492--501.
• 16: Richard R. Muntz , John C. S. Lui, Performance Analysis of Disk Arrays under Failure, Proceedings of the 16th International Conference on Very Large Data Bases, p.162-173, August 13-16, 1990
• 17: Zhisheng Niu , Tao Shu , Yoshitaka Takahashi, A vacation queue with setup and close-down times and batch Markovian arrival processes, Performance Evaluation, v.54 n.3, p.225-248, November 2003 [doi>10.1016/S0166-5316(03)00058-0]
• 18: Takayuki Osogami , Mor Harchol-Balter , Alan Scheller-Wolf, Analysis of cycle stealing with switching times and thresholds, Performance Evaluation, v.61 n.4, p.347-369, August 2005 [doi>10.1016/j.peva.2004.09.003]
• 19: Alma Riska , Erik Riedel, Disk drive level workload characterization, Proceedings of the annual conference on USENIX '06 Annual Technical Conference, p.9-9, May 30-June 03, 2006, Boston, MA
• 20: Alma Riska , Erik Riedel, Idle read after write: IRAW, USENIX 2008 Annual Technical Conference on Annual Technical Conference, p.43-56, June 22-27, 2008, Boston, Massachusetts
• 21: Thomas J. E. Schwarz , Qin Xin , Ethan L. Miller , Darrell D. E. Long , Andy Hospodor , Spencer Ng, Disk Scrubbing in Large Archival Storage Systems, Proceedings of the The IEEE Computer Society's 12th Annual International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunications Systems, p.409-418, October 04-08, 2004
• 22: Muthian Sivathanu , Vijayan Prabhakaran , Andrea C. Arpaci-Dusseau , Remzi H. Arpaci-Dusseau, Awarded Best Student Paper! -- Improving Storage System Availability with D-GRAID, Proceedings of the 3rd USENIX Conference on File and Storage Technologies, March 31-31, 2004, San Francisco, CA
• 23: Takagi, H. 1991. Queuing Analysis Volume 1: Vacations and Priority Systems. North-Holland, New York.
• 24: Marvin M. Theimer , Keith A. Lantz , David R. Cheriton, Preemptable remote execution facilities for the V-system, Proceedings of the tenth ACM symposium on Operating systems principles, p.2-12, December 1985, Orcas Island, Washington, United States
• 25: Eno Thereska , Jiri Schindler , John Bucy , Brandon Salmon , Christopher R. Lumb , Gregory R. Ganger, Awarded Best Student Paper! -- A Framework for Building Unobtrusive Disk Maintenance Applications, Proceedings of the 3rd USENIX Conference on File and Storage Technologies, March 31-31, 2004, San Francisco, CA
• 26: A. Thomasian , V. F. Nicola, Performance Evaluation of a Threshold Policy for Scheduling Readers and Writers, IEEE Transactions on Computers, v.42 n.1, p.83-98, January 1993 [doi>10.1109/12.192216]
• 27: Arun Venkataramani , Ravi Kokku , Mike Dahlin, TCP Nice: a mechanism for background transfers, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060320]
• 28: Eric Xu , Attahiru Sule Alfa, A vacation model for the non-saturated readers and writers system with a threshold policy, Performance Evaluation, v.50 n.4, p.233-244, December 2002 [doi>10.1016/S0166-5316(02)00084-6]

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Design studies

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Reliability, availability, and serviceability

General Terms:
Algorithms, Design, Management, Performance, Reliability

Keywords:
Resource management, background jobs, continuous data histogram, foreground jobs, idle periods, idleness, low priority work, performance guarrante, storage systems