Deconstructing storage arrays

ABSTRACT

We introduce Shear, a user-level software tool that characterizes RAID storage arrays. Shear employs a set of controlled algorithms combined with statistical techniques to automatically determine the important properties of a RAID system, including the number of disks, chunk size, level of redundancy, and layout scheme. We illustrate the correctness of Shear by running it upon numerous simulated configurations, and then verify its real-world applicability by running Shear on both software-based and hardware-based RAID systems. Finally, we demonstrate the utility of Shear through three case studies. First, we show how Shear can be used in a storage management environment to verify RAID construction and detect failures. Second, we demonstrate how Shear can be used to extract detailed characteristics about the individual disks within an array. Third, we show how an operating system can use Shear to automatically tune its storage subsystems to specific RAID configurations.

REFERENCES

• 1: T. Bray. The Bonnie File System Benchmark. http://www.textuality.com/bonnie/.
• 2: Nathan C. Burnett , John Bent , Andrea C. Arpaci-Dusseau , Remzi H. Arpaci-Dusseau, Exploiting Gray-Box Knowledge of Buffer-Cache Management, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.29-44, June 10-15, 2002
• 3: Peter M. Chen , Edward K. Lee, Striping in a RAID level 5 disk array, Proceedings of the 1995 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems, p.136-145, May 15-19, 1995, Ottawa, Ontario, Canada
• 4: Peter M. Chen , Edward K. Lee , Garth A. Gibson , Randy H. Katz , David A. Patterson, RAID: high-performance, reliable secondary storage, ACM Computing Surveys (CSUR), v.26 n.2, p.145-185, June 1994 [doi>10.1145/176979.176981]
• 5: Peter M. Chen , David A. Patterson, Maximizing performance in a striped disk array, Proceedings of the 17th annual international symposium on Computer Architecture, p.322-331, May 28-31, 1990, Seattle, Washington, United States
• 6: Peter M. Chen , David A. Patterson, A new approach to I/O performance evaluation: self-scaling I/O benchmarks, predicted I/O performance, Proceedings of the 1993 ACM SIGMETRICS conference on Measurement and modeling of computer systems, p.1-12, May 10-14, 1993, Santa Clara, California, United States
• 7: Timothy E. Denehy , Andrea C. Arpaci-Dusseau , Remzi H. Arpaci-Dusseau, Bridging the Information Gap in Storage Protocol Stacks, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.177-190, June 10-15, 2002
• 8: EMC Corporation. Symmetrix Enterprise Information Storage Systems. http://www.emc.com, 2002.
• 9: T. Glaser. TCP/IP Stack Fingerprinting Principles. http://www.sans.org/newlook/resources/IDFAQ/TCP_fingerprinting.htm, October 2000.
• 10: E. Grochowski. Emerging Trends in Data Storage on Magnetic Hard Disk Drives. Datatech, September 1999.
• 11: Hui-I Hsiao , David J. DeWitt, Chained Declustering: A New Availability Strategy for Multiprocessor Database Machines, Proceedings of the Sixth International Conference on Data Engineering, p.456-465, February 05-09, 1990
• 12: J. Katcher. PostMark: A New File System Benchmark. Technical Report TR-3022, Network Appliance Inc., Oct 1997.
• 13: Edward K. Lee , Randy H. Katz, Performance consequences of parity placement in disk arrays, Proceedings of the fourth international conference on Architectural support for programming languages and operating systems, p.190-199, April 08-11, 1991, Santa Clara, California, United States
• 14: C. Lumb, J. Schindler, G. Ganger, D. Nagle, and E. Riedel. Towards Higher Disk Head Utilization: Extracting "Free" Bandwidth From Busy Disk Drives. In OSDI '00, pages 87--102, San Diego, CA, October 2000.
• 15: W. Norcutt. The IOzone Filesystem Benchmark. http://www.iozone.org/.
• 16: J. Padhye and S. Floyd. Identifying the TCP Behavior of Web Servers. In SIGCOMM '01, San Diego, California, August 2001.
• 17: David A. Patterson , Garth Gibson , Randy H. Katz, A case for redundant arrays of inexpensive disks (RAID), Proceedings of the 1988 ACM SIGMOD international conference on Management of data, p.109-116, June 01-03, 1988, Chicago, Illinois, United States
• 18: Dan Pelleg , Andrew W. Moore, X-means: Extending K-means with Efficient Estimation of the Number of Clusters, Proceedings of the Seventeenth International Conference on Machine Learning, p.727-734, June 29-July 02, 2000
• 19: John Regehr, Inferring Scheduling Behavior with Hourglass, Proceedings of the FREENIX Track: 2002 USENIX Annual Technical Conference, p.143-156, June 10-15, 2002
• 20: Chris Ruemmler , John Wilkes, An introduction to disk drive modeling, Computer, v.27 n.3, p.17-28, March 1994 [doi>10.1109/2.268881]
• 21: Alan Jay Smith , Rafael H. Saavedra, Measuring Cache and TLB Performance and Their Effect on Benchmark Runtimes, IEEE Transactions on Computers, v.44 n.10, p.1223-1235, October 1995 [doi>10.1109/12.467697]
• 22: S. Savage and J. Wilkes. AFRAID --- A Frequently Redundant Array of Independent Disks. In USENIX 1996, pages 27--39, San Diego, CA, January 1996.
• 23: J. Schindler and G. Ganger. Automated Disk Drive Characterization. Technical Report CMU-CS-99-176, Carnegie Mellon University, November 1999.
• 24: Jiri Schindler , John Linwood Griffin , Christopher R. Lumb , Gregory R. Ganger, Track-Aligned Extents: Matching Access Patterns to Disk Drive Characteristics, Proceedings of the Conference on File and Storage Technologies, p.259-274, January 28-30, 2002
• 25: Jiri Schindler , Steven W. Schlosser , Minglong Shao , Anastassia Ailamaki , Gregory R. Ganger, Atropos: A Disk Array Volume Manager for Orchestrated Use of Disks, Proceedings of the 3rd USENIX Conference on File and Storage Technologies, March 31-31, 2004, San Francisco, CA
• 26: C. Staelin and L. McVoy. mhz: Anatomy of a micro-benchmark. In USENIX '98, pages 155--166, New Orleans, LA, June 1998.
• 27: N. Talagala , R. Arpaci-Dusseau , D. Patterson, Micro-Benchmark Based Extraction of Local and Global Disk, University of California at Berkeley, Berkeley, CA, 2000
• 28: Elizabeth Varki , Arif Merchant , Jianzhang Xu , Xiaozhou Qiu, Issues and Challenges in the Performance Analysis of Real Disk Arrays, IEEE Transactions on Parallel and Distributed Systems, v.15 n.6, p.559-574, June 2004 [doi>10.1109/TPDS.2004.9]
• 29: John Wilkes , Richard Golding , Carl Staelin , Tim Sullivan, The HP AutoRAID hierarchical storage system, ACM Transactions on Computer Systems (TOCS), v.14 n.1, p.108-136, Feb. 1996 [doi>10.1145/225535.225539]
• 30: Bruce L. Worthington , Gregory R. Ganger , Yale N. Patt , John Wilkes, On-line extraction of SCSI disk drive parameters, Proceedings of the 1995 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems, p.146-156, May 15-19, 1995, Ottawa, Ontario, Canada

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.2 Storage Management
          Subjects: Storage hierarchies

General Terms:
Measurement, Performance

Keywords:
RAID, storage