Petal: distributed virtual disks

ABSTRACT

The ideal storage system is globally accessible, always available, provides unlimited performance and capacity for a large number of clients, and requires no management. This paper describes the design, implementation, and performance of Petal, a system that attempts to approximate this ideal in practice through a novel combination of features. Petal consists of a collection of network-connected servers that cooperatively manage a pool of physical disks. To a Petal client, this collection appears as a highly available block-level storage system that provides large abstract containers called virtual disks. A virtual disk is globally accessible to all Petal clients on the network. A client can create a virtual disk on demand to tap the entire capacity and performance of the underlying physical resources. Furthermore, additional resources, such as servers and disks, can be automatically incorporated into Petal.We have an initial Petal prototype consisting of four 225 MHz DEC 3000/700 workstations running Digital Unix and connected by a 155 Mbit/s ATM network. The prototype provides clients with virtual disks that tolerate and recover from disk, server, and network failures. Latency is comparable to a locally attached disk, and throughput scales with the number of servers. The prototype can achieve I/O rates of up to 3150 requests/sec and bandwidth up to 43.1 Mbytes/sec.

REFERENCES

• 1: Thomas E. Anderson , Michael D. Dahlin , Jeanna M. Neefe , David A. Patterson , Drew S. Roselli , Randolph Y. Wang, Serverless network file systems, ACM Transactions on Computer Systems (TOCS), v.14 n.1, p.41-79, Feb. 1996 [doi>10.1145/225535.225537]
• 2: Thomas E. Anderson , Susan S. Owicki , James B. Saxe , Charles P. Thacker, High-speed switch scheduling for local-area networks, ACM Transactions on Computer Systems (TOCS), v.11 n.4, p.319-352, Nov. 1993 [doi>10.1145/161541.161736]
• 3: Andrew D. Birrell , Bruce Jay Nelson, Implementing remote procedure calls, ACM Transactions on Computer Systems (TOCS), v.2 n.1, p.39-59, February 1984 [doi>10.1145/2080.357392]
• 4: Luis-Felipe Cabrera and Darrel D. E. Long. Swift: Using distributed disk striping to provide high I/O data rates. ACM Computing Systems, 4:405--436, Fall 1991.
• 5: Pei Cao , Swee Boon Lin , Shivakumar Venkataraman , John Wilkes, The TickerTAIP parallel RAID architecture, ACM Transactions on Computer Systems (TOCS), v.12 n.3, p.236-269, Aug. 1994 [doi>10.1145/185514.185517]
• 6: C. Chao, R. English, D. Jacobson, A. Stepanov, and J. Wilkes. Mime: A high performance parallel storage device with strong recovery guarantees. Technical Report HPL-CSP-92- 9, Hewlett-Packard Laboratories, November 1992.
• 7: Peter M. Chen , Edward K. Lee , Ann L. Drapeau , Ken Lutz , Ethan L. Miller , Srinivasan Seshan , Ken Shirriff , David A. Patterson , Randy H. Katz, Performance and design evaluation of the RAID-II storage server, Distributed and Parallel Databases, v.2 n.3, p.243-260, July 1994 [doi>10.1007/BF01266330]
• 8: Wiebren de Jonge , M. Frans Kaashoek , Wilson C. Hsieh, The logical disk: a new approach to improving file systems, Proceedings of the fourteenth ACM symposium on Operating systems principles, p.15-28, December 05-08, 1993, Asheville, North Carolina, United States
• 9: Peter Druschel , Larry L. Peterson , Bruce S. Davie, Experiences with a high-speed network adaptor: a software perspective, Proceedings of the conference on Communications architectures, protocols and applications, p.2-13, August 31-September 02, 1994, London, United Kingdom
• 10: R. M. English and A. A. Stepanov. Loge: A self-organizing disk controller. In Proceedings of the Winter 1992 USENIX Conference, pages 237-251, January 1992.
• 11: Garth A. Gibson, David F. Nagle, KhaliI Amid, Fay W. Chang, Eugene Feinberg, Howard Gobioff, Chen Lee, Berend Ozceri, Erik Riedel, and David Rochberg. A case for network-attached secure disks. Technical Report CMU-CS- 96-142, Department of Electrical and Computer Engineering, Carnegie-Mellon University, June 1996.
• 12: John H. Hartman , John K. Ousterhout, The Zebra striped network file system, ACM Transactions on Computer Systems (TOCS), v.13 n.3, p.274-310, Aug. 1995 [doi>10.1145/210126.210131]
• 13: Hui-I Hsiao and David J. DeWitt. Chained declustering: A new availability strategy for multiprocessor database machines. Technical Report CS TR 854, University of Wisconsin, Madison, June 1989.
• 14: Leslie Lamport. The Part-Time Parliament. Technical Report 49, Digital Equipment Corporation, Systems Research Center, 130 Lytton Ave., Palo Alto, CA 94301-1044, September 1989.
• 15: Timothy Mann , Andrew Birrell , Andy Hisgen , Charles Jerian , Garret Swart, A coherent distributed file cache with directory write-behind, ACM Transactions on Computer Systems (TOCS), v.12 n.2, p.123-164, May 1994 [doi>10.1145/176575.176577]
• 16: Mahadev Satyanarayanan, Scalable, Secure, and Highly Available Distributed File Access, Computer, v.23 n.5, p.9-18, 20-21, May 1990 [doi>10.1109/2.53351]
• 17: Daniel Stodolsky , Mark Holland , William V. Courtright, II , Garth A. Gibson, Parity logging disk arrays, ACM Transactions on Computer Systems (TOCS), v.12 n.3, p.206-235, Aug. 1994 [doi>10.1145/185514.185516]
• 18: Chandramohan A. Thekkath , Henry M. Levy, Limits to low-latency communication on high-speed networks, ACM Transactions on Computer Systems (TOCS), v.11 n.2, p.179-203, May 1993 [doi>10.1145/151244.151247]
• 19: J. Wilkes , R. Golding , C. Staelin , T. Sullivan, The HP AutoRAID hierarchical storage system, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.96-108, December 03-06, 1995, Copper Mountain, Colorado, United States

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.2 Storage Management
          Subjects: Virtual memory

Additional Classification:
  B. Hardware
  B.3 MEMORY STRUCTURES
      B.3.2 Design Styles
          Subjects: Virtual memory

  D. Software
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design
          Subjects: Distributed systems
      D.4.8 Performance

General Terms:
Design, Performance, Reliability