Separating Abstractions from Resources in a Tactical Storage System

ABSTRACT

Sharing data and storage space in a distributed system remains a difficult task for ordinary users, who are constrained to the fixed abstractions and resources provided by administrators. To remedy this situation, we introduce the concept of a tactical storage system (TSS) that separates storage abstractions from storage resources, leaving users free to create, reconfigure, and destroy abstractions as their needs change. In this paper, we describe how a TSS can provide a variety of filesystem and database abstractions for unmodified applications without requiring special privileges or kernel changes. A TSS provides performance competitive with NFS for single clients and also scales well for multiple servers and multiple clients. A prototype TSS of 120 disks and 6 TB of storage has been deployed at the University of Notre Dame and used for applications in high energy physics and bioinformatics.

REFERENCES

• 1: Atul Adya , William J. Bolosky , Miguel Castro , Gerald Cermak , Ronnie Chaiken , John R. Douceur , Jon Howell , Jacob R. Lorch , Marvin Theimer , Roger P. Wattenhofer, Farsite: federated, available, and reliable storage for an incompletely trusted environment, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060291]
• 2: [2] W. Allcock, A. Chervenak, I. Foster, C. Kesselman, and S. Tuecke. Protocols and services for distributed data-intensive science. In Proceedings of Advanced Computing and Analysis Techniques in Physics Research, pages 161-163, 2000.
• 3: Chaitanya Baru , Reagan Moore , Arcot Rajasekar , Michael Wan, The SDSC storage resource broker, Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research, p.5, November 30-December 03, 1998, Toronto, Ontario, Canada
• 4: Sergey Brin , Lawrence Page, The anatomy of a large-scale hypertextual Web search engine, Computer Networks and ISDN Systems, v.30 n.1-7, p.107-117, April 1, 1998
• 5: [5] Cluster File Systems. Lustre: A scalable, high performance file system. white paper, November 2002.
• 6: Karl Czajkowski , Ian T. Foster , Nicholas T. Karonis , Carl Kesselman , Stuart Martin , Warren Smith , Steven Tuecke, A Resource Management Architecture for Metacomputing Systems, Proceedings of the Workshop on Job Scheduling Strategies for Parallel Processing, p.62-82, March 30, 1998
• 7: Ian Foster , Carl Kesselman , Gene Tsudik , Steven Tuecke, A security architecture for computational grids, Proceedings of the 5th ACM conference on Computer and communications security, p.83-92, November 02-05, 1998, San Francisco, California, United States [doi>10.1145/288090.288111]
• 8: James Frey , Todd Tannenbaum , Miron Livny , Ian Foster , Steven Tuecke, Condor-G: A Computation Management Agent for Multi-Institutional Grids, Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing, p.55, August 07-09, 2001
• 9: I. Foster , J. Gieraltowski , S. Gose , N. Maltsev , E. May , A. Rodriguez , D. Sulakhe , A. Vaniachine , J. Shank , S. Youssef , D. Adams , R. Baker , W. Deng , J. Smith , D. Yu , I. Legrand , S. Singh , C. Steenberg , Y. Xia , A. Afaq , E. Berman , J. Annis , L. A. T. Bauerdick , M. Ernst , I. Fisk , L. Giacchetti , G. Graham , A. Heavey , J. Kaiser , N. Kuropatkin , R. Pordes , V. Sekhri , J. Weigand , Y. Wu , K. Baker , L. Sorrillo , J. Huth , M. Allen , L. Grundhoefer , J. Hicks , F. Luehring , S. Peck , R. Quick , S. Simms , G. Fekete , J. vandenBerg , K. Cho , K. Kwon , D. Son , H. Park , S. Canon , K. Jackson , D. E. Konerding , J. Lee , D. Olson , I. Sakrejda , B. Tierney , M. Green , R. Miller , J. Letts , T. Martin , D. Bury , C. Dumitrescu , D. Engh , R. Gardner , M. Mambelli , Y. Smirnov , J. Voeckler , M. Wilde , Y. Zhao , X. Zhao , P. Avery , R. Cavanaugh , B. Kim , C. Prescott , J. Rodriguez , A. Zahn , S. McKee , C. Jordan , J. Prewett , T. Thomas , H. Severini , B. Clifford , E. Deelman , L. Flon , C. Kesselman , G. Mehta , N. Olomu , K. Vahi , K. De , P. McGuigan , M. Sosebee , D. Bradley , P. Couvares , A. De Smet , C. Kireyev , E. Paulson , A. Roy , S. Koranda , B. Moe , B. Brown , P. Sheldon, The Grid2003 Production Grid: Principles and Practice, Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing, p.236-245, June 04-06, 2004 [doi>10.1109/HPDC.2004.36]
• 10: Sanjay Ghemawat , Howard Gobioff , Shun-Tak Leung, The Google file system, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 11: John H. Howard , Michael L. Kazar , Sherri G. Menees , David A. Nichols , M. Satyanarayanan , Robert N. Sidebotham , Michael J. West, Scale and performance in a distributed file system, ACM Transactions on Computer Systems (TOCS), v.6 n.1, p.51-81, Feb. 1988 [doi>10.1145/35037.35059]
• 12: James J. Kistler , M. Satyanarayanan, Disconnected operation in the Coda file system, ACM SIGOPS Operating Systems Review, v.25 n.5, p.213-225, Oct. 1991
• 13: Sander Klous , Jaime Frey , Se-Chang Son , Douglas Thain , Alain Roy , Miron Livny , Jo van den Brand, Transparent access to Grid resources for user software: Research Articles, Concurrency and Computation: Practice & Experience, v.18 n.7, p.787-801, June 2006 [doi>10.1002/cpe.v18:7]
• 14: Nancy P. Kronenberg , Henry M. Levy , William D. Strecker, VAXcluster: a closely-coupled distributed system, ACM Transactions on Computer Systems (TOCS), v.4 n.2, p.130-146, May 1986 [doi>10.1145/214419.214421]
• 15: John Kubiatowicz , David Bindel , Yan Chen , Steven Czerwinski , Patrick Eaton , Dennis Geels , Ramakrishna Gummadi , Sean Rhea , Hakim Weatherspoon , Chris Wells , Ben Zhao, OceanStore: an architecture for global-scale persistent storage, Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, p.190-201, November 2000, Cambridge, Massachusetts, United States
• 16: Butler W. Lampson, Hints for computer system design, Proceedings of the ninth ACM symposium on Operating systems principles, p.33-48, October 10-13, 1983, Bretton Woods, New Hampshire, United States
• 17: Edward K. Lee , Chandramohan A. Thekkath, Petal: distributed virtual disks, Proceedings of the seventh international conference on Architectural support for programming languages and operating systems, p.84-92, October 01-04, 1996, Cambridge, Massachusetts, United States
• 18: John MacCormick , Nick Murphy , Marc Najork , Chandramohan A. Thekkath , Lidong Zhou, Boxwood: abstractions as the foundation for storage infrastructure, Proceedings of the 6th conference on Symposium on Opearting Systems Design & Implementation, p.8-8, December 06-08, 2004, San Francisco, CA
• 19: [19] T. Matthey and J. Izaguirre. ProtoMol: A molecular dynamic framework with incremental parallelization. In SIAM Conference on Parallel Processing for Scientific Computing, March 2001.
• 20: David Mazieres , M. Frans Kaashoek, Self-certifying file system, Massachusetts Institute of Technology, 2000
• 21: David Mazières, A Toolkit for User-Level File Systems, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.261-274, June 25-30, 2001
• 22: [22] M. Mesnier, G. Ganger, and E. Riedel. Object based storage. IEEE Communications, 41(8), August 2003.
• 23: Kalman Z. Meth , Julian Satran, Design of the iSCSI Protocol, Proceedings of the 20 th IEEE/11 th NASA Goddard Conference on Mass Storage Systems and Technologies (MSS'03), p.116, April 07-10, 2003
• 24: Sape J. Mullender , Guido van Rossum , Andrew S. Tanenbaum , Robbert van Renesse , Hans van Staveren, Amoeba: A Distributed Operating System for the 1990s, Computer, v.23 n.5, p.44-53, May 1990 [doi>10.1109/2.53354]
• 25: David A. Patterson , Garth Gibson , Randy H. Katz, A case for redundant arrays of inexpensive disks (RAID), Proceedings of the 1988 ACM SIGMOD international conference on Management of data, p.109-116, June 01-03, 1988, Chicago, Illinois, United States
• 26: [26] J. Plank, M. Beck, W. Elwasif, T. Moore, M. Swany, and R. Wolski. The Internet Backplane Protocol: Storage in the network. In Proceedings of the Network Storage Symposium, 1999.
• 27: J. Postel, File Transfer Protocol specification, RFC Editor, 1980
• 28: [28] B. Randell. Recursively structured distributed computing systems. In Symposium on Reliable Distributed Computing Systems, pages 3-11, 1983.
• 29: [29] R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh, and B. Lyon. Design and implementation of the Sun network filesystem. In Proceedings of the USENIX Summer Technical Conference, pages 119-130, 1985.
• 30: [30] J. Steiner, C. Neuman, and J. I. Schiller. Kerberos: An authentication service for open network systems. In Proceedings of the USENIX Winter Technical Conference, pages 191-200, 1988.
• 31: Douglas Thain , Miron Livny, Error Scope on a Computational Grid: Theory and Practice, Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing, p.199, July 24-26, 2002
• 32: [32] D. Thain and M. Livny. Parrot: Transparent user-level middleware for data-intensive computing. In Proceedings of the Workshop on Adaptive Grid Middleware, New Orleans, September 2003.
• 33: [33] J. Wozniak, P. Brenner, D. Thain, A. Striegel, and J. Izaguirre. Generosity and gluttony in GEMS: Grid enabled molecular simulations. In IEEE Symposium on High Performance Distributed Computing, July 2005.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.3 File Systems Management
          Subjects: Distributed file systems

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
  C.4 PERFORMANCE OF SYSTEMS

General Terms:
Design, Experimentation, Measurement, Performance