LegionFS: a secure and scalable file system supporting cross-domain high-performance applications

ABSTRACT

Realizing that current file systems can not cope with the diverse requirements of wide-area collaborations, researchers have developed data access facilities to meet their needs. Recent work has focused on comprehensive data access architectures. In order to fulfill the evolving requirements in this environment, we suggest a more fully-integrated architecture built upon the fundamental tenets of naming, security, scalability, extensibility, and adaptability. These form the underpinning of the Legion File System (LegionFS). This paper motivates the need for these requirements and presents benchmarks that highlight the scalability of LegionFS. LegionFS aggregate throughput follows the linear growth of the network, yielding an aggregate read bandwidth of 193.8 MB/s on a 100 Mbps Ethernet backplane with 50 simultaneous readers. The serverless architecture of LegionFS is shown to benefit important scientific applications, such as those accessing the Protein Data Bank, within both local- and wide-area environments.

REFERENCES

• 1: Gridftp: Ftp extensions for the grid. Grid Forum Remote Data Access group, October 2000.
• 2: A. D. Alexandrov, M. Ibel, K. E. Schauser, and C. J. Scheiman. Extending the operating system at the user level: the ufo global file system. In 1997 Annual Technical Conference on Unix and Advanced Computing Systems (USENIX '97), January 1997.
• 3: R. Altman and R. Moore. Knowledge from biological data collections. enVision, 16(2), April 2000.
• 4: T. E. Anderson , M. D. Dahlin , J. M. Neefe , D. A. Patterson , D. S. Roselli , R. Y. Wang, Serverless network file systems, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.109-126, December 03-06, 1995, Copper Mountain, Colorado, United States
• 5: Chaitanya Baru , Reagan Moore , Arcot Rajasekar , Michael Wan, The SDSC storage resource broker, Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research, p.5, November 30-December 03, 1998, Toronto, Ontario, Canada
• 6: Joseph Bester , Ian Foster , Carl Kesselman , Jean Tedesco , Steven Tuecke, GASS: a data movement and access service for wide area computing systems, Proceedings of the sixth workshop on I/O in parallel and distributed systems, p.78-88, May 05-05, 1999, Atlanta, Georgia, United States [doi>10.1145/301816.301839]
• 7: William J. Bolosky , John R. Douceur , David Ely , Marvin Theimer, Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs, Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.34-43, June 18-21, 2000, Santa Clara, California, United States
• 8: Pei Cao , Edward W. Felten , Anna R. Karlin , Kai Li, A study of integrated prefetching and caching strategies, Proceedings of the 1995 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems, p.188-197, May 15-19, 1995, Ottawa, Ontario, Canada
• 9: A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, and S. Tuecke. The data grid: Towards an architecture for the distributed management and analysis of large scientific datasets. Journal of Network and Computer Applications, 1999.
• 10: Kenneth M. Curewitz , P. Krishnan , Jeffrey Scott Vitter, Practical prefetching via data compression, Proceedings of the 1993 ACM SIGMOD international conference on Management of data, p.257-266, May 25-28, 1993, Washington, D.C., United States
• 11: Peter Druschel , Antony Rowstron, PAST: A Large-Scale, Persistent Peer-to-Peer Storage Utility, Proceedings of the Eighth Workshop on Hot Topics in Operating Systems, p.75, May 20-22, 2001
• 12: Adam Ferrari , Frederick Knabe , Marty Humphrey , Steve Chapin , Andrew Grimshaw, A Flexible Security System for Metacomputing Environments, University of Virginia, Charlottesville, VA, 1998
• 13: Renato J. Figueiredo , Nirav H. Kapadia , José A. B. Fortes, The PUNCH Virtual File System: Seamless Access to Decentralized Storage Services in a Computational Grid, Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing (HPDC-10'01), p.334, August 07-09, 2001
• 14: I. Foster and C. Kesselman. Globus: A metacomputing infrastructure toolkit. International Journal of Supercomputer Applications, 11(2):115-128, 1997.
• 15: R. Golding, P. Bosch, C. Staelin, T. Sullivan, and J. Wilkes. Idleness is not sloth. In USENIX Technical Conference, pages 201-212, January 1995.
• 16: C. Gray , D. Cheriton, Leases: an efficient fault-tolerant mechanism for distributed file cache consistency, Proceedings of the twelfth ACM symposium on Operating systems principles, p.202-210, November 1989
• 17: Andrew Grimshaw , Adam Ferrari , Frederick Knabe , Marty Humphrey, Wide-Area Computing: Resource Sharing on a Large Scale, Computer, v.32 n.5, p.29-37, May 1999 [doi>10.1109/2.762790]
• 18: Björn Grönvall , Assar Westerlund , Stephen Pink, The design of a multicast-based distributed file system, Proceedings of the third symposium on Operating systems design and implementation, p.251-264, February 1999, New Orleans, Louisiana, United States
• 19: R. G. Guy, J. S. Heidemann, W. Mak, J. Thomas W. Page, G. J. Popek, and D. Rothmeier. Implementation of the ficus replicated file system. In USENIX Conference Proceedings, Berkeley, CA, June 1990. USENIX Association.
• 20: John S. Heidemann , Gerald J. Popek, File-system development with stackable layers, ACM Transactions on Computer Systems (TOCS), v.12 n.1, p.58-89, Feb. 1994 [doi>10.1145/174613.174616]
• 21: John H. Howard , Michael L. Kazar , Sherri G. Menees , David A. Nichols , M. Satyanarayanan , Robert N. Sidebotham , Michael J. West, Scale and performance in a distributed file system, ACM Transactions on Computer Systems (TOCS), v.6 n.1, p.51-81, Feb. 1988 [doi>10.1145/35037.35059]
• 22: M. L. Kazar, B. W. Leverett, O. T. Anderson, V. Apostolides, B. A. Bottos, S. Chutani, C. F. Everhart, W. A. Mason, S.-T. Tu, and E. R. Zayas. Decorum file system architectural overview. In Proceedings of the 1990 Summer USENIX Conference, pages 151-163, Anaheim, CA, June 1990. USENIX Association.
• 23: Yousef A. Khalidi , Michael N. Nelson, Extensible file systems in spring, Proceedings of the fourteenth ACM symposium on Operating systems principles, p.1-14, December 05-08, 1993, Asheville, North Carolina, United States
• 24: James J. Kistler , M. Satyanarayanan, Disconnected operation in the Coda file system, Proceedings of the thirteenth ACM symposium on Operating systems principles, p.213-225, October 13-16, 1991, Pacific Grove, California, United States
• 25: Orran Krieger , Michael Stumm, HFS: a performance-oriented flexible file system based on building-block compositions, ACM Transactions on Computer Systems (TOCS), v.15 n.3, p.286-321, Aug. 1997 [doi>10.1145/263326.263356]
• 26: T. M. Kroeger and D. D. E. Long. The case for efficient file access pattern modeling. In Proceedings of the 1996 USENIX Technical Conference, January 1996.
• 27: Y. G. Leclerc, M. Reddy, L. Iverson, and N. Bletter. Terravisionii: An overview. Technical report, SRI International, 2000.
• 28: Greg Lindahl , Steve J. Chapin , Norman Beekwilder , Andrew Grimshaw, Experiences with Legion on the Centurion Cluster, University of Virginia, Charlottesville, VA, 1998
• 29: Tara M. Madhyastha , Daniel A. Reed, Input/output access pattern classification using hidden Markov models, Proceedings of the fifth workshop on I/O in parallel and distributed systems, p.57-67, November 17-17, 1997, San Jose, California, United States [doi>10.1145/266220.266226]
• 30: Yousef A. Khalidi , Vlada Matena , Ken Shirriff, Solaris MC File System Framework, Sun Microsystems, Inc., Mountain View, CA, 1996
• 31: David Mazières , Michael Kaminsky , M. Frans Kaashoek , Emmett Witchel, Separating key management from file system security, Proceedings of the seventeenth ACM symposium on Operating systems principles, p.124-139, December 12-15, 1999, Charleston, South Carolina, United States
• 32: J. G. Mitchell, J. J. Gibbons, G. Hamilton, P. B. Kessler, Y. A. Khalidi, P. Kougiouris, P. W. Madany, M. N. Nelson, M. L. Powell, and S. R. Radia. An overview of the spring system. In CompCon Conference Proceedings, 1994.
• 33: Yousef A. Khalidi , Peter W. Madany, Spring File System, The, Sun Microsystems, Inc., Mountain View, CA, 1993
• 34: Brian D. Noble , M. Satyanarayanan , Dushyanth Narayanan , James Eric Tilton , Jason Flinn , Kevin R. Walker, Agile application-aware adaptation for mobility, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.276-287, October 05-08, 1997, Saint Malo, France
• 35: R. H. Patterson , G. A. Gibson , E. Ginting , D. Stodolsky , J. Zelenka, Informed prefetching and caching, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.79-95, December 03-06, 1995, Copper Mountain, Colorado, United States
• 36: R. Raman , M. Livny , M. Solomon, Matchmaking: Distributed Resource Management for High Throughput Computing, Proceedings of the The Seventh IEEE International Symposium on High Performance Distributed Computing, p.140, July 28-31, 1998
• 37: R. Sandberg, D. Goldberg, S. Kleiman, D. Walsh, and B. Lyon. Design and implementation of the sun network filesystem. In USENIX Conference Proceedings, Berkeley, CA, Summer 1985. USENIX Association.
• 38: Mahadev Satyanarayanan, Scalable, Secure, and Highly Available Distributed File Access, Computer, v.23 n.5, p.9-18, 20-21, May 1990 [doi>10.1109/2.53351]
• 39: G. C. Skinner and T. K. Wong. "stacking" vnodes: A progress report. In USENIX Conference Proceedings, pages 61-74. USENIX Association, Summer 1993.
• 40: A. M. Vahdat, P. C. Eastham, and T. E. Anderson. Webfs: A global cache coherent file system. Technical report, University of California, Berkeley, 1996.
• 41: Sudharshan Vazhkudai , Steven Tuecke , Ian Foster, Replica Selection in the Globus Data Grid, Proceedings of the 1st International Symposium on Cluster Computing and the Grid, p.106, May 15-18, 2001
• 42: Brian S. White , Andrew S. Grimshaw , Anh Nguyen-Tuong, Grid-Based File Access: The Legion I/O Model, Proceedings of the Ninth IEEE International Symposium on High Performance Distributed Computing (HPDC'00), p.165, August 01-04, 2000
• 43: John Wilkes , Richard Golding , Carl Staelin , Tim Sullivan, The HP AutoRAID hierarchical storage system, ACM Transactions on Computer Systems (TOCS), v.14 n.1, p.108-136, Feb. 1996 [doi>10.1145/225535.225539]
• 44: Rich Wolski , Neil T. Spring , Jim Hayes, The network weather service: a distributed resource performance forecasting service for metacomputing, Future Generation Computer Systems, v.15 n.5-6, p.757-768, Oct. 1999 [doi>10.1016/S0167-739X(99)00025-4]
• 45: E. Zadok and J. Nieh. Fist: A language for stackable file systems. In Proceedings of the 2000 USENIX Annual Technical Conference, San Diego, California, June 2000. USENIX Association.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.5 Local and Wide-Area Networks
          Subjects: Access schemes

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems
          Subjects: Distributed applications

  E. Data
  E.5 FILES
      Subjects: Organization/structure

General Terms:
Algorithms, Design, Performance