Evolving RPC for active storage

ABSTRACT

We introduce Scriptable RPC (SRPC), an RPC-based framework that enables distributed system services to take advantage of active components. Technology trends point to a world where each component in a system (whether disk, network interface, or memory) has substantial computational capabilities; however, traditional methods of building distributed services are not designed to take advantage of these new architectures, mandating wholesale change of the software base to exploit more powerful hardware. In contrast, SRPC provides a direct and simple migration path for traditional services into the active environment.We demonstrate the power and flexibility of the SRPC framework through a series of case studies, with a focus on active storage servers. Specifically, we find three advantages to our approach. First, SRPC improves the performance of distributed file servers, reducing latency by combining the execution of operations at the file server. Second, SRPC enables the ready addition of new functionality; for example, more powerful cache consistency models can be realized on top of a server that exports a simple NFS-like interface. Third, SRPC simplifies the construction of distributed services; operations that are difficult to coordinate across client and server can now be co-executed at the server, thus avoiding costly agreement and crash-recovery protocols.

REFERENCES

• 1: A. Acharya, M. Uysal, and J. Saltz. Active Disks. In Proceedings of the 8th Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS VIII), San Jose, CA, October 1998.
• 2: K. Amiri, D. Petrou, G. R. Ganger, and G. A. Gibson. Dynamic Function Placement for Data-intensive Cluster Computing. In Proceedings of the 2000 USENIX Annual Technical Conference, pages 307-322, June 2000.
• 3: Interposed request routing for scalable network storage, ACM Transactions on Computer Systems (TOCS), v.20 n.1, p.25-48, February 2002 [doi>10.1145/505452.505454]
• 4: T. Berners-Lee, R. T. Fielding, H. F. Nielsen, J. Gettys, and J. Mogul. Hypertext Transfer Protocol -- HTTP/1.1. Technical Report 2068, Internet Engineering Task Force, January 1997.
• 5: B. N. Bershad , S. Savage , P. Pardyak , E. G. Sirer , M. E. Fiuczynski , D. Becker , C. Chambers , S. Eggers, Extensibility safety and performance in the SPIN operating system, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.267-283, December 03-06, 1995, Copper Mountain, Colorado, United States
• 6: Andrew D. Birrell , Bruce Jay Nelson, Implementing remote procedure calls, ACM Transactions on Computer Systems (TOCS), v.2 n.1, p.39-59, February 1984 [doi>10.1145/2080.357392]
• 7: Nanette J. Boden , Danny Cohen , Robert E. Felderman , Alan E. Kulawik , Charles L. Seitz , Jakov N. Seizovic , Wen-King Su, Myrinet: A Gigabit-per-Second Local Area Network, IEEE Micro, v.15 n.1, p.29-36, February 1995 [doi>10.1109/40.342015]
• 8: R. H. Campbell , See-Mong Tan, /spl mu/Choices: an object-oriented multimedia operating system, Proceedings of the Fifth Workshop on Hot Topics in Operating Systems (HotOS-V), p.90, May 04-05, 1995
• 9: J. Bradley Chen , Brian N. Bershad, The impact of operating system structure on memory system performance, Proceedings of the fourteenth ACM symposium on Operating systems principles, p.120-133, December 05-08, 1993, Asheville, North Carolina, United States
• 10: M. Dahlin, R. Wang, T. Anderson, and D. Patterson. Cooperative Caching: Using Remote Client Memory to Improve File System Performance. In Proceedings of the 1st USENIX Symposium on Operating Systems Design and Implementation, pages 267-280, November 14-17 1994.
• 11: Alan M. Davis, 201 principles of software development, McGraw-Hill, Inc., New York, NY, 1995
• 12: Timothy E. Denehy , Andrea C. Arpaci-Dusseau , Remzi H. Arpaci-Dusseau, Bridging the Information Gap in Storage Protocol Stacks, Proceedings of the General Track: 2002 USENIX Annual Technical Conference, p.177-190, June 10-15, 2002
• 13: D. J. DeWitt and P. B. Hawthorn. A Performance Evaluation of Data Base Machine Architectures. In Proceedings of the Seventh Annual Conference Very Large Data Bases (VLDB '81), pages 199-214, 1981.
• 14: Eric Eide , Kevin Frei , Bryan Ford , Jay Lepreau , Gary Lindstrom, Flick: a flexible, optimizing IDL compiler, Proceedings of the ACM SIGPLAN 1997 conference on Programming language design and implementation, p.44-56, June 16-18, 1997, Las Vegas, Nevada, United States
• 15: D. R. Engler , M. F. Kaashoek , J. O'Toole, Jr., Exokernel: an operating system architecture for application-level resource management, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.251-266, December 03-06, 1995, Copper Mountain, Colorado, United States
• 16: M. Fiuczynski, R. Martin, B. Bershad, and D. Culler. SPINE: An operating system for intelligent network adapters. Technical Report TR-98-08-01, University of Washington, Department of Computer Science and Engineering, August 1998.
• 17: Garth A. Gibson , David F. Nagle , Khalil Amiri , Jeff Butler , Fay W. Chang , Howard Gobioff , Charles Hardin , Erik Riedel , David Rochberg , Jim Zelenka, A cost-effective, high-bandwidth storage architecture, Proceedings of the eighth international conference on Architectural support for programming languages and operating systems, p.92-103, October 02-07, 1998, San Jose, California, United States
• 18: Garth A. Gibson , David F. Nagle , Khalil Amiri , Fay W. Chang , Eugene M. Feinberg , Howard Gobioff , Chen Lee , Berend Ozceri , Erik Riedel , David Rochberg , Jim Zelenka, File server scaling with network-attached secure disks, Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.272-284, June 15-18, 1997, Seattle, Washington, United States
• 19: G. A. Gibson, D. F. Nagle, K. Amiri, F. W. Chang, H. Gobioff, E. Riedel, D. Rochberg, and J. Zelenka. Filesystems for Network-Attached Secure Disks. Technical Report CMU-CS-97-118, Carnegie-Mellon University, 1997.
• 20: A. Gokhale and D. C. Schmidt. Measuring the Performance of the CORBA Internet Inter-ORB Protocol over ATM. Technical Report WUCS-97-09, Washington University at St. Louis, 1997.
• 21: Jim Gray, Storage Bricks Have Arrived, Proceedings of the 1st USENIX Conference on File and Storage Technologies, January 28-30, 2002, Monterey, CA
• 22: S. D. Gribble, E. A. Brewer, J. M. Hellerstein, and D. Culler. Scalable, Distributed Data Structures for Internet Service Construction. In Proceedings of the Fourth Symposium on Operating Systems Design and Implementation (OSDI '00), San Diego, CA, October 2000.
• 23: The Swarm Scalable Storage System, Proceedings of the 19th IEEE International Conference on Distributed Computing Systems, p.74, May 31-June 04, 1999
• 24: John H. Howard , Michael L. Kazar , Sherri G. Menees , David A. Nichols , M. Satyanarayanan , Robert N. Sidebotham , Michael J. West, Scale and performance in a distributed file system, ACM Transactions on Computer Systems (TOCS), v.6 n.1, p.51-81, Feb. 1988 [doi>10.1145/35037.35059]
• 25: J. Katcher. PostMark: A New File System Benchmark. Technical Report TR-3022, Network Appliance Inc., October 1997.
• 26: James J. Kistler , M. Satyanarayanan, Disconnected operation in the Coda File System, ACM Transactions on Computer Systems (TOCS), v.10 n.1, p.3-25, Feb. 1992 [doi>10.1145/146941.146942]
• 27: Edward K. Lee , Chandramohan A. Thekkath, Petal: distributed virtual disks, Proceedings of the seventh international conference on Architectural support for programming languages and operating systems, p.84-92, October 01-04, 1996, Cambridge, Massachusetts, United States
• 28: David E. Lowell , Peter M. Chen, Free transactions with Rio Vista, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.92-101, October 05-08, 1997, Saint Malo, France
• 29: Ken Mai , Tim Paaske , Nuwan Jayasena , Ron Ho , William J. Dally , Mark Horowitz, Smart Memories: a modular reconfigurable architecture, Proceedings of the 27th annual international symposium on Computer architecture, p.161-171, June 2000, Vancouver, British Columbia, Canada
• 30: Richard P. Martin , Amin M. Vahdat , David E. Culler , Thomas E. Anderson, Effects of communication latency, overhead, and bandwidth in a cluster architecture, Proceedings of the 24th annual international symposium on Computer architecture, p.85-97, June 01-04, 1997, Denver, Colorado, United States
• 31: Athicha Muthitacharoen , Benjie Chen , David Mazières, A low-bandwidth network file system, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 32: Michael N. Nelson , Brent B. Welch , John K. Ousterhout, Caching in the Sprite network file system, ACM Transactions on Computer Systems (TOCS), v.6 n.1, p.134-154, Feb. 1988 [doi>10.1145/35037.42183]
• 33: Christian Nester , Michael Philippsen , Bernhard Haumacher, A more efficient RMI for Java, Proceedings of the ACM 1999 conference on Java Grande, p.152-159, June 12-14, 1999, San Francisco, California, United States [doi>10.1145/304065.304117]
• 34: Sean O'Malley , Todd Proebsting , Allen Brady Montz, USC: a universal stub compiler, Proceedings of the conference on Communications architectures, protocols and applications, p.295-306, August 31-September 02, 1994, London, United Kingdom
• 35: J. K. Ousterhout. Tcl: An Embedable Command Language. In Proceedings of the 1990 USENIX Association Winter Conference, 1990.
• 36: D. Patterson, T. Anderson, N. Cardwell, R. Fromm, K. Keeton, C. Kozyrakis, R. Thomas, and K. Yelick. Intelligent RAM (IRAM): Chips That Remember And Compute. In 1997 IEEE International Solid-State Circuits Conference, San Francisco, CA, February 1997.
• 37: Erik Riedel , Garth A. Gibson , Christos Faloutsos, Active Storage for Large-Scale Data Mining and Multimedia, Proceedings of the 24rd International Conference on Very Large Data Bases, p.62-73, August 24-27, 1998
• 38: Theodore H. Romer , Dennis Lee , Geoffrey M. Voelker , Alec Wolman , Wayne A. Wong , Jean-Loup Baer , Brian N. Bershad , Henry M. Levy, The structure and performance of interpreters, Proceedings of the seventh international conference on Architectural support for programming languages and operating systems, p.150-159, October 01-04, 1996, Cambridge, Massachusetts, United States
• 39: Margo I. Seltzer , Yasuhiro Endo , Christopher Small , Keith A. Smith, Dealing with disaster: surviving misbehaved kernel extensions, Proceedings of the second USENIX symposium on Operating systems design and implementation, p.213-227, October 29-November 01, 1996, Seattle, Washington, United States
• 40: C. Small and M. Seltzer. A Comparison of OS Extension Technologies. In Proceedings of the 1996 USENIX Annual Technical Conference, January 1996.
• 41: V. Srinivasan , J. Mogul, Spritely NFS: experiments with cache-consistency protocols, Proceedings of the twelfth ACM symposium on Operating systems principles, p.44-57, November 1989
• 42: Chandramohan A. Thekkath , Timothy Mann , Edward K. Lee, Frangipani: a scalable distributed file system, Proceedings of the sixteenth ACM symposium on Operating systems principles, p.224-237, October 05-08, 1997, Saint Malo, France
• 43: Transaction Processing Council. TPC Benchmark B Standard Specification, Revision 3.2. Technical Report, 1990.
• 44: T. von Eicken , A. Basu , V. Buch , W. Vogels, U-Net: a user-level network interface for parallel and distributed computing (includes URL), Proceedings of the fifteenth ACM symposium on Operating systems principles, p.40-53, December 03-06, 1995, Copper Mountain, Colorado, United States
• 45: Thorsten von Eicken , David E. Culler , Seth Copen Goldstein , Klaus Erik Schauser, Active messages: a mechanism for integrated communication and computation, Proceedings of the 19th annual international symposium on Computer architecture, p.256-266, May 19-21, 1992, Queensland, Australia
• 46: Matt Welsh , David E. Culler, Achieving Robust, Scalable Cluster I/O in Java, Selected Papers from the 5th International Workshop on Languages, Compilers, and Run-Time Systems for Scalable Computers, p.16-31, May 25-27, 2000