How to scale transactional storage systems

ABSTRACT

Applications of the future will need to support large numbers of clients and will require scalable storage systems that allow state to be shared reliably. Recent research in distributed file systems provides technology that increases the scalability of storage systems. But file systems only support sharing with weak consistency guarantees and can not support applications that require transactional consistency. The challenge is how to provide scalable storage systems that support transactional applications.We are developing technology for scalable transactional storage systems. Our approach combines scalable caching and coherence techniques developed in serverless file systems and DSM systems, with recovery techniques developed in traditional databases. This position paper describes the design rationale for split caching, a new scalable memory management technique for network-based transactional object storage systems, and fragment reconstruction, a new coherence protocol that supports fine-grained sharing.

REFERENCES

• 1: Atul Adya , Robert Gruber , Barbara Liskov , Umesh Maheshwari, Efficient optimistic concurrency control using loosely synchronized clocks, Proceedings of the 1995 ACM SIGMOD international conference on Management of data, p.23-34, May 22-25, 1995, San Jose, California, United States
• 2: B. Liskov , A. Adya , M. Castro , S. Ghemawat , R. Gruber , U. Maheshwari , A. C. Myers , M. Day , L. Shrira, Safe and efficient sharing of persistent objects in Thor, Proceedings of the 1996 ACM SIGMOD international conference on Management of data, p.318-329, June 04-06, 1996, Montreal, Quebec, Canada
• 3: Michael J. Carey , Michael J. Franklin , Markos Zaharioudakis, Fine-grained sharing in a page server OODBMS, Proceedings of the 1994 ACM SIGMOD international conference on Management of data, p.359-370, May 24-27, 1994, Minneapolis, Minnesota, United States
• 4: Michael J. Carey , Michael J. Franklin , Miron Livny , Eugene J. Shekita, Data caching tradeoffs in client-server DBMS architectures, Proceedings of the 1991 ACM SIGMOD international conference on Management of data, p.357-366, May 29-31, 1991, Denver, Colorado, United States
• 5: M. Dahlin, R. Wang, T.Anderson, and D. Patterson. Cooperative Caching: Using Remote Client Memory to Improve File System Performance. In Proceedings of OSDI, 1994.
• 6: M. J. Feeley , W. E. Morgan , E. P. Pighin , A. R. Karlin , H. M. Levy , C. A. Thekkath, Implementing global memory management in a workstation cluster, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.201-212, December 03-06, 1995, Copper Mountain, Colorado, United States
• 7: Michael J. Franklin , Michael J. Carey , Miron Livny, Global Memory Management in Client-Server Database Architectures, Proceedings of the 18th International Conference on Very Large Data Bases, p.596-609, August 23-27, 1992
• 8: Sanjay Ghemawat , Barbara H. Liskov , M. Frans Kaashoek, The modified object buffer: a storage management technique for object-oriented databases, 1995
• 9: Robert E. Gruber , Barbara H. Liskov, Optimism vs. locking: a study of concurrency control for client-server object-oriented databases, 1997
• 10: M.Feeley, J.Chase, V.Narasayya, and H.Levy. Integrating Coherency and Recoverability in Distributed Systems. In Proceedings o} OSDI, 1994.
• 11: D. Muntz and P. Honeyman. Multi-level Caching in Distributed File Systems or Your Cache ain't nothin' but trash. In Winter Useniz Technical ConJerenee, 1992.
• 12: J. O'Toole , L. Shrira, Shared data management needs adaptive methods, Proceedings of the Fifth Workshop on Hot Topics in Operating Systems (HotOS-V), p.129, May 04-05, 1995
• 13: James O'Toole and Liuba Shrira. Opportunistic Log: Efficient Installation Reads in a Reliable Object Server. In Proceedings of OSDI, 1994.
• 14: Thomas E. Anderson , Michael D. Dahlin , Jeanna M. Neefe , David A. Patterson , Drew S. Roselli , Randolph Y. Wang, Serverless network file systems, ACM Transactions on Computer Systems (TOCS), v.14 n.1, p.41-79, Feb. 1996 [doi>10.1145/225535.225537]
• 15: Seth J. White , David J. DeWitt, Implementing crash recovery in QuickStore: a performance study, Proceedings of the 1995 ACM SIGMOD international conference on Management of data, p.187-198, May 22-25, 1995, San Jose, California, United States