Contributing storage using the transparent file system

ABSTRACT

Contributory applications allow users to donate unused resources on their personal computers to a shared pool. Applications such as SETI@home, Folding@home, and Freenet are now in wide use and provide a variety of services, including data processing and content distribution. However, while several research projects have proposed contributory applications that support peer-to-peer storage systems, their adoption has been comparatively limited. We believe that a key barrier to the adoption of contributory storage systems is that contributing a large quantity of local storage interferes with the principal user of the machine.

To overcome this barrier, we introduce the Transparent File System (TFS). TFS provides background tasks with large amounts of unreliable storage—all of the currently available space—without impacting the performance of ordinary file access operations. We show that TFS allows a peer-to-peer contributory storage system to provide 40% more storage at twice the performance when compared to a user-space storage mechanism. We analyze the impact of TFS on replication in peer-to-peer storage systems and show that TFS does not appreciably increase the resources needed for file replication.

REFERENCES

• 1: Atul Adya , William J. Bolosky , Miguel Castro , Gerald Cermak , Ronnie Chaiken , John R. Douceur , Jon Howell , Jacob R. Lorch , Marvin Theimer , Roger P. Wattenhofer, Farsite: federated, available, and reliable storage for an incompletely trusted environment, Proceedings of the 5th symposium on Operating systems design and implementation

  Due to copyright restrictions we are not able to make the PDFs for this conference available for downloading

  , December 09-11, 2002, Boston, Massachusetts [doi>10.1145/1060289.1060291]
• 2: Ranjita Bhagwan , Kiran Tati , Yu-Chung Cheng , Stefan Savage , Geoffrey M. Voelker, Total recall: system support for automated availability management, Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation, p.25-25, March 29-31, 2004, San Francisco, California
• 3: Charles Blake , Rodrigo Rodrigues, High availability, scalable storage, dynamic peer networks: pick two, Proceedings of the 9th conference on Hot Topics in Operating Systems, p.1-1, May 18-21, 2003, Lihue, Hawaii
• 4: William J. Bolosky , John R. Douceur , David Ely , Marvin Theimer, Feasibility of a serverless distributed file system deployed on an existing set of desktop PCs, Proceedings of the 2000 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.34-43, June 18-21, 2000, Santa Clara, California, United States
• 5: Card, R., Ts'o, T., and Tweedie, S. 1994. Design and implementation of the second extended filesystem. In Proceedings of the 1st Dutch International Symposium on Linux, Laboratoire MASI, Institut Blaise Pascal and Massachussets Institute of Technology and University of Edinburgh.
• 6: Chu, J., Labonte, K., and Levine, B. N. 2002. Availability and locality measurements of peer-to-peer file systems. In Proceedings of the 2nd Conference on Scalability and Traffic Control in IP Networks (ITCom).
• 7: James Cipar , Mark D. Corner , Emery D. Berger, Transparent contribution of memory, Proceedings of the annual conference on USENIX '06 Annual Technical Conference, p.11-11, May 30-June 03, 2006, Boston, MA
• 8: Ian Clarke , Oskar Sandberg , Brandon Wiley , Theodore W. Hong, Freenet: a distributed anonymous information storage and retrieval system, International workshop on Designing privacy enhancing technologies: design issues in anonymity and unobservability, p.46-66, January 2001, Berkeley, California, United States
• 9: Frank Dabek , M. Frans Kaashoek , David Karger , Robert Morris , Ion Stoica, Wide-area cooperative storage with CFS, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 10: John R. Douceur , William J. Bolosky, A large-scale study of file-system contents, Proceedings of the 1999 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.59-70, May 01-04, 1999, Atlanta, Georgia, United States
• 11: Michael J. Freedman , Eric Freudenthal , David Mazières, Democratizing content publication with coral, Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation, p.18-18, March 29-31, 2004, San Francisco, California
• 12: Freenet. 2007. Freenet FAQ. http://freenetproject.org/faq.html.
• 13: P. Brighten Godfrey , Scott Shenker , Ion Stoica, Minimizing churn in distributed systems, Proceedings of the 2006 conference on Applications, technologies, architectures, and protocols for computer communications, September 11-15, 2006, Pisa, Italy
• 14: Guha, S., Daswani, N., and Jain, R. 2006. An experimental study of the Skype peer-to-peer VoIP system. In Proceedings of the 5th International Workshop on Peer-to-Peer Systems (IPTPS), Santa Barbara, CA.
• 15: John H. Howard , Michael L. Kazar , Sherri G. Menees , David A. Nichols , M. Satyanarayanan , Robert N. Sidebotham , Michael J. West, Scale and performance in a distributed file system, ACM Transactions on Computer Systems (TOCS), v.6 n.1, p.51-81, Feb. 1988 [doi>10.1145/35037.35059]
• 16: Hai Huang , Wanda Hung , Kang G. Shin, FS2: dynamic data replication in free disk space for improving disk performance and energy consumption, Proceedings of the twentieth ACM symposium on Operating systems principles, October 23-26, 2005, Brighton, United Kingdom
• 17: Larson, S. M., Snow, C. D., Shirts, M., and Pande, V. S. 2002. Computational Genomics. Horizon. Folding@Home and Genome@Home: Using distributed computing to tackle previously intractable problems in computational biology.
• 18: Leonard, O. C., Neigh, J., Zadok, E., Osborn, J., Shater, A., and Wright, C. 2002. The design and implementation of elastic quotas. Tech. Rep. CUCS-014-02, Columbia University. June.
• 19: Christopher R. Lumb , Jiri Schindler , Gregory R. Ganger, Freeblock Scheduling Outside of Disk Firmware, Proceedings of the Conference on File and Storage Technologies, p.275-288, January 28-30, 2002
• 20: Mahajan, R., Castro, M., and Rowstron, A. 2003. Controlling the cost of reliability in peer-to-peer overlays. In Proceedings of the International Workshop on Peer-to-Peer Systems (IPTPS).
• 21: Marshall K. McKusick , William N. Joy , Samuel J. Leffler , Robert S. Fabry, A fast file system for UNIX, ACM Transactions on Computer Systems (TOCS), v.2 n.3, p.181-197, Aug. 1984 [doi>10.1145/989.990]
• 22: James W. Mickens , Brian D. Noble, Exploiting availability prediction in distributed systems, Proceedings of the 3rd conference on Networked Systems Design & Implementation, p.6-6, May 08-10, 2006, San Jose, CA
• 23: Microsoft Corporation. 2007. http://www.microsoft.com/technet/prodtechnol/winxppro/reskit/c28621675.mspx.
• 24: Ion Stoica , Robert Morris , David Karger , M. Frans Kaashoek , Hari Balakrishnan, Chord: A scalable peer-to-peer lookup service for internet applications, ACM SIGCOMM Computer Communication Review, v.31 n.4, p.149-160, October 2001
• 25: Ousterhout, J. K. 1990. Why aren't operating systems getting faster as fast as hardware? In Proceedings of the USENIX Summer Conference, 247--256.
• 26: Zachary Peterson , Randal Burns, Ext3cow: a time-shifting file system for regulatory compliance, ACM Transactions on Storage (TOS), v.1 n.2, p.190-212, May 2005 [doi>10.1145/1063786.1063789]
• 27: Rhea, S., Geels, D., Roscoe, T., and Kubiatowicz, J. 2003. Handling churn in a DHT. Tech. Rep. UCB/CSD-03-1299, EECS Department, University of California, Berkeley.
• 28: Rodrigues, R. and Liskov, B. 2005. High availability in DHTs: Erasure coding vs. replication. In Proceedings of the 4th International Workshop on Peer-to-Peer Systems (IPTPS).
• 29: Rowstron, A. and Druschel, P. 2001a. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Proceedings of the Middleware Conference.
• 30: Antony Rowstron , Peter Druschel, Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 31: Margo Seltzer , Keith Bostic , Marshall Kirk Mckusick , Carl Staelin, An implementation of a log-structured file system for UNIX, Proceedings of the USENIX Winter 1993 Conference Proceedings on USENIX Winter 1993 Conference Proceedings, p.3-3, January 25-29, 1993, San Diego, California
• 32: Keith A. Smith , Margo I. Seltzer, File system aging—increasing the relevance of file system benchmarks, Proceedings of the 1997 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.203-213, June 15-18, 1997, Seattle, Washington, United States
• 33: Stutzbach, D. and Rejaie, R. 2004. Towards a better understanding of churn in peer-to-peer networks. Tech. Rep. UO-CIS-TR-04-06, Department of Computer Science, University of Oregon. November.
• 34: Arun Venkataramani , Ravi Kokku , Mike Dahlin, TCP Nice: a mechanism for background transfers, ACM SIGOPS Operating Systems Review, v.36 n.SI, Winter 2002 [doi>10.1145/844128.844159]

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.3 File Systems Management
          Subjects: File organization

General Terms:
Performance

Keywords:
Contributory systems, aging, fragmentation, peer-to-peer