Efficient maintenance of distributed data in highly dynamic opportunistic grids

ABSTRACT

Opportunistic grids are a class of computational grids that can leverage the idle processing and storage capacity of shared workstations in laboratories, companies, and universities to perform useful computation. OppStore is a middleware that allows using the free disk space of machines from an opportunistic grid for the distributed storage of application data.

But when machines depart from the grid, it is necessary to reconstruct the fragments that were stored in that machines. Depending on the amount of stored data and the rate of machine departures, the generated traffic may make the distributed storage of data infeasible. In this work we present and evaluate a fragment recovery mechanism that makes viable to achieve redundancy and large data scale in a dynamic environment.

REFERENCES

• 1: C. Batten, K. Barr, A. Saraf, and S. Trepetin. pStore: A secure peer-to-peer backup system. Technical Report MIT-LCS-TM-632, MIT LCS, October 2002.
• 2: Charles Blake , Rodrigo Rodrigues, High availability, scalable storage, dynamic peer networks: pick two, Proceedings of the 9th conference on Hot Topics in Operating Systems, p.1-1, May 18-21, 2003, Lihue, Hawaii
• 3: Frank Dabek , M. Frans Kaashoek , David Karger , Robert Morris , Ion Stoica, Wide-area cooperative storage with CFS, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 4: Raphael Y. De Camargo , Fabio Kon, Design and Implementation of a Middleware for Data Storage in Opportunistic Grids, Proceedings of the Seventh IEEE International Symposium on Cluster Computing and the Grid, p.23-30, May 14-17, 2007 [doi>10.1109/CCGRID.2007.37]
• 5: I. Foster. Globus toolkit version 4: Software for service-oriented systems. In NPC 2005: IFIP International Conf. on Network and Parallel Computing, pages 2--13, Beijing, China, 2005.
• 6: James Frey , Todd Tannenbaum , Miron Livny , Ian Foster , Steven Tuecke, Condor-G: A Computation Management Agent for Multi-Institutional Grids, Cluster Computing, v.5 n.3, p.237-246, July 2002 [doi>10.1023/A:1015617019423]
• 7: Andrei Goldchleger , Fabio Kon , Alfredo Goldman , Marcelo Finger , Germano Capistrano Bezerra, InteGrade object-oriented Grid middleware leveraging the idle computing power of desktop machines: Research Articles, Concurrency and Computation: Practice & Experience, v.16 n.5, p.449-459, April 2004 [doi>10.1002/cpe.v16:5]
• 8: John Kubiatowicz , David Bindel , Yan Chen , Steven Czerwinski , Patrick Eaton , Dennis Geels , Ramakrishna Gummadi , Sean Rhea , Hakim Weatherspoon , Chris Wells , Ben Zhao, OceanStore: an architecture for global-scale persistent storage, Proceedings of the ninth international conference on Architectural support for programming languages and operating systems, p.190-201, November 2000, Cambridge, Massachusetts, United States
• 9: Martin Landers , Han Zhang , Kian-Lee Tan, PeerStore: Better Performance by Relaxing in Peer-to-Peer Backup, Proceedings of the Fourth International Conference on Peer-to-Peer Computing, p.72-79, August 25-27, 2004 [doi>10.1109/P2P.2004.38]
• 10: Qutaibah M. Malluhi , William E. Johnston, Coding for High Availability of a Distributed-Parallel Storage System, IEEE Transactions on Parallel and Distributed Systems, v.9 n.12, p.1237-1252, December 1998 [doi>10.1109/71.737699]
• 11: Michael O. Rabin, Efficient dispersal of information for security, load balancing, and fault tolerance, Journal of the ACM (JACM), v.36 n.2, p.335-348, April 1989 [doi>10.1145/62044.62050]
• 12: Matei Ripeanu , Ian Foster, A Decentralized, Adaptive Replica Location Mechanism, Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing, p.24, July 24-26, 2002
• 13: Antony Rowstron , Peter Druschel, Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility, Proceedings of the eighteenth ACM symposium on Operating systems principles, October 21-24, 2001, Banff, Alberta, Canada
• 14: Antony I. T. Rowstron , Peter Druschel, Pastry: Scalable, Decentralized Object Location, and Routing for Large-Scale Peer-to-Peer Systems, Proceedings of the IFIP/ACM International Conference on Distributed Systems Platforms Heidelberg, p.329-350, November 12-16, 2001
• 15: Hakim Weatherspoon , John Kubiatowicz, Erasure Coding Vs. Replication: A Quantitative Comparison, Revised Papers from the First International Workshop on Peer-to-Peer Systems, p.328-338, March 07-08, 2002

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS

General Terms:
Design, Performance, Reliability

Keywords:
data redundancy, distributed storage, grid computing