Redundant parallel data transfer schemes for the grid environment

ABSTRACT

In this paper, we proposed dynamic co-allocation schemes for parallel data transfer in a grid environment, which copes up with highly inconsistent network performances of the servers. We have developed an algorithm using circular queue, with which, the data transfer tasks are allocated onto the servers in duplication. It is further improved to maintain the ordered delivery of data blocks, to suit such applications, like streaming. The schemes nicely tackle even when the link to servers under consideration is broken or become idle. We used Globus toolkit for our framework and utilized the partial copy feature of GridFTP. We compared our schemes with the existing schemes and the preliminary results show notable improvement in overall completion time of data transfer.

REFERENCES

• 1: Bill Allcock , Joe Bester , John Bresnahan , Ann L. Chervenak , Ian Foster , Carl Kesselman , Sam Meder , Veronika Nefedova , Darcy Quesnel , Steven Tuecke, Data management and transfer in high-performance computational grid environments, Parallel Computing, v.28 n.5, p.749-771, May 2002 [doi>10.1016/S0167-8191(02)00094-7]
• 2: Bit Torrent home page, accessed Nov 2005 http://www.bittorent.com/introduction.html
• 3: Chao-Tung Yang, I Hsien Yang, Chun Hsiang Chen (2005), Improve Dynamic Adjustment Mechanism in Co-allocation data Grid Environments, Workshop on Compiler Techniques for High-Performance Computing (CTHPC 05), Taiwan, pp. 189-194.
• 4: Ann Chervenak , Ewa Deelman , Ian Foster , Leanne Guy , Wolfgang Hoschek , Adriana Iamnitchi , Carl Kesselman , Peter Kunszt , Matei Ripeanu , Bob Schwartzkopf , Heinz Stockinger , Kurt Stockinger , Brian Tierney, Giggle: a framework for constructing scalable replica location services, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-17, November 16, 2002, Baltimore, Maryland
• 5: Chervenak A, Foster I, et al (2001), The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets, Journal of Network and Computer Applications, 23:187-200.
• 6: Chun Hsiang Chen, Chao-Tung Yang, Chuan-Lin Lai (2004), Towards an Efficient Replica selection for Data Grid, The First Workshop on Grid Technologies and Application.
• 7: Karl Czajkowski , Ian Foster , Carl Kesselman, Resource Co-Allocation in Computational Grids, Proceedings of the The Eighth IEEE International Symposium on High Performance Distributed Computing, p.37, August 03-06, 1999
• 8: Data Grid Project (EU Data Grid), Accessed Nov 2005 http://eu-datagrid.web.cern.ch/eu-datagrid/.
• 9: Ghisellir A(2002), Datagrid Prototype 1, TERENA Networking Conference.
• 10: GridPhyN project (Grid Physics Network), http://www.griphyn.org, accessed Nov 2005. Introduction to Grids and the Globus Toolkit, Globus Project, http://www.globus.org accessed Nov 2005.
• 11: Malon D, et al(2001), Grid enabled Data Access in the ATLAS Athena Framework, Computing and HighEnergy Physics Conference.
• 12: Noriyuki Fujimoto, Kenichi Hagihara(2003), Near Optimal Dynamic Task Scheduling of Independent Coarse Grained Tasks onto a Computational Grid, International Conference on Parallel Processing (ICPP-03), pp. 391-398, IEEE Press, Taiwan, Oct 6-9.
• 13: Sudharshan Vazhkudai, Enabling the Co-Allocation of Grid Data Transfers, Proceedings of the Fourth International Workshop on Grid Computing, p.44, November 17-17, 2003
• 14: Sudharshan Vazhkudai , Steven Tuecke , Ian Foster, Replica Selection in the Globus Data Grid, Proceedings of the 1st International Symposium on Cluster Computing and the Grid, p.106, May 15-18, 2001
• 15: Xia Qin and Hong Jiang (2003), Data Grids: Supporting Data-Intensive Applications in Wide Area Networks, Dept of CSE, University of Nebraska Lincoln, NE, No. TR 03-05-01.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design
          Subjects: Distributed systems

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.4 Communications Management

  F. Theory of Computation
  F.1 COMPUTATION BY ABSTRACT DEVICES
      F.1.2 Modes of Computation
          Subjects: Parallelism and concurrency

  G. Mathematics of Computing
  G.1 NUMERICAL ANALYSIS
      G.1.0 General
          Subjects: Parallel algorithms

General Terms:
Algorithms, Design, Management

Keywords:
coallocation, datagrid, parallel data transfer