Towards dynamic data grid framework for eResearch

ABSTRACT

The scale at which scientific data is produced will undergo a massive change in the near future. Many sophisticated scientific discovery laboratories or the installation of sensor networks would produce a large amount of data. Research in protein crystallography for instance can produce hundreds of Terabytes of data from a single crystallography beamline. These data have to be saved for future use and made available for collaborative use by researchers. There is a need to develop a framework which can deal with storing such data volumes. This framework should also handle disparate data sources tightly integrated with the users' applications and large data streams arising from instruments and sensors. This paper presents an initial study into the framework for servicing large dynamic data sets over a national grid for eResearch.

REFERENCES

• 1: [1] Australia's Academic and Research Network. Available online: http://www.aarnet.edu.au/engineering/aarnet3/
• 2: [2] Allcock, W., Bester, J., Bresnahan, J., Chervenak, A., Liming, L., Meder, S., Tuecke, S., "GridFTP protocol specification", Technical report, Global Grid Forum GridFTP Working Group, September 2002.
• 3: [3] Atkinson, M., "Grid & Web services: Web Service Resource Framework (WSRF), Presentation at Core e-Science Programme Town Meeting to Discuss Grid & Web Services, Friday, 23rd April 2004, Jolly St Ermins Hotel, London.
• 4: Jim Almond , Dave Snelling, UNICORE: uniform access to supercomputing as an element of electronic commerce, Future Generation Computer Systems, v.15 n.5-6, p.539-548, Oct. 1999 [doi>10.1016/S0167-739X(99)00007-2]
• 5: [5] Australian Synchrotron. Available online: http://www.synchrotron.vic.gov.au/
• 6: [6] Bramley, R., Chiu, K., Huffman, J.C., Huffman, K. and McMullen, D.F., "Instruments and Sensors as Network Services: Making Instruments First Class Members of the Grid, Indiana University CS Department Technical Report 588, 2003.
• 7: [7] The Biomedical Informatics Research Network. Available online: http://www.nbirn.net
• 8: [8] Buyya, R., "The Virtual Laboratory Project: Molecular Modeling for Drug Design on Grid", IEEE Distributed Systems Online, Vol. 2, No. 5, 2001.
• 9: [9] Condor. Available online: http://www.cs.wisc.edu/condor/
• 10: [10] European DataGrid, Available online: http://eu-datagrid.web.cern.ch/eu-datagrid/
• 11: Ian Foster , Carl Kesselman, The grid: blueprint for a new computing infrastructure, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 12: Ian Foster , Carl Kesselman, The grid: blueprint for a new computing infrastructure, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 13: [13] Freier, A.O. Karlton, P. Kocher, P.C. "The SSL Protocol Version 3.0 Transport Layer Security Working Group INTERNET-DRAFT", 18 November 1996. Available online: http://wp.netscape.com/eng/ssl3/
• 14: Ian Foster , Carl Kesselman , Gene Tsudik , Steven Tuecke, A security architecture for computational grids, Proceedings of the 5th ACM conference on Computer and communications security, p.83-92, November 02-05, 1998, San Francisco, California, United States [doi>10.1145/288090.288111]
• 15: [15] Globus Project (The Globus Alliance). Available online: http://www.globus.org
• 16: [16] GRID computing and BUSiness technologies (GRIDBUS). Available online: http://www.gridbus.org/
• 17: Andrew S. Grimshaw , Wm. A. Wulf , CORPORATE The Legion Team, The Legion vision of a worldwide virtual computer, Communications of the ACM, v.40 n.1, p.39-45, Jan. 1997 [doi>10.1145/242857.242867]
• 18: [18] Grid and Next Generation Network. Available online: http://www.grangenet.net/
• 19: Tony Hey , Anne E. Trefethen, The UK e-science core programme and the grid, Future Generation Computer Systems, v.18 n.8, p.1017-1031, October 2002 [doi>10.1016/S0167-739X(02)00082-1]
• 20: R. Housley , W. Ford , W. Polk , D. Solo, Internet X.509 Public Key Infrastructure Certificate and CRL Profile, RFC Editor, 1999
• 21: [21] Iperf. Available online: http://dast.nlanr.net/Project/Iperf
• 22: [22] The Joint Information Systems Committee, 2004. "The Data Deluge: Preparing for the explosion in data", Available online: http://www.jisc.ac.uk/index.cfm?name=pub_datadeluge
• 23: William E. Johnston, Computational and data Grids in large-scale science and engineering, Future Generation Computer Systems, v.18 n.8, p.1085-1100, October 2002 [doi>10.1016/S0167-739X(02)00087-0]
• 24: [24] Kerberos: The Network Authentication Protocol. Available online: http://web.mit.edu/kerberos/www/
• 25: [25] Load Sharing Facility, Platform Computing Inc. Available online: http://www.platform.com/products/LSF/
• 26: [26] The Long Term Ecological Research (LTER) Network. Avaiable online: http://www.lternet.edu/
• 27: [27] Networked File Systems (NFS), RFC 1094 (Version 2), RFC 1814 (Version 3 ), RFC 3010 (Version 4)
• 28: Jason Novotny , Steven Tuecke , Von Welch, An Online Credential Repository for the Grid: MyProxy, Proceedings of the 10th IEEE International Symposium on High Performance Distributed Computing (HPDC-10'01), p.104, August 07-09, 2001
• 29: [29] OpenAFS UserGuide. Avaiable online: http://www.openafs.org/pages/doc/UserGuide/auusg010.htm
• 30: [30] Portable Batch System, Altair Grid Technologies Online: http://www.openpbs.org/
• 31: [31] Particle Physics data Grid. Available online: http://www.ppdg.net
• 32: Arcot Rajasekar , Reagan Moore, Data and Metadata Collections for Scientific Applications, Proceedings of the 9th International Conference on High-Performance Computing and Networking, p.72-80, June 25-27, 2001
• 33: Arcot Rajasekar , Michael Wan , Reagan Moore, MySRB & SRB: Components of a Data Grid, Proceedings of the 11 th IEEE International Symposium on High Performance Distributed Computing HPDC-11 20002 (HPDC'02), p.301, July 24-26, 2002
• 34: [34] Security Assertion Markup Language Technical Overview. Available online: http://xml.coverpages.org/SAML- TechOverviewV20-Draft7874.pdf
• 35: [35] Shibboleth. Available online: http://shibboleth.internet2.edu/shib-tech-intro.html
• 36: [36] Sun N1 Grid Engine 6, Sun Microsystems Inc. Online: http://www.sun.com/software/gridware/index.xml
• 37: [37] Sun, H, et al, 1994, "Study on an algorithm of multisensor data fusion", Proc. IEEE Proceedings of the National Aerospace and Electronics Conference, Vol.1, pp.239-245.
• 38: [38] Tuecke, S. Welch, V. Engert, D. Pearlman, L. Thompson, M. "RFC3820 Internet X.509 Public Key Infrastructure (PKI) Proxy Certificate Profile", Jun 2004.
• 39: [39] Wald, L., "Some terms of reference in data fusion". IEEE Transactions on Geosciences and Remote Sensing, 37, 3, 1190- 1193, 1999.
• 40: [40] Welch, V. Foster, I. Kesselman, C. Mulmo, O. Pearlman, L. Tuecke, S. Gawor, J. Meder, S. Siebenlist, F. "X.509 Proxy Certificates for Dynamic Delegation". 3rd Annual PKI R&D Workshop, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Distributed systems

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services
          Subjects: Web-based services

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management

General Terms:
Design, Management

Keywords:
CIMA, SRB, data grid, eResearch, gridFTP