Generic support for bulk operations in grid applications

ABSTRACT

Within grid environments, latencies for remote operations of any kind can, as the number of operations increases, become a dominant factor for overall application performance. Amongst various approaches for latency hiding, bulk operations provide one possible solution to reduce latencies for large numbers of similar operations. The identification of bulks can, however, pose a non-trivial exercise for application developers, often requiring changes to the implemented remote API, and hence direct code modifications to the application themselves.In this paper we show how bulk operations can be integrated into existing API implementations, and identify the required properties of the API to make this approach feasible. We also show that our approach considers any type of bulk operation, and is independent of the underlying middleware support for bulks. We further describe a prototype implementation (within the SAGA C++ reference implementation effort), and present performance measurements for bulks of remote file copy operations.

REFERENCES

• 1: Jim Waldo , Geoff Wyant , Ann Wollrath , Samuel C. Kendall, A Note on Distributed Computing, Selected Presentations and Invited Papers Second International Workshop on Mobile Object Systems - Towards the Programmable Internet, p.49-64, July 08-09, 1996
• 2: R. M. J. Badia, J. J. Labarta, R. J. Sirvent, J. M. J. Pérez, J. M. J. Cela, and R. J. Grima. Programming Grid Applications with GRID Superscalar. Journal of Grid Computing, 1(2):151--170, 2003.
• 3: Gabrielle Allen, Kelly Davis, Tom Goodale, Andrei Hutanu, Hartmut Kaiser, Thilo Kielmann, Andre Merzky, Rob van Nieuwpoort, Alexander Reinefeld, Florian Schintke, Thorsten Schütt, Ed Seidel, and Brygg Ullmer. The Grid Application Toolkit: Towards Generic and Easy Application Programming Interfaces for the Grid. Proceedings of the IEEE, 2004.
• 4: H. Rajic, I. A. Inc, W. Chan, I.B.M.F. Ferstl, A. Haas, and J. Tollefsrud. Distributed Resource Management Application API Specification. Technical report, Global Grid Forum, September 2002. GFD.22.
• 5: Keith Seymour , Hidemoto Nakada , Satoshi Matsuoka , Jack Dongarra , Craig Lee , Henri Casanova, Overview of GridRPC: A Remote Procedure Call API for Grid Computing, Proceedings of the Third International Workshop on Grid Computing, p.274-278, November 18, 2002
• 6: Gregor von Laszewski, Ian Foster, Jarek Gawor, and Peter Lane. A Java commodity grid kit. Concurrency and Computation: Practice and Experience, 13(8-9):645--662, 2001.
• 7: Andre Merzky and Shantenu Jha. A Requirements Analysis for a Simple API for Grid Applications. Technical report, Global Grid Forum, 2006. GFD.71.
• 8: Tom Goodale, Shantenu Jha, Hartmut Kaiser, Thilo Kielmann, Pascal Kleijer, Gregor von Laszewski, Craig Lee, Andre Merzky, Hrabri Rajic, and John Shalf. SAGA: A Simple API for Grid Applications - High-Level Application Programming on the Grid. Computational Methods in Science and Technology: special issue "Grid Applications: New Challenges for Computational Methods", 8(2), SC05, November 2005.
• 9: Andre Merzky and Shantenu Jha. Simple API for Grid Applications - Use Case Document. Technical report, Global Grid Forum, March 2006. GFD.70.
• 10: Rüdiger Berlich , Marcel Kunze , Kilian Schwarz, Grid computing in Europe: from research to deployment, Proceedings of the 2005 Australasian workshop on Grid computing and e-research, p.21-27, January 01, 2005, Newcastle, New South Wales, Australia
• 11: William Allcock , John Bresnahan , Rajkumar Kettimuthu , Michael Link, The Globus Striped GridFTP Framework and Server, Proceedings of the 2005 ACM/IEEE conference on Supercomputing, p.54, November 12-18, 2005 [doi>10.1109/SC.2005.72]
• 12: EF Walker, R. Floyd, and P. Neves. Asynchronous remote operation execution in distributed systems. Distributed Computing Systems, 1990. Proceedings., 10th International Conference on, pages 253--259.
• 13: Andrei Hutanu , Stephan Hirmer , Gabrielle Allen , Andre Merzky, Analysis of remote execution models for grid middleware, Proceedings of the 4th international workshop on Middleware for grid computing, November 27-December 01, 2006, Melbourne, Australia [doi>10.1145/1186675.1186687]
• 14: T. Schutt , A. Merzky , A. Hutanu , F. Schintke, Remote partial file access using compact pattern descriptions, Proceedings of the 2004 IEEE International Symposium on Cluster Computing and the Grid, p.482-489, April 19-22, 2004

INDEX TERMS

Primary Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Distributed programming

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.6 Programming Environments
          Subjects: Integrated environments
  D.3 PROGRAMMING LANGUAGES
      D.3.3 Language Constructs and Features
          Subjects: Concurrent programming structures
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design
          Subjects: Distributed systems

General Terms:
Design, Management