Efficient scheduling of scientific workflows in a high performance computing cluster

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Efficient scheduling of scientific workflows in a high performance computing cluster

Full text

Pdf (386 KB)

Source

International Workshop on Challenges of Large Applications in Distributed Environments archive
Proceedings of the 6th international workshop on Challenges of large applications in distributed environments table of contents

Boston, MA, USA

SESSION: HPC table of contents

Pages 63-68

Year of Publication: 2008

ISBN:978-1-60558-156-9

Authors

Roger S. Barga	Microsoft Research, Redmond, WA, USA
Dan Fay	Microsoft Research, Redmond, WA, USA
Dean Guo	Microsoft Research, Redmond, WA, USA
Steven Newhouse	Microsoft Corporation, Redmond, WA, USA
Yogesh Simmhan	Microsoft Research, Redmond, WA, USA
Alex Szalay	The Johns Hopkins University, Baltimore, MD, USA

Sponsors

ACM: Association for Computing Machinery
SIGARCH: ACM Special Interest Group on Computer Architecture

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 22, Downloads (12 Months): 188, Citation Count: 1

Additional Information:

abstract references cited by index terms collaborative colleagues

Tools and Actions:	Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/1383529.1383545 What is a DOI?

ABSTRACT

The scientific computing community, especially academia is clearly in need of technology to handle and organize the 1-100⁺ Terabyte datasets coming from computer simulations and scientific instrumentation. In this paper we briefly describe GrayWulf, an exemplar cluster for data intensive applications using SQL Server and HPC Clusters. One of the key software components of GrayWulf is Trident, a scientific workflow workbench that performs automatic scheduling of workflows across the cluster. We examine the challenges of scheduling workflows on GrayWulf, algorithms to improve performance, and present early results from applying Trident to schedule data loading workflows on GrayWulf for an actual e-Science project

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Project Neptune http://www.neptune.washington.edu/.
	2	Swiss Experiment, http://www.swiss-experiment.ch.
	3	Life Under Your Feet http://lifeunderyourfeet.org.
	4	Gordon Bell , Jim Gray , Alex Szalay, Petascale Computational Systems, Computer, v.39 n.1, p.110-112, January 2006 [doi>10.1109/MC.2006.29]
	5	Microsoft Windows Workflow Foundation (WinWF) http://en.wikipedia.org/wiki/Windows_Workflow_Foundation.
	6	Technical Computing Group of Microsoft Research http://www.microsoft.com/science.
	7	Microsoft Silverlight http://silverlight.net/.
	8	Pan-STARRS http://pan-starrs.ifa.hawaii.edu/public/.