Scientific computing using virtual high-performance computing: a case study using the Amazon elastic computing cloud

ABSTRACT

High-performance computing systems are important in scientific computing. Clusters of computer systems --- which range greatly in size --- are a common architecture for high-performance computing. Small, dedicated clusters are affordable and cost-effective, but may not be powerful enough for real applications. Larger dedicated systems are expensive in absolute terms and may be inefficient because many individual groups may not be able to provide sustained workload for the cluster. Shared systems are cost-effective, but then availability and access become a problem.

An alternative model is that of a virtual cluste, as exemplified by Amazon's Elastic Computing Cloud (EC2). This provides customers with storage and CPU power on an ondemand basis, and allows a researcher to dynamically build their own, dedicated cluster of computers when they need it. Used by commercial web services deployers, this technology can be used in scientific computing applications. This paper presents a case study of the use of EC2 for scientific computing. The case study concludes that EC2 provides a feasible, cost-effective model in many application areas.

REFERENCES

• 1: Amazon. Amazon Elastic Compute Cloud developer guide. http://s3.amazonaws.com/awsdocs/EC2/2008-02-01/ec2-dg-2008-02-01.pdf, Feb. 2008. API version 2008-02-01.
• 2: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 3: M. Boguski, T. Lowe, and C. Tolstoshev. dbEST--database for expressed sequence tags. Nature Genetics, 4(4):332--3, Aug. 1993.
• 4: J. Y. Choi, Y. Yang, S. Kim, and D. Gannon. V-lab-protein: Virtual collaborative lab for protein sequence analysis. In Proceedings of the IEEE Workshop on High-Throughput Data Analysis for Proteomics and Genomics, Nov. 2007.
• 5: S. Garfinkel. An evaluation of Amazon's grid computing services: EC2, S3 and SQS. Technical report, Harvard University, 2008. Technical Report TR-08-07.
• 6: S. Hazelhurst. Algorithms for clustering EST sequences: the wcd tool. South African Computer Journal, 40:51--62, June 2008.
• 7: Scott Hazelhurst , Winston Hide , Zsuzsanna Lipták , Ramon Nogueira , Richard Starfield, An overview of the wcd EST clustering tool, Bioinformatics, v.24 n.13, p.1542-1546, July 2008 [doi>10.1093/bioinformatics/btn203]
• 8: S. Nagaraj, R. Gasser, and S. Ranganathan. A hitchhiker's guide to expressed sequence tag (EST) analysis. Briefings in Bioinformatics, 8(1):6--21, 2007.
• 9: Mayur R. Palankar , Adriana Iamnitchi , Matei Ripeanu , Simson Garfinkel, Amazon S3 for science grids: a viable solution?, Proceedings of the 2008 international workshop on Data-aware distributed computing, p.55-64, June 24-24, 2008, Boston, MA, USA [doi>10.1145/1383519.1383526]
• 10: Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill Education Group, 2003
• 11: Aaron Weiss, Computing in the clouds, netWorker, v.11 n.4, p.16-25, December 2007 [doi>10.1145/1327512.1327513]

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.0 General

  J. Computer Applications
  J.0 GENERAL

General Terms:
Algorithms, Measurement, Performance

Keywords:
Amazon elastic computing cloud, clusters, high-performance computing, virtualisation