Effects of virtualization on a scientific application running a hyperspectral radiative transfer code on virtual machines

ABSTRACT

The topic of system-level virtualization has recently begun to receive interest for high performance computing (HPC). This is in part due to the isolation and encapsulation offered by the virtual machine. These traits enable applications to customize their environments and maintain consistent software configurations in their virtual domains. Additionally, there are mechanisms that can be used for fault tolerance like live virtual machine migration. Given these attractive benefits to virtualization, a fundamental question arises, how does this effect my scientific application? We use this as the premise for our paper and observe a real-world scientific code running on a Xen virtual machine. We studied the effects of running a radiative transfer simulation, Hydrolight, on a virtual machine. We discuss our methodology and report observations regarding the usage of virtualization with this application.

REFERENCES

• 1: Paul Barham , Boris Dragovic , Keir Fraser , Steven Hand , Tim Harris , Alex Ho , Rolf Neugebauer , Ian Pratt , Andrew Warfield, Xen and the art of virtualization, Proceedings of the nineteenth ACM symposium on Operating systems principles, October 19-22, 2003, Bolton Landing, NY, USA
• 2: W. Emeneker and D Stanzione. HPC Cluster Readiness of Xen and UserMode Linux. In IEEE International Conference on Cluster Computing, September 2006.
• 3: Christian Engelmann, Stephen L. Scott, Hong Ong, Geoffroy Vallée, and Thomas Naughton. Configurable Virtualized System Environments for High Performance Computing. In Proceedings of the 1^st Workshop on System-level Virtualization for High Performance Computing (HPCVirt'07), held in conjunction with the ACM EuroSys'07, Lisbon, Portugal, March 20, 2007.
• 4: Anthony M. Filippi, Budhendra L. Bhaduri, Thomas Naughton, Amy L. King, and Stephen L. Scott. High-performance cluster computing-based approach to hyperspectral aquatic radiative transfer modeling. (In preparation) Photogrammetric Engineering & Remote Sensing (PE&RS), 2008.
• 5: Wei Huang , Jiuxing Liu , Bulent Abali , Dhabaleswar K. Panda, A case for high performance computing with virtual machines, Proceedings of the 20th annual international conference on Supercomputing, June 28-July 01, 2006, Cairns, Queensland, Australia [doi>10.1145/1183401.1183421]
• 6: Wei Huang, Qi Gao, Jiuxing Liu, and Dhabaleswar K. Panda. High Performance Virtual Machine Migration with RDMA Over Modern Interconnects. In Proceedings of IEEE Cluster (Cluster'07). IEEE, September 17--20, 2007.
• 7: Aravind Menon , Jose Renato Santos , Yoshio Turner , G. (John) Janakiraman , Willy Zwaenepoel, Diagnosing performance overheads in the xen virtual machine environment, Proceedings of the 1st ACM/USENIX international conference on Virtual execution environments, June 11-12, 2005, Chicago, IL, USA [doi>10.1145/1064979.1064984]
• 8: Mark F. Mergen , Volkmar Uhlig , Orran Krieger , Jimi Xenidis, Virtualization for high-performance computing, ACM SIGOPS Operating Systems Review, v.40 n.2, April 2006 [doi>10.1145/1131322.1131328]
• 9: Curtis D. Mobley. Light and Water: Radiative Transfer in Natural Waters. Academic Press, San Diego, 1994.
• 10: Curtis D. Mobley, Bernard Gentili, Howard R. Gordon, Zhonghai Jin, George W. Kattawar, Andre Morel, Phillip Reinersman, Knut Stamnes, and Robert H. Stavn. Comparison of numerical models for computing underwater light fields. Applied Optics, 32(36):7484, 1993.
• 11: Curtis D. Mobley, Lydia K. Sundman, Curtiss O. Davis, Jeffrey H. Bowles, Trijntje Valerie Downes, Robert A. Leathers, Marcos J. Montes, William Paul Bissett, David D. R. Kohler, Ruth Pamela Reid, Eric M. Louchard, and Arthur Gleason. Interpretation of hyperspectral remote-sensing imagery by spectrum matching and look-up tables. Applied Optics, 44(17):3576--3592, 2005.
• 12: Hussam Mousa, Chandra Krintz, Lamia Youseff, and Rich Wolski. VIProf: Vertically integrated full-system performance profiler. In Proceedings of the Workshop on Next-Generation Software (NGS), March 2007. Co-hosted with IPDPS 2007.
• 13: Arun Babu Nagarajan , Frank Mueller , Christian Engelmann , Stephen L. Scott, Proactive fault tolerance for HPC with Xen virtualization, Proceedings of the 21st annual international conference on Supercomputing, June 17-21, 2007, Seattle, Washington [doi>10.1145/1274971.1274978]
• 14: OProfile: A system-wide profiler for Linux. Available at: http://oprofile.sourceforge.net. (Last visited: Feb. 2008).
• 15: Sequoia Scientific, Inc., Redmond, WA, USA. Hydrolight 4.2, 2000.
• 16: Geoffroy Vallee , Thomas Naughton , Stephen L. Scott, System management software for virtual environments, Proceedings of the 4th international conference on Computing frontiers, May 07-09, 2007, Ischia, Italy [doi>10.1145/1242531.1242555]
• 17: Lamia Youseff, Rich Wolski, Brent Gorda, and Chandra Krintz. Paravirtualization for HPC Systems. In ISPA Workshop on XEN in HPC Cluster and Grid Computing Environments (XHPC'06), pages 474--486, December 2006.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.8 Performance

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.1 Process Management
      D.4.2 Storage Management
      D.4.4 Communications Management
          Subjects: Input/output

General Terms:
Design, Experimentation, Measurement, Performance