Transparent grid enablement of weather research and forecasting

ABSTRACT

The impact of hurricanes is so devastating throughout different levels of society that there is a pressing need to provide a range of users with accurate and timely information that can enable effective planning for and response to potential hurricane landfalls. The Weather Research and Forecasting (WRF) code is the latest numerical model that has been adopted by meteorological services worldwide. The current version of WRF has not been designed to scale out of a single organization's local computing resources. However, the high resource requirements of WRF for fine-resolution and ensemble forecasting demand a large number of computing nodes, which typically cannot be found within one organization. Therefore, there is a pressing need for the Grid-enablement of the WRF code such that it can utilize resources available in partner organizations. In this paper, we present our research on Grid enablement of WRF by leveraging our work in transparent shaping, GRID superscalar, profiling, code inspection, code modeling, meta-scheduling, and job flow management.

REFERENCES

• 1: The Weather Research & Forecasting Model, http://www.wrfmodel.org/index.php.
• 2: Patrick T. Welsh and Peter Bogenschutz. Weather Research and Forecast (WRF) Model: Precipitation Prognostics from the WRF Model during Recent Tropical Cyclones, 59th Interdepartmental Hurricane Conference, March 7--11, 2005 Jacksonville, FL.
• 3: S. M. Sadjadi et al. Improving separation of concerns in the development of scientific applications. In Proceedings of The Nineteenth International Conference on Software Engineering and Knowledge Engineering, pages 456--461, Boston, USA, 2007.
• 4: S. Masoud Sadjadi , Philip K. McKinley , Betty H. C. Cheng, Transparent shaping of existing software to support pervasive and autonomic computing, Proceedings of the 2005 workshop on Design and evolution of autonomic application software, May 21-21, 2005, St. Louis, Missouri
• 5: R. Badia, R. Sirvent, J. Labarta, and J. Perez. Programming the GRID: An Imperative Language Based Approach. Book chapter in Engineering the Grid, Section 4, Chapter 12, Jan. 2006.
• 6: S. Shimizu, R. Rangaswami, and H. Duran-Limon. "Platform-independent Modeling and Prediction of Application Resource Usage Characteristics", Florida International University Technical Report FIU-SCIS-TR-2007-07-05, July 2007.
• 7: L. Fong et al. The LA Grid meta-scheduling project. In the poster presentation session of the 5th Latin American Grid (LA Grid) Summit, The IBM T. J. Watson Research Center, NY, U.S.A., September 2007.
• 8: G. Dasgupta et al. Fault-tolerant job-flow management in grid environment. In the poster presentation session of the 5th Latin American Grid (LA Grid) Summit, The IBM T. J. Watson Research Center, NY, U.S.A., September 2007.
• 9: S. M. Sadjadi et al. The Latin American Grid (LA Grid) Weather Research and Forecast (WRF) Portal. In the poster presentation session of the 5th Latin American Grid Summit, The IBM T. J. Watson Research Center, NY, U.S.A., September 2007.
• 10: Chen, S. S., J. F. Price, W. Zhao, M. A. Donelan, and E. J. Walsh, 2007: The CBLAST-Hurricane Program and the next-generation fully coupled atmosphere-wave-ocean models for hurricane research and prediction. Bull. Amer. Meteor. Soc, 88, 311--317.
• 11: Houze, R. A., S. S. Chen, B. Smull, W.-C. Lee, M. Bell, 2007: Hurricane intensity and eyewall replacement. Science, 315, 1235--1239.
• 12: Davis, C., W. Wang, S. S. Chen, Y. Chen, K. Corbosiero, M. DeMaria, J. Dudhia, G. Holland, J. Klemp, J. Michalakes, H. Reeves, R. Rotunno1, and Q. Xiao, 2007: Prediction of landfalling hurricanes with the Advanced Hurricane WRF Model, Mon. Wea. Rev., in press.
• 13: T. N. Krishnamurti et al. Improved Weather and Seasonal Climate Forecasts from Multimodel Superensemble, Science 3, Vol. 285. no. 5433, 1548--1550 pp, 1999.
• 14: V. Kumar et al. Multimodel Superensemble Forecasting of Tropical Cyclones in the Pacific, Mon Wea Rev., 131, 574--583, 2003.
• 15: W. C. Eric et al. Real-Time Multimodel Superensemble Forecasts of Atlantic Tropical Systems of 1999, Mon Wea Rev, 131, 1878--1894, 2003.
• 16: Revised Papers from the Second International Workshop on Engineering Distributed Objects, November 2000
• 17: The Globus Project (Globus Alliance)", http://www.globus.org
• 18: Legion Web Site. http://legion.virginia.edu
• 19: M. Litzkow, M. Livny, and M. Mutka. "Condor - A Hunter of Idle Workstations", Proc. of the 8th International Conference of Distributed Computing Systems (ICDCS 1988), January 1988, San Jose, CA, IEEE CS Press, USA, 1998.
• 20: A. Anjomshoaa, M. Drescher, et. al. "Job Submission Description Language (JSDL) Specification"; Version 1.0, 2005.
• 21: V. Gucer, J. Biggs-Finstad, et. al. "Getting Started with Tivoli Dynamic Workload Broker 1.1". www.redbooks.ibm.com, SG24-7442-00.
• 22: GridWay Web Site. http://www.gridway.org
• 23: I. Rodero, R. M. Badia, J. Corbalan, J. Labarta, "eNANOS Grid Resource Broker", European Grid Conference 2005, LNCS 3470, Amsterdam, Netherlands, 14--16 Feb., 2005, pp. 111-121.
• 24: Oneyka Ezenwoye , S. Masoud Sadjadi, Composing aggregate web services in BPEL, Proceedings of the 44th annual Southeast regional conference, March 10-12, 2006, Melbourne, Florida [doi>10.1145/1185448.1185549]
• 25: O. Ezenwoye and S. M. Sadjadi. TRAP/BPEL: A framework for dynamic adaptation of composite services. In Proceedings of the International Conference on Web Information Systems and Technologies (WEBIST 2007), Barcelona, Spain, March 2007.
• 26: S. M. Sadjadi, P. K. McKinley, B. Cheng, and R. Stirewalt. TRAP/J: Transparent generation of adaptable Java programs. In Proceedings of the International Symposium on Distributed Objects and Applications, Agia Napa, Cyprus, October 2004.
• 27: S. M. Sadjadi et al. Finding an appropriate profiler for the weather research and forecasting code. Technical Report FIU-SCIS-2007-09-03, School of Computing and Information Sciences, Florida International University, 11200 SW 8th St., Miami, FL 33199, August 2007.
• 28: Sameer S. Shende , Allen D. Malony, The Tau Parallel Performance System, International Journal of High Performance Computing Applications, v.20 n.2, p.287-311, May 2006 [doi>10.1177/1094342006064482]
• 29: IBM HPC Toolkit. https://domino.research.ibm.com/comm/research_projects.nsf/pages/actc.index.html.
• 30: Vincent Pillet, Jesus Labarta, Toni Cortes and Sergi Girona. "PARAVER: A Tool to Visualize and Analyze Parallel Code," CEPBA/UPC Report No. RR-95/03 February 1995.
• 31: O'Reilly, T. "What Is Web 2.0 - Design Patterns and Business Models for the Next Generation of Software," http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.html, 09/30/2005.
• 32: Yahoo UI, http://developer.yahoo.com/yui/
• 33: Google Maps, http://www.google.com/apis/maps/
• 34: PostgreSQL, http://www.postgresql.org/
• 35: Global Forecast System, http://en.wikipedia.org/wiki/Global_Forecast_System
• 36: VisAD, http://www.ssec.wisc.edu/~billh/visad.html
• 37: CPUlimit - CPU Usage Limiter for Linux: http://CPUlimit.sourceforge.net. Sept. 15 2007.
• 38: S. M. Sadjadi et al. Weather research and forecasting model 2.2 documentation: A step-by-step guide of a model run. Technical Report FIU-SCIS-2007-09-02, Florida International University, 11200 SW 8th St., Miami, FL 33199, August 2007.
• 39: R. Badia et al. HPC and Grids in Action, chapter Innovative Grid Technologies Applied to Bioinformatics and Hurricane Mitigation. IOS Press, Amsterdam, 2007.

INDEX TERMS

Primary Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design

Additional Classification:
  D. Software
  D.4 OPERATING SYSTEMS
      D.4.8 Performance

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management

General Terms:
Design, Experimentation, Measurement, Performance

Keywords:
WRF, grid enablement, job flow management, meta-scheduling, modeling, portal, profiling, scientific applications