A grid-enabled workflow system for reservoir uncertainty analysis

ABSTRACT

Reservoir uncertainty analysis is significant for petroleum engineers for predictions of reservoir performance. However, analysis of reservoir performance uncertainty is challenging because large amounts of data must be transferred efficiently, reliably and securely between sites, and thousands of simulations are executed across different resources. There are several steps in conducting reservoir performance prediction, including: (a) transferring input files to remote resources, (b) running thousands of simulations in different scheduler systems, (c) monitoring the jobs, (d) transferring output files from remote sites to the local system, and (e) post-processing to determine whether simulations have resolved uncertainties adequately. This whole process may have to be repeated as new data are obtained, or if uncertainty thresholds change. Therefore, it is essential to automate end-to-end processing for this complex, composite application with many tasks that are executed in a specific order. We implemented an end-to-end automated system for reservoir uncertainty analysis using Grid technologies such as Condor-G, DAGMan, and Stork. This paper describes the requirements, design and implementation of such a system.

REFERENCES

• 1: Ian Foster , Carl Kesselman , Steven Tuecke, The Anatomy of the Grid: Enabling Scalable Virtual Organizations, International Journal of High Performance Computing Applications, v.15 n.3, p.200-222, August 2001 [doi>10.1177/109434200101500302]
• 2: D. Hughes, P. Greenwood, G. Blair, G. Coulson, F. Pappenberger, P. Smith, and K. Beven. An intelligent and adaptable Grid-based flood monitoring and warning system. UK eScience All Hands Meeting, 2006.
• 3: Danny Hughes , Phil Greenwood , Geoff Coulson , Gordon Blair, GridStix: Supporting Flood Prediction using Embedded Hardware and Next Generation Grid Middleware, Proceedings of the 2006 International Symposium on on World of Wireless, Mobile and Multimedia Networks, p.621-626, June 26-29, 2006 [doi>10.1109/WOWMOM.2006.49]
• 4: Salman Alsairafi , Filippia-Sofia Emmanouil , Moustafa Ghanem , Nikolaos Giannadakis , Yike Guo , Dimitrios Kalaitzopoulos , Michelle Osmond , Anthony Rowe , Jameel Syed , Patrick Wendel, The Design of Discovery Net: Towards Open Grid Services for Knowledge Discovery, International Journal of High Performance Computing Applications, v.17 n.3, p.297-315, August 2003 [doi>10.1177/1094342003173003]
• 5: Chalermek Intanagonwiwat , Ramesh Govindan , Deborah Estrin, Directed diffusion: a scalable and robust communication paradigm for sensor networks, Proceedings of the 6th annual international conference on Mobile computing and networking, p.56-67, August 06-11, 2000, Boston, Massachusetts, United States [doi>10.1145/345910.345920]
• 6: Y. Yao and J. E. Gehrke.
• 7: T. Kosar, G. Kola, R. J. Brunner, M. Livny, and M. Remijan Reliable, Automatic Transfer and Processing of Large Scale Astronomy Datasets. In Proceedings of 14th Astronomical Data Analysis Software & Systems Conference (ADASS 2004), Pasadena, CA.
• 8: George Kola , Tevfik Kosar , Miron Livny, A fully automated fault-tolerant system for distributed video processing and off-site replication, Proceedings of the 14th international workshop on Network and operating systems support for digital audio and video, June 16-18, 2004, Cork, Ireland [doi>10.1145/1005847.1005874]
• 9: Samuel R. Madden , Michael J. Franklin , Joseph M. Hellerstein , Wei Hong, TinyDB: an acquisitional query processing system for sensor networks, ACM Transactions on Database Systems (TODS), v.30 n.1, p.122-173, March 2005 [doi>10.1145/1061318.1061322]
• 10: B. Ludascher, I. Altintas, C. Berkley, D. Higgins, E. Jaeger, M. Jones, E. Lee , J. Tao, Y. Zhao. Scientific workflow management and the Kepler system.
• 11: D. Hollinsworth The Workflow Reference Model, Workflow Management Coalition, TC00-1003, 1994.
• 12: D. P. Spooner , J. Cao , S. A. Jarvis , L. He , G. R. Nudd, Performance-Aware Workflow Management for Grid Computing, The Computer Journal, v.48 n.3, p.347-357, May 2005 [doi>10.1093/comjnl/bxh090]
• 13: The UCoMS Project Home Page. November, 2007, http://www.ucoms.org/
• 14: G. Allen, K. Davis. The Grid Application Toolkit:Towards Generic and Easy Application Programming Interfaces for the Grid. Proceedings of the IEEE, Vol. 93, No. 3, pp. 534--550, March 2005.
• 15: K. Aziz, A. Settari. "Petroleum Reservoir Simulation". Applied Science Publishers Ltd., London, 1979.
• 16: J. Yu, and R. Buyya. A Taxonomy of Workflow Management Systems for Grid Computing. Technical Report GRIDS-TR-2005-1, Grid Computing and Distributed Systems Laboratory, University of Melbourne, 2005.
• 17: Z. Lei, D. Huang, A. Kulshrestha, S. Pena, G. Allen, X. Li, R. Duff, S. Kalla, C. D. White, J. R. Smith. , Leveraging Grid Technologies For Reservoir Uncertainty Analysis. , in the proceeding of High Performance Computing Symposium (HPC 2006), April 3-6,2006, Huntsville, AL, 2006.
• 18: I. Taylor, M. Shields, and I. Wang. Resource Management of Triana P2P Services. Grid Resource Management, Kluwer, Netherlands, June 2003.
• 19: Tom Oinn , Matthew Addis , Justin Ferris , Darren Marvin , Martin Senger , Mark Greenwood , Tim Carver , Kevin Glover , Matthew R. Pocock , Anil Wipat , Peter Li, Taverna: a tool for the composition and enactment of bioinformatics workflows, Bioinformatics, v.20 n.17, p.3045-3054, November 2004 [doi>10.1093/bioinformatics/bth361]
• 20: E. Deelman, J. Blythe, Y. Gil, C. Kesselman, G. Mehta, S. Patil, M. H. Su, K. Vahi, M. Livny. Pegasus: Mapping Scientific Workflow onto the Grid. Across Grids Conference 2004, Nicosia, Cyprus, 2004.
• 21: R. D. Stevens, A. J. Robinson, and C. A. Goble. myGrid:Personalized Bioinformatics on the Information Grid. Bioinformatics, 19(Suppl. 1):i302-i304, Oxford University Press, London, UK, 2003.
• 22: I. Altintas, A. Birnbaum, K. Baldridge, W. Sudholt, M. Miller, C. Amoreira, Y. Potier, and B. Ludaescher. A Framework for the Design and Reuse of Grid Workflows. International Workshop on Scientific Applications on Grid Computing (SAG?04), LNCS 3458, Springer, 2005.
• 23: X. Liu, J. Liu, J. Eker, and E. A. Lee. Heterogeneous Modeling and Design of Control Systems, Software-Enabled Control: Information Technology for Dynamical Systems, Tariq Samad and Gary Balas (eds.), Wiley- IEEE Press, April 2003.
• 24: M. Litzkow, M. Livny, and M. Mutka. Condor - A Hunter of Idle Workstations. In 8th International Conference of Distributed Computing Systems (ICDCS), IEEE CS Press, Los Alamitos, CA, USA, June 1988; 104--111.
• 25: Todd Tannenbaum , Derek Wright , Karen Miller , Miron Livny, Condor: a distributed job scheduler, Beowulf cluster computing with Linux, MIT Press, Cambridge, MA, 2001
• 26: J. Frey,I. Foster, M. Livny, T. Tannenbaum, and S. Tuecke, S. Condor-G: A Computation Management Agent for Multi-Institutional Grids, University of Wisconsin Madison, 2001.
• 27: Mary Papakhian, Comparing Job-Management Systems: The User's Perspective, IEEE Computational Science & Engineering, v.5 n.2, p.4-9, April 1998 [doi>10.1109/99.683732]
• 28: Loadleveler accessed September 2007. {Online}. Available: http://www.mhpcc.edu/training/workshop/loadleveler/.
• 29: T. Kosar and M.Livny Stork: Making Data Placement a First Class Citizen in the Grid. To appear in Journal of Parallel and Distributed Computing, (2005)
• 30: Graphviz - Graph Visualization Software. {Online}. Available: http://www.graphviz.org/.
• 31: T. Kosar. PetaShare: an innovative distributed data archival, analysis and visualisation instrument for data intensive collaborative research. http://www.petashare.org
• 32: Ann Chervenak , Ewa Deelman , Ian Foster , Leanne Guy , Wolfgang Hoschek , Adriana Iamnitchi , Carl Kesselman , Peter Kunszt , Matei Ripeanu , Bob Schwartzkopf , Heinz Stockinger , Kurt Stockinger , Brian Tierney, Giggle: a framework for constructing scalable replica location services, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-17, November 16, 2002, Baltimore, Maryland
• 33: E. Deelman, C. Kesselman, and G. Mehta. Transformation Catalog Design for GriPhyN. Technical Report GriPhN-2001-17, 2001.
• 34: Pegasus. {Online}. Available: http://pegasus.isi.edu/index.php.

INDEX TERMS

Primary Classification:
  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Distributed programming

General Terms:
Design, Experimentation, Performance, Reliability

Keywords:
Stork, data management, distributed systems, end-to-end-processing, reservoir modeling, uncertainty analysis, workflows