Scientific formats for object-relational database systems: a study of suitability and performance

ABSTRACT

Commercial database management systems (DBMSs) have historically seen very limited use within the scientific computing community. One reason for this absence is that previous database systems lacked support for the extensible data structures and performance features required within a high-performance computing context. However, database vendors have recently enhanced the functionality of their systems by adding object extensions to the relational engine. In principle, these extensions allow for the representation of a rich collection of scientific datatypes and common statistical operations. Utilizing these new extensions, this paper presents a study of the suitability of incorporating two popular scientific formats, NetCDF and HDF, into an object-relational system. To assess the performance of the database approach, a series of solution variables from a regional weather forecast model are used to build representative small, medium and large databases. Common statistical operations and array element queries are then performed using the object-relational database, and the execution timings are compared against native NetCDF and HDF operations.

REFERENCES

• 1: Gray, J., Liu, D., Nieto-Santisteban, M., Szalay, A., DeWitt, D., and Heber, G., Scientific Data Management in the Coming Decade. Microsoft Research Technical Report MSR-TR-2005-10 (2005).
• 2: Rew, R. K., G. P. Davis, S. Emmerson, and H. Davies, "NetCDF User's Guide for C, An Interface for Data Access", Version 3, April 1997.
• 3: Hierarchical Data Format (HDF): http://hdf.ncsa.uiuc.edu/hdf4.html
• 4: Network Common Data Form -- NetCDF is Not a Database Management System: http://my.unidata.ucar.edu/content/software/netcdf/docs/netcdf/Not-DBMS.html
• 5: Michael Stonebraker , Dorothy Moore , Paul Brown, Object-Relational DBMSs: Tracking the Next Great Wave, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 6: Michalakes, J., J. Dudhia, D. Gill, T. Henderson, J. Klemp, W. Skamarock, and W. Wang, "The Weather Research and Forecast Model: Software Architecture and Performance", Proceedings of the 11th ECMWF Workshop on the Use of High Performance Computing In Meteorology, October, 2004.
• 7: Won Kim, Modern database systems: the object model, interoperability, and beyond, ACM Press/Addison-Wesley Publishing Co., New York, NY, 1995
• 8: Vishu Krishnamurthy , Sandeepan Banerjee , Anil Nori, Bringing object-relational technology to the mainstream, Proceedings of the 1999 ACM SIGMOD international conference on Management of data, p.513-514, May 31-June 03, 1999, Philadelphia, Pennsylvania, United States
• 9: Oracle Database. Application Developer's Guide - Object Relational Features 10g Release 1 (10.1). December 2003.
• 10: Oracle Corporation. Cost Based Optimizer (CBO) Overview. Note: 10626.1. June 2002.
• 11: Oracle Corporation. Cost Based Optimizer -- Common Misconceptions and Issues. Note: 35934.1. April 2004.
• 12: The NetCDF2HDF conversion utility: http://ioc.unesco.org/oceanteacher/resourcekit/M3/Converters/NetCDF2HDF/.
• 13: Oracle Corporation. Interpreting Explain plan. Note 46234.1. October, 2003.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Object-oriented databases

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.1 Logical Design
      H.2.2 Physical Design
      H.2.4 Systems
          Subjects: Relational databases
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)

General Terms:
Design, Management, Performance