Model management 2.0: manipulating richer mappings

ABSTRACT

Model management is a generic approach to solving problems of data programmability where precisely engineered mappings are required. Applications include data warehousing, e-commerce, object-to-relational wrappers, enterprise information integration, database portals, and report generators. The goal is to develop a model management engine that can support tools for all of these applications. The engine supports operations to match schemas, compose mappings, diff schemas, merge schemas, translate schemas into different data models, and generate data transformations from mappings.

Much has been learned about model management since it was proposed seven years ago. This leads us to a revised vision that differs from the original in two main respects: the operations must handle more expressive mappings, and the runtime that executes mappings should be added as an important model management component. We review what has been learned from recent experience, explain the revised model management vision based on that experience, and identify the research problems that the revised vision opens up.

REFERENCES

• 1: Serge Abiteboul , Oliver M. Duschka, Complexity of answering queries using materialized views, Proceedings of the seventeenth ACM SIGACT-SIGMOD-SIGART symposium on Principles of database systems, p.254-263, June 01-04, 1998, Seattle, Washington, United States [doi>10.1145/275487.275516]
• 2: Serge Abiteboul , Rakesh Agrawal , Phil Bernstein , Mike Carey , Stefano Ceri , Bruce Croft , David DeWitt , Mike Franklin , Hector Garcia Molina , Dieter Gawlick , Jim Gray , Laura Haas , Alon Halevy , Joe Hellerstein , Yannis Ioannidis , Martin Kersten , Michael Pazzani , Mike Lesk , David Maier , Jeff Naughton , Hans Schek , Timos Sellis , Avi Silberschatz , Mike Stonebraker , Rick Snodgrass , Jeff Ullman , Gerhard Weikum , Jennifer Widom , Stan Zdonik, The Lowell database research self-assessment, Communications of the ACM, v.48 n.5, p.111-118, May 2005 [doi>10.1145/1060710.1060718]
• 3: S. Abiteboul, R. Hull, V. Vianu: Foundations of Databases. Addison-Wesley, 1995.
• 4: A. Adya, J. A. Blakeley, S. Melnik, S. Muralidhar, and the ADO.NET Team: Anatomy of the ADO.NET Entity Framework, SIGMOD 2007.
• 5: Altova, http://www.altova.com/
• 6: Paolo Atzeni , Riccardo Torlone, Management of Multiple Models in an Extensible Database Design Tool, Proceedings of the 5th International Conference on Extending Database Technology: Advances in Database Technology, p.79-95, March 25-29, 1996
• 7: P. Atzeni, P. Cappellari and P. Bernstein: ModelGen: Model Independent Schema Translation. EDBT 2006, 368--385.
• 8: F. Bancilhon , N. Spyratos, Update semantics of relational views, ACM Transactions on Database Systems (TODS), v.6 n.4, p.557-575, Dec. 1981 [doi>10.1145/319628.319634]
• 9: BEA Aqualogic User Interaction, http://www.bea.com
• 10: P. A. Bernstein: Applying Model Management to Classical Meta Data Problems. CIDR 2003.
• 11: David Shutt , Philip A. Bernstein , Thomas Bergstraesser , Jason Carlson , Shankar Pal , Paul Sanders, Microsoft repository version 2 and the open information model, Information Systems, v.24 n.2, p.71-98, April 1999 [doi>10.1016/S0306-4379(99)00006-X]
• 12: Phil Bernstein , Michael Brodie , Stefano Ceri , David DeWitt , Mike Franklin , Hector Garcia-Molina , Jim Gray , Jerry Held , Joe Hellerstein , H. V. Jagadish , Michael Lesk , Dave Maier , Jeff Naughton , Hamid Pirahesh , Mike Stonebraker , Jeff Ullman, The Asilomar report on database research, ACM SIGMOD Record, v.27 n.4, p.74-80, Dec. 1998 [doi>10.1145/306101.306137]
• 13: P. A. Bernstein, Dayal, U., DeWitt, D. J., Gawlick, D., Gray, J., Jarke, M., Lindsay, B. G., Lockemann, P. C., Maier, D., Neuhold, E. J., Reuter, A., Rowe, L. A., Schek, H. J., Schmidt, J. W., Schrefl, M., and Stonebraker: M. Future Directions in DBMS research. The Laguna Beach Participants. SIGMOD Record (18)1: 17--26 (1989).
• 14: P. A. Bernstein, F. Giunchiglia, A. Kementsietsidis, J. Mylopoulos, L. Serafini, I. Zaihrayeu: Data Management for Peer-to-Peer Computing : A Vision. WebDB 2002: 89--94.
• 15: Philip A. Bernstein , Todd J. Green , Sergey Melnik , Alan Nash, Implementing mapping composition, Proceedings of the 32nd international conference on Very large data bases, September 12-15, 2006, Seoul, Korea
• 16: Philip A. Bernstein , Laura M. Haas , Matthias Jarke , Erhard Rahm , Gio Wiederhold, Panel: Is Generic Metadata Management Feasible?, Proceedings of the 26th International Conference on Very Large Data Bases, p.660-662, September 10-14, 2000
• 17: Phillip A. Bernstein , Alon Y. Halevy , Rachel A. Pottinger, A vision for management of complex models, ACM SIGMOD Record, v.29 n.4, p.55-63, Dec. 2000 [doi>10.1145/369275.369289]
• 18: Philip A. Bernstein , Sergey Melnik , John E. Churchill, Incremental schema matching, Proceedings of the 32nd international conference on Very large data bases, September 12-15, 2006, Seoul, Korea
• 19: Philip A. Bernstein , Sergey Melnik , Peter Mork, Interactive schema translation with instance-level mappings, Proceedings of the 31st international conference on Very large data bases, August 30-September 02, 2005, Trondheim, Norway
• 20: P. A. Bernstein and E. Rahm: Data Warehouse Scenarios for Model Management. ER 2000: 1--15.
• 21: J. Biskup and B. Convent: A Formal View Integration Method. SIGMOD 1986: 398--407.
• 22: José A. Blakeley , David Campbell , S. Muralidhar , Anil Nori, The ADO.NET entity framework: making the conceptual level real, ACM SIGMOD Record, v.35 n.4, December 2006 [doi>10.1145/1228268.1228275]
• 23: M. S. Bloor and J. Owen: Product Data Exchange. CRC Press, 1995.
• 24: Philip Bohannon , Wenfei Fan , Michael Flaster , P. P. S. Narayan, Information preserving XML schema embedding, Proceedings of the 31st international conference on Very large data bases, August 30-September 02, 2005, Trondheim, Norway
• 25: Peter Buneman , Susan B. Davidson , Anthony Kosky, Theoretical Aspects of Schema Merging, Proceedings of the 3rd International Conference on Extending Database Technology: Advances in Database Technology, p.152-167, March 23-27, 1992
• 26: Michael Carey, Data delivery in a service-oriented world: the BEA aquaLogic data services platform, Proceedings of the 2006 ACM SIGMOD international conference on Management of data, June 27-29, 2006, Chicago, IL, USA [doi>10.1145/1142473.1142551]
• 27: T. Catarci and M. Lenzerini: Representing and Using Interschema Knowledge in Cooperative Information Systems. Int. J. Cooperative Inf. Syst. 2(4): 375--398 (1993).
• 28: R.G.G. Cattell and D.K. Barry (editors) et al.: The Object Data Standard: ODMG 3.0. Morgan Kaufmann, 2000.
• 29: Surajit Chaudhuri , Umeshwar Dayal, An overview of data warehousing and OLAP technology, ACM SIGMOD Record, v.26 n.1, p.65-74, March 1997 [doi>10.1145/248603.248616]
• 30: Laura Chiticariu , Wang-Chiew Tan, Debugging schema mappings with routes, Proceedings of the 32nd international conference on Very large data bases, September 12-15, 2006, Seoul, Korea
• 31: G.P. Copeland and D. Maier: Making Smalltalk a Database System. SIGMOD 1984: 316--325.
• 32: Crystal Reports, http://www.businessobjects.com/ products/reporting/crystalreports/default.asp.
• 33: S.B. Davidson, G. Christian Overton, V. Tannen, L. Wong: BioKleisli: A Digital Library for Biomedical Researchers. Int. J. on Digital Libraries 1(1): 36--53 (1997).
• 34: Umeshwar Dayal, Processing Queries Over Generalization Hierarchies in a Multidatabase System, Proceedings of the 9th International Conference on Very Large Data Bases, p.342-353, October 31-November 02, 1983
• 35: L. Donelson, P. Tarczy-Hornoch, P. Mork, C. Dolan, JA Mitchell, M. Barrier, H. Mei: The BioMediator System as a Data Integration Tool to Answer Diverse Biologic Queries. Medinfo: 768--72, 2003.
• 36: ETL Tool Survey 2006-2007, http://www.etltool.com/
• 37: Ronald Fagin, Inverting schema mappings, Proceedings of the twenty-fifth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, June 26-28, 2006, Chicago, IL, USA [doi>10.1145/1142351.1142359]
• 38: Ronald Fagin , Phokion G. Kolaitis , Renée J. Miller , Lucian Popa, Data exchange: semantics and query answering, Theoretical Computer Science, v.336 n.1, p.89-124, 25 May 2005 [doi>10.1O16/j.tcs.2004.10.033]
• 39: Ronald Fagin , Phokion G. Kolaitis , Lucian Popa, Data exchange: getting to the core, ACM Transactions on Database Systems (TODS), v.30 n.1, p.174-210, March 2005 [doi>10.1145/1061318.1061323]
• 40: Ronald Fagin , Phokion G. Kolaitis , Lucian Popa , Wang-Chiew Tan, Composing schema mappings: Second-order dependencies to the rescue, ACM Transactions on Database Systems (TODS), v.30 n.4, p.994-1055, December 2005 [doi>10.1145/1114244.1114249]
• 41: R. Fagin, P. G. Kolaitis, L. Popa, and W. C. Tan: Quasi-inverses of Schema Mappings. PODS 2007.
• 42: S. M. Falconer and M. Storey: Cognitive Support for Human-Guided Mapping Systems. Tech. Report DCS-318-IR, 2007, Univ. of Victoria , http://www.cs.uvic.ca/~seanf/ files/cog_support_mapping_systems.pdf
• 43: Michael Franklin , Alon Halevy , David Maier, From databases to dataspaces: a new abstraction for information management, ACM SIGMOD Record, v.34 n.4, p.27-33, December 2005 [doi>10.1145/1107499.1107502]
• 44: Marc Friedman , Alon Levy , Todd Millstein, Navigational plans for data integration, Proceedings of the sixteenth national conference on Artificial intelligence and the eleventh Innovative applications of artificial intelligence conference innovative applications of artificial intelligence, p.67-73, July 18-22, 1999, Orlando, Florida, United States
• 45: Ariel Fuxman , Mauricio A. Hernandez , Howard Ho , Renee J. Miller , Paolo Papotti , Lucian Popa, Nested mappings: schema mapping reloaded, Proceedings of the 32nd international conference on Very large data bases, September 12-15, 2006, Seoul, Korea
• 46: A. Gal: Managing Uncertainty in Schema Matching with Top-K Schema Mappings. J. Data Semantics VI: 90--114, Springer LNCS Vol. 4090/2006.
• 47: M. Gubanov, P. A. Bernstein: Structural Text Search and Comparison using Automatically Extracted Schema. WebDB 2006.
• 48: L. M. Haas: Beauty and the Beast: The Theory and Practice of Information Integration. ICDT 2007: 28--43.
• 49: Laura M. Haas , Mauricio A. Hernández , Howard Ho , Lucian Popa , Mary Roth, Clio grows up: from research prototype to industrial tool, Proceedings of the 2005 ACM SIGMOD international conference on Management of data, June 14-16, 2005, Baltimore, Maryland [doi>10.1145/1066157.1066252]
• 50: Alon Y. Halevy, Answering queries using views: A survey, The VLDB Journal — The International Journal on Very Large Data Bases, v.10 n.4, p.270-294, December 2001 [doi>10.1007/s007780100054]
• 51: Alon Y. Halevy , Naveen Ashish , Dina Bitton , Michael Carey , Denise Draper , Jeff Pollock , Arnon Rosenthal , Vishal Sikka, Enterprise information integration: successes, challenges and controversies, Proceedings of the 2005 ACM SIGMOD international conference on Management of data, June 14-16, 2005, Baltimore, Maryland [doi>10.1145/1066157.1066246]
• 52: Alon Halevy , Michael Franklin , David Maier, Principles of dataspace systems, Proceedings of the twenty-fifth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, p.1-9, June 26-28, 2006, Chicago, IL, USA [doi>10.1145/1142351.1142352]
• 53: The Piazza Peer Data Management System, IEEE Transactions on Knowledge and Data Engineering, v.16 n.7, p.787-798, July 2004 [doi>10.1109/TKDE.2004.1318562]
• 54: Hibernate, http://www.hibernate.org
• 55: Richard Hull, Relative information capacity of simple relational database schemata, SIAM Journal on Computing, v.15 n.3, p.856-886, August 1986 [doi>10.1137/0215061]
• 56: IBM FileNet Forms Manager, http://www.filenet.com/English/Products/Forms_Manager/.
• 57: IBM Rational Data Architect, http://www--306.ibm.com/software/data/integration/rda/
• 58: C. Keene: Data Services for Next-Generation SOAs. SOA WebServices Journal, 4(12), 2004. http://webservices.sys-con.com/read/47283.htm
• 59: R. Kimball and J. Caserta. The Data Warehouse ETL Toolkit, Wiley and Sons, 2004.
• 60: Phokion G. Kolaitis, Schema mappings, data exchange, and metadata management, Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, June 13-15, 2005, Baltimore, Maryland [doi>10.1145/1065167.1065176]
• 61: Jens Lechtenbörger , Gottfried Vossen, On the computation of relational view complements, ACM Transactions on Database Systems (TODS), v.28 n.2, p.175-208, June 2003 [doi>10.1145/777943.777946]
• 62: F. Legler and F. Naumann: A Classification of Schema Mappings and Analysis of Mapping Tools. BTW 2007: 449--464.
• 63: Maurizio Lenzerini, Data integration: a theoretical perspective, Proceedings of the twenty-first ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, June 03-05, 2002, Madison, Wisconsin [doi>10.1145/543613.543644]
• 64: J. Lin and A. O. Mendelzon: Merging Databases Under Constraints. Int. J. Cooperative Inf. Syst. 7(1): 55--76 (1998).
• 65: Peter McBrien , Alexandra Poulovassilis, A Uniform Approach to Inter-model Transformations, Proceedings of the 11th International Conference on Advanced Information Systems Engineering, p.333-348, June 14-18, 1999
• 66: S. Melnik: Generic Model Management: Concepts and Algorithms, Springer LNCS 2967, 2004.
• 67: Sergey Melnik , Philip A. Bernstein , Alon Halevy , Erhard Rahm, Supporting executable mappings in model management, Proceedings of the 2005 ACM SIGMOD international conference on Management of data, June 14-16, 2005, Baltimore, Maryland [doi>10.1145/1066157.1066177]
• 68: S. Melnik, P. A. Bernstein, A. Y. Halevy, and E. Rahm: A Semantics for Model Management Operators. MSR-TR-2004-59, http://research.microsoft.com, June 2004. An early but somewhat extended version of {67}.
• 69: Sergey Melnik , Erhard Rahm , Philip A. Bernstein, Rondo: a programming platform for generic model management, Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872782]
• 70: S. Melnik, A. Adya and P. A. Bernstein, Compiling Mappings to Bridge Applications and Databases, SIGMOD 2007.
• 71: Microsoft BizTalk, http://www.microsoft.com/biztalk/
• 72: Microsoft Office InfoPath, http://office.microsoft.com/en-us/infopath
• 73: Microsoft Sharepoint Server, http://www.microsoft.com/sharepoint
• 74: Microsoft SQL Server Reporting Services, http://www.microsoft.com/sql/technologies/reporting/
• 75: Renée J. Miller , Laura M. Haas , Mauricio A. Hernández, Schema Mapping as Query Discovery, Proceedings of the 26th International Conference on Very Large Data Bases, p.77-88, September 10-14, 2000
• 76: Renée J. Miller , Mauricio A. Hernández , Laura M. Haas , Lingling Yan , C. T. Howard Ho , Ronald Fagin , Lucian Popa, The Clio project: managing heterogeneity, ACM SIGMOD Record, v.30 n.1, p.78-83, March 2001 [doi>10.1145/373626.373713]
• 77: MSDN Library: The ADO.NET Entity Framework Overview. June 2006. http://msdn2.microsoft.com/en-us/library/aa697427(vs.80).aspx
• 78: A. Nash, P. A. Bernstein, and S. Melnik: Composition of Mappings Given by Embedded Dependencies. PODS 2005: 172--183. Extended version to appear in ACM TODS.
• 79: Oracle Toplink, http://www.oracle.com/technology/products/ias/toplink/index.html
• 80: OWL Web Ontology Language Reference, http://www.w3.org/TR/owl-ref/
• 81: P. Papotti and R. Torlone: An Approach to Heterogeneous Data Translation based on XML Conversion. CAiSE Workshops (1) 2004: 7--19.
• 82: R. Pottinger and P. A. Bernstein: Merging Models Based on Given Correspondences. VLDB 2003: 826--873.
• 83: Alexandra Poulovassilis , Peter Mc. Brien, A general formal framework for schema transformation, Data & Knowledge Engineering, v.28 n.1, p.47-71, Oct. 30, 1998 [doi>10.1016/S0169-023X(98)00013-5]
• 84: Query Tools: Products, http://www.bitpipe.com/plist/term/Query-Tools.html
• 85: Resource Description Framework, http://www.w3.org/RDF/
• 86: Erhard Rahm , Philip A. Bernstein, A survey of approaches to automatic schema matching, The VLDB Journal — The International Journal on Very Large Data Bases, v.10 n.4, p.334-350, December 2001 [doi>10.1007/s007780100057]
• 87: Erhard Rahm , Philip A. Bernstein, An online bibliography on schema evolution, ACM SIGMOD Record, v.35 n.4, p.30-31, December 2006 [doi>10.1145/1228268.1228273]
• 88: George G. Robertson , Mary P. Czerwinski , John E. Churchill, Visualization of mappings between schemas, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055032]
• 89: M. Roth , M. A. Hernandez , P. Coulthard , L. Yan , L. Popa , H. C.-T. Ho , C. C. Salter, XML Mapping technology: making connections in an XML-centric world, IBM Systems Journal, v.45 n.2, p.389-409, January 2006
• 90: SAP Netweaver Portal, http://www.sap.com/usa/platform/netweaver/components/portal/index.epx
• 91: Service Modeling Language, http://www.serviceml.org/
• 92: P. Shvaiko and J. Euzenat: A Survey of Schema-based Matching Approaches. J. Data Semantics IV:146--171 (2005).
• 93: N. C. Shu , B. C. Housel , R. W. Taylor , S. P. Ghosh , V. Y. Lum, EXPRESS: a data EXtraction, Processing, and Restructuring System, ACM Transactions on Database Systems (TODS), v.2 n.2, p.134-174, June 1977 [doi>10.1145/320544.320549]
• 94: Avi Silberschatz , Michael Stonebraker , Jeff Ullman, Database systems: achievements and opportunities, Communications of the ACM, v.34 n.10, p.110-120, Oct. 1991 [doi>10.1145/125223.125272]
• 95: J. M. Smith, P. A. Bernstein, U. Dayal, N. Goodman, T. Landers, K. W. T. Lin, E. Wong, "MULTIBASE-Integrating Heterogeneous Distributed Database Systems," Proc. of 1981 National Computer Conf., AFIPS Press, 487--499.
• 96: Solidworks, http://www.solidworks.com/
• 97: S. Spaccapietra , C. Parent, View Integration: A Step Forward in Solving Structural Conflicts, IEEE Transactions on Knowledge and Data Engineering, v.6 n.2, p.258-274, April 1994 [doi>10.1109/69.277770]
• 98: Stylus Studio, http://www.stylusstudio.com/
• 99: Y. Velegrakis, R. J. Miller, and L. Popa: Mapping Adaptation under Evolving Schemas. VLDB 2003: 584--595.
• 100: Gio Wiederhold, Mediators in the Architecture of Future Information Systems, Computer, v.25 n.3, p.38-49, March 1992 [doi>10.1109/2.121508]
• 101: Workshop on Information Integration, Oct. 2006, http://db.cis.upenn.edu/iiworkshop/index.htm
• 102: L. -L. Yan, R. J. Miller, L. M. Haas, R. Fagin: Data-Driven Understanding and Refinement of Schema Mappings. SIGMOD 2001: 485--496.
• 103: Cong Yu , Lucian Popa, Semantic adaptation of schema mappings when schemas evolve, Proceedings of the 31st international conference on Very large data bases, August 30-September 02, 2005, Trondheim, Norway

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.5 Heterogeneous Databases

General Terms:
Algorithms, Design, Theory

Keywords:
data exchange, data integration, data translation, engineered mapping, model management, schema evolution, schema mapping, schema matching