Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

The Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) began as an alternative to distributed searching of scholarly eprint repositories. The model embraced by the OAI-PMH is that of metadata harvesting, where value-added services (by a "service provider") are constructed on cached copies of the metadata extracted from the repositories of the harvester's choosing. While this model dispenses with the well known problems of distributed searching, it introduces the problem of synchronization. Stated simply, this problem arises when the service provider's copy of the metadata does not match the metadata currently at the constituent repositories. We define some metrics for describing the synchronization problem in the OAI-PMH. Based on these metrics, we study the synchronization problem of the OAI-PMH framework and propose several approaches for harvesters to implement better synchronization. In particular, if a repository knows its update frequency, it can publish it in an OAI-PMH Identify response using an optional About container that borrows from RDF Site Syndication (RSS) Format.

REFERENCES

• 1: G. Beged-Dov, D. Brickley, R. Dornfest, I. Davis, L. Dodds, J. Eisenzopf, D. Galbraith, R. Guha, K. MacLeod, E. Miller, A. Swartz, and E. van der Vlist. RDF Site Summary 1.0 Modules: Syndication, 2000. http://purl.org/rss/1.0/modules/syndication/.
• 2: C. Mic Bowman , Peter B. Danzig , Darren R. Hardy , Udi Manber , Michael F. Schwartz, The Harvest information discovery and access system, Computer Networks and ISDN Systems, v.28 n.1-2, p.119-125, Dec. 1995 [doi>10.1016/0169-7552(95)00098-5]
• 3: T. Brody. Mining the social life of an eprint archive. http://opcit.eprints.org/tdb198/opcit/.
• 4: Junghoo Cho , Hector Garcia-Molina, Crawling the web: discovery and maintenance of large-scale web data, 2002
• 5: Junghoo Cho , Hector Garcia-Molina, Synchronizing a database to improve freshness, Proceedings of the 2000 ACM SIGMOD international conference on Management of data, p.117-128, May 15-18, 2000, Dallas, Texas, United States [doi>10.1145/342009.335391]
• 6: James R. Davis , Carl Lagoze, NCSTRL: design and deployment of a globally distributed digital library, Journal of the American Society for Information Science, v.51 n.3, p.273-280, Feb. 2000 [doi>10.1002/(SICI)1097-4571(2000)51:3<273::AID-ASI6>3.0.CO;2-6]
• 7: Luis Gravano , Chen-Chuan K. Chang , Héctor García-Molina , Andreas Paepcke, STARTS: Stanford proposal for Internet meta-searching, Proceedings of the 1997 ACM SIGMOD international conference on Management of data, p.207-218, May 11-15, 1997, Tucson, Arizona, United States [doi>10.1145/253260.253299]
• 8: Alexandros Labrinidis , Nick Roussopoulos, Update Propagation Strategies for Improving the Quality of Data on the Web, Proceedings of the 27th International Conference on Very Large Data Bases, p.391-400, September 11-14, 2001
• 9: Carl Lagoze , William Arms , Stoney Gan , Diane Hillmann , Christopher Ingram , Dean Krafft , Richard Marisa , Jon Phipps , John Saylor , Carol Terrizzi , Walter Hoehn , David Millman , James Allan , Sergio Guzman-Lara , Tom Kalt, Core services in the architecture of the national science digital library (NSDL), Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, July 14-18, 2002, Portland, Oregon, USA [doi>10.1145/544220.544264]
• 10: C. Lagoze, H. Van de Sompel, M. Nelson, and S. Warner. The Open Archives Initiative Protocol for Metadata Harvesting, version 2.0. http://www.openarchives.org/OAI/openarchivesprotocol.html.
• 11: Xiaoming Liu , Kurt Maly , Mohammad Zubair, Federating heterogeneous digital libraries by metadata harvesting, 2002
• 12: Enhanced Kepler Framework for Self-Archiving, Proceedings of the 2002 International Conference on Parallel Processing Workshops, p.455, August 18-21, 2002
• 13: X. Liu, K. Maly, M. Zubair, and M. L. Nelson. Arc - an OAI service provider for digital library federation. DLib Magazine, 7(4), 2001. http://www.dlib.org/dlib/april01/liu/04liu.html.
• 14: Xiaoming Liu , Kurt Maly , Mohammad Zubair , Michael L. Nelson, Arc: an OAI service provider for cross-archive searching, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.65-66, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379451]
• 15: Clifford A. Lynch, When documents deceive: trust and provenance as new factors for information retrieval in a tangled web, Journal of the American Society for Information Science and Technology, v.52 n.1, p.12-17, Jan. 2001 [doi>10.1002/1532-2890(2000)52:1<12::AID-ASI1062>3.3.CO;2-M]
• 16: R. Rivest. The MD5 message-digest algorithm. Technical Report Internet RFC-1321, IETF, 1992. http://www.ietf.org/rfc/rfc1321.txt.
• 17: A. Van Hoff, J. Giannandrea, M. Hapner, S. Carter, and M. M. The HTTP distribution and replication protocol. Technical Report NOTE-DRP, World Wide Web Consortium, 1997. http://www.w3.org/TR/NOTE-drp.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.7 Database Administration
          Subjects: Data warehouse and repository

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.5 On-line Information Services
          Subjects: Data sharing
      H.3.6 Library Automation
          Subjects: Large text archives
      H.3.7 Digital Libraries
          Subjects: Dissemination

General Terms:
Management, Performance