A metadata generation system for scanned scientific volumes

ABSTRACT

Large scale digitization projects have been conducted at digital libraries to preserve cultural artifacts and to provide permanent access. The increasing amount of digitized resources, including scanned books and scientific publications, requires development of tools and methods that will efficiently analyze and manage large collections of digitized resources. In this work, we tackle the problem of extracting metadata from scanned volumes of journals. Our goal is to extract information describing internal structures and content of scanned volumes, which is necessary for providing effective content access functionalities to digital library users. We propose methods for automatically generating volume level, issue level, and article level metadata based on format and text features extracted from OCRed text. We show the performance of our system on scanned bound historical documents nearly two centuries old. We have developed the system and integrated it into an operational digital library, the Internet Archive, for real-world usage.

REFERENCES

• 1: Biodiversity Heritage Library. http://www.biodiversitylibrary.org.
• 2: Djvu Zone. http://www.djvuzone.org.
• 3: Dublin Core Metadata Initiative. http://dublincore.org.
• 4: Gem. http://www.thegateway.org.
• 5: Google Book Search. http://books.google.com.
• 6: Internet Archive. http://www.archive.org.
• 7: Smithsonian Institute. http://www.si.edu.
• 8: The Universal Digital Library. In http://www.ulib.org.
• 9: W. Y. Arms, C. Blanchi, and E. A. Overly. An Architecture for Information in Digital Libraries. 1997.
• 10: H. Besser. The Next Stage: Moving from Isolated Digtal Collections to Interoperable Digital Libraries. http://www.firstmonday.dk/issues/issue7_6/besser. 2002.
• 11: Francesca Cesarini , Marco Lastri , Simone Marinai , Giovanni Soda, Page Classification for Meta-data Extraction from Digital Collections, Proceedings of the 12th International Conference on Database and Expert Systems Applications, p.82-91, September 03-05, 2001
• 12: Naomi Dushay, Localizing experience of digital content via structural metadata, Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, July 14-18, 2002, Portland, Oregon, USA [doi>10.1145/544220.544274]
• 13: C. Lee Giles , Kurt D. Bollacker , Steve Lawrence, CiteSeer: an automatic citation indexing system, Proceedings of the third ACM conference on Digital libraries, p.89-98, June 23-26, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276675.276685]
• 14: Giovanni Giuffrida , Eddie C. Shek , Jihoon Yang, Knowledge-based metadata extraction from PostScript files, Proceedings of the fifth ACM conference on Digital libraries, p.77-84, June 02-07, 2000, San Antonio, Texas, United States [doi>10.1145/336597.336639]
• 15: Hui Han , C. Lee Giles , Eren Manavoglu , Hongyuan Zha , Zhenyue Zhang , Edward A. Fox, Automatic document metadata extraction using support vector machines, Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, May 27-31, 2003, Houston, Texas
• 16: Yunhua Hu , Hang Li , Yunbo Cao , Dmitriy Meyerzon , Qinghua Zheng, Automatic extraction of titles from general documents using machine learning, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065418]
• 17: Nathalie Japkowicz , Shaju Stephen, The class imbalance problem: A systematic study, Intelligent Data Analysis, v.6 n.5, p.429-449, October 2002
• 18: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 19: M. Kubat and S. Matwin. Addressing the curse of imbalanced training sets: one-sided selection. In International Conference on Machine Learning, pages 179--186, 1997.
• 20: V. I. Levenshtein. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet Physics Doklady, pages 707--710, 1966.
• 21: Y. Liu, E. Shriberg, A. Stolcke, B. Peskin, J. Ang, D. Hillard, M. Ostendorf, M. Tomalin, P. Woodland, and M. Harper. Structural metadata research in the ears program. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing, pages 957--960, 2005.
• 22: Song Mao , Jong Woo Kim , George R. Thoma, A Dynamic Feature Generation System for Automated Metadata Extraction in Preservation of Digital Materials, Proceedings of the First International Workshop on Document Image Analysis for Libraries (DIAL'04), p.225, January 23-24, 2004
• 23: Yves Petinot , C. Lee Giles , Vivek Bhatnagar , Pradeep B. Teregowda , Hui Han , Isaac Councill, A service-oriented architecture for digital libraries, Proceedings of the 2nd international conference on Service oriented computing, November 15-19, 2004, New York, NY, USA [doi>10.1145/1035167.1035205]
• 24: R. Wendler. LDI Update: Metadata in the Library. Library Notes, no. 1286 (July/August): 4--5, 1999.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.7 Digital Libraries
          Subjects: Collection

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
      H.3.7 Digital Libraries
          Subjects: Systems issues

General Terms:
Algorithms, Design, Experimentation

Keywords:
metadata generation, scanned journal