Automatic categorization of figures in scientific documents

ABSTRACT

Figures are very important non-textual information contained in scientific documents. Current digital libraries do not provide users tools to retrieve documents based on the information available within the figures. We propose an architecture for retrieving documents by integrating figures and other information. The initial step in enabling integrated document search is to categorize figures into a set of pre-defined types. We propose several categories of figures based on their functionalities in scholarly articles. We have developed a machine-learning-based approach for automatic categorization of figures. Both global features, such as texture, and part features, such as lines, are utilized in the architecture for discriminating among figure categories. The proposed approach has been evaluated on a testbed document set collected from the CiteSeer scientific literature digital library. Experimental evaluation has demonstrated that our algorithms can produce acceptable results for realworld use. Our tools will be integrated into a scientific document digital library.

REFERENCES

• 1: Henry S. Baird , Daniel Lopresti , Brian D. Davison , William M. Pottenger, Robust document image understanding technologies, Proceedings of the 1st ACM workshop on Hardcopy document processing, p.9-14, November 12-12, 2004, Washington, DC, USA [doi>10.1145/1031442.1031444]
• 2: I. Biederman. Recognition-by-components: A theory of human image understanding. Psychological Review, 94:115--147, 1987.
• 3: Dorothea Blostein , Edward Lank , Richard Zanibbi, Treatment of Diagrams in Document Image Analysis, Proceedings of the First International Conference on Theory and Application of Diagrams, p.330-344, September 01-03, 2000
• 4: J Canny, A computational approach to edge detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.8 n.6, p.679-698, Nov. 1986 [doi>10.1109/TPAMI.1986.4767851]
• 5: Yixin Chen , James Z. Wang, A Region-Based Fuzzy Feature Matching Approach to Content-Based Image Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.9, p.1252-1267, September 2002 [doi>10.1109/TPAMI.2002.1033216]
• 6: Michael G. Christel , Ronald M. Conescu, Addressing the challenge of visual information access from digital image and video libraries, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065402]
• 7: Ritendra Datta , Jia Li , James Z. Wang, Content-based image retrieval: approaches and trends of the new age, Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval, November 10-11, 2005, Hilton, Singapore [doi>10.1145/1101826.1101866]
• 8: David Doermann, The indexing and retrieval of document images: a survey, Computer Vision and Image Understanding, v.70 n.3, p.287-298, June 1, 1998 [doi>10.1006/cviu.1998.0692]
• 9: Richard O. Duda , Peter E. Hart, Use of the Hough transformation to detect lines and curves in pictures, Communications of the ACM, v.15 n.1, p.11-15, Jan. 1972 [doi>10.1145/361237.361242]
• 10: E. Giladi, M. G. Walker, J. Z. Wang, and W. Volkmuth. SST: An algorithm for finding near-exact sequence matches in time proportional to the logarithm of the database size. Bioinformatics, 18(6):873--879, 2002.
• 11: C. Lee Giles , Kurt D. Bollacker , Steve Lawrence, CiteSeer: an automatic citation indexing system, Proceedings of the third ACM conference on Digital libraries, p.89-98, June 23-26, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276675.276685]
• 12: Hui Han , C. Lee Giles , Eren Manavoglu , Hongyuan Zha , Zhenyue Zhang , Edward A. Fox, Automatic document metadata extraction using support vector machines, Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, May 27-31, 2003, Houston, Texas
• 13: Hui Han , Hongyuan Zha , C. Lee Giles, Name disambiguation in author citations using a K-way spectral clustering method, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065462]
• 14: T. Hastie, R. Tibshirani, and J. Friedman. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag, New York, NY, 2001.
• 15: Mike Heath , Sudeep Sarkar , Thomas Sanocki , Kevin Bowyer, Comparison of edge detectors: a methodology and initial study, Computer Vision and Image Understanding, v.69 n.1, p.38-54, Jan. 1998 [doi>10.1006/cviu.1997.0587]
• 16: Yunhua Hu , Hang Li , Yunbo Cao , Dmitriy Meyerzon , Qinghua Zheng, Automatic extraction of titles from general documents using machine learning, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065418]
• 17: T. Joachims. Making Large-Scale Support Vector Machine Learning Practical. MIT Press, Cambridge, MA, 1998.
• 18: D. Joshi, J. Li, and J. Z. Wang. A computationally efficient approach to the estimation of two- and three-dimensional hidden markov models. IEEE Transactions on Image Processing, 2006, to appear.
• 19: J. Li and R. M. Gray. Context-based multiscale classification of document images using wavelet coefficient distributions. IEEE Transactions on Image Processing, 9(9):1604--1616, 2000.
• 20: Jia Li , James Z. Wang, Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1075-1088, September 2003 [doi>10.1109/TPAMI.2003.1227984]
• 21: S. Mao and A. Rosenfeld. Document structure analysis algorithms: a literature survey. In Proceedings of SPIE, pages 197--207, 2003.
• 22: Mor Naaman , Ron B. Yeh , Hector Garcia-Molina , Andreas Paepcke, Leveraging context to resolve identity in photo albums, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065430]
• 23: D. Niyogi , S. N. Srihari, Knowledge-based derivation of document logical structure, Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 1), p.472, August 14-15, 1995
• 24: Byung-Won On , Dongwon Lee , Jaewoo Kang , Prasenjit Mitra, Comparative study of name disambiguation problem using a scalable blocking-based framework, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065463]
• 25: Arnold W. M. Smeulders , Marcel Worring , Simone Santini , Amarnath Gupta , Ramesh Jain, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1349-1380, December 2000 [doi>10.1109/34.895972]
• 26: Emanuele Trucco , Alessandro Verri, Introductory Techniques for 3-D Computer Vision, Prentice Hall PTR, Upper Saddle River, NJ, 1998
• 27: Wei-Ho Tsai , Hsin-Min Wang, On the extraction of vocal-related information to facilitate the management of popular music collections, Proceedings of the 5th ACM/IEEE-CS joint conference on Digital libraries, June 07-11, 2005, Denver, CO, USA [doi>10.1145/1065385.1065432]
• 28: Martin Vetterli , Jelena Kovačevic, Wavelets and subband coding, Prentice-Hall, Inc., Upper Saddle River, NJ, 1995
• 29: James Z. Wang , Jia Li , Gio Wiederhold, SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.9, p.947-963, September 2001 [doi>10.1109/34.955109]
• 30: Yefeng Zheng , Huiping Li , David Doermann, A Parallel-Line Detection Algorithm Based on HMM Decoding, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.5, p.777-792, May 2005 [doi>10.1109/TPAMI.2005.89]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Design

Keywords:
documents, feature extraction, figures, machine learning, scientific literature