Book search: indexing the valuable parts

ABSTRACT

With massive book digitization efforts underway, there is a need for developing effective book retrieval strategies. This paper explores the relative contribution of different parts of digitized and OCR'ed books towards effective retrieval. The examined parts include the entire content of books, book headings, book titles, and table of content entries. Results show that indexing the headers and titles of books is nearly as effective as indexing the entire contents of books. These results indicate that certain portions of the books, specifically titles and headers, are more valuable than other parts of books. This is akin to web search where hypertext and page titles are more valuable to index than the rest of the webpage. Also, using a combination of evidence approach provides further improved retrieval effectiveness compared to using any portion of the book in isolation.

REFERENCES

• 1: William Barrett , Luke Hutchison , Dallan Quass , Heath Nielson , Douglas Kennard, Digital Mountain: From Granite Archive to Global Access, Proceedings of the First International Workshop on Document Image Analysis for Libraries (DIAL'04), p.104, January 23-24, 2004
• 2: Croft, W. B., S. Harding, K. Taghva, and J. Andborsak. An evaluation of information retrieval accuracy with simulated OCR output. In Proceedings of the 3rd Annual Symposium on Document Analysis and Information Retrieval, University of Nevada, Las Vegas, Nev., 115--126, (1994).
• 3: Darwish, K. and O. Emam. The Effect of Blind Relevance Feedback on a New Arabic OCR Degraded Text Collection. In International Conference on Machine Intelligence: Special Session on Arabic Document Image Analysis, (2005).
• 4: Kareem Darwish , Douglas W. Oard, Term selection for searching printed Arabic, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564423]
• 5: David S. Doermann, The Retrieval of Document Images: A Brief Survey, Proceedings of the 4th International Conference on Document Analysis and Recognition, p.945-949, August 18-20, 1997
• 6: David Doermann, The indexing and retrieval of document images: a survey, Computer Vision and Image Understanding, v.70 n.3, p.287-298, June 1, 1998 [doi>10.1006/cviu.1998.0692]
• 7: Stephen M. Harding , W. Bruce Croft , C. Weir, Probabilistic Retrieval of OCR Degraded Text Using N-Grams, Proceedings of the First European Conference on Research and Advanced Technology for Digital Libraries, p.345-359, September 01-03, 1997
• 8: Donna Harman, Overview of the first TREC conference, Proceedings of the 16th annual international ACM SIGIR conference on Research and development in information retrieval, p.36-47, June 27-July 01, 1993, Pittsburgh, Pennsylvania, United States [doi>10.1145/160688.160692]
• 9: Hawking, D. Document Retrieval in OCR-Scanned Text. Sixth Parallel Computing Workshop, paper P2-F (1996).
• 10: Kantor, P. and E. Voorhees. Report on the TREC-5 Confusion Track. TREC-5, pp 65, (1996).
• 11: Adenike M. Lam-Adesina , Gareth J. F. Jones, Examining and improving the effectiveness of relevance feedback for retrieval of scanned text documents, Information Processing and Management: an International Journal, v.42 n.3, p.633-649, May 2006 [doi>10.1016/j.ipm.2005.06.006]
• 12: Irina Matveeva , Chris Burges , Timo Burkard , Andy Laucius , Leon Wong, High accuracy retrieval with multiple nested ranker, Proceedings of the 29th annual international ACM SIGIR conference on Research and development in information retrieval, August 06-11, 2006, Seattle, Washington, USA [doi>10.1145/1148170.1148246]
• 13: Donald Metzler , W. Bruce Croft, Combining the language model and inference network approaches to retrieval, Information Processing and Management: an International Journal, v.40 n.5, p.735-750, September 2004 [doi>10.1016/j.ipm.2004.05.001]
• 14: Mark Sanderson , Justin Zobel, Information retrieval system evaluation: effort, sensitivity, and reliability, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076064]
• 15: Steven Simske , Xiaofan Lin, Creating Digital Libraries: Content Generation and Re-Mastering, Proceedings of the First International Workshop on Document Image Analysis for Libraries (DIAL'04), p.33, January 23-24, 2004
• 16: Smith, S., An Analysis of the Effects of Data Corruption on Text Retrieval Performance. Technical Report DR90-1, Thinking Machines Corp: Cambridge, MA, (1990).
• 17: Mark D. Smucker , James Allan , Ben Carterette, A comparison of statistical significance tests for information retrieval evaluation, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal [doi>10.1145/1321440.1321528]
• 18: Song, R., J.R. Wen, S. Shi, G. Xin, T.Y. Liu, et al., Microsoft Research Asia at Web Track and Terabyte Track of TREC 2004. In 2004 Text REtrieval Conference (2004).
• 19: Taghva, K., J. Borsack, and A. Condit. An Expert System for Automatically Correcting OCR Output. Proc. IS&T/SPIE 1994 Intl. Symp. on Electronic Imaging Science and Technology, , San Jose, CA, pp 270--278 (1994a).
• 20: Taghva, K., J. Borasack, A. Condit, and J. Gilbreth. Results and Implications of the Noisy Data Projects. Technical Report 94-01, Information Science Research Institute, University of Nevada, Las Vegas, (1994b).
• 21: Taghva, K., J. Borasack, A. Condit, and P. Inaparthy. Querying Short OCR'd Documents. Technical Report 94-10, Information Science Research Institute, University of Nevada, Las Vegas, (1995).
• 22: Kazem Taghva , Julie Borsack , Allen Condit, Evaluation of model-based retrieval effectiveness with OCR text, ACM Transactions on Information Systems (TOIS), v.14 n.1, p.64-93, Jan. 1996 [doi>10.1145/214174.214180]
• 23: Kazem Taghva , Julie Borsack , Allen Condit, Effects of OCR errors on ranking and feedback using the vector space model, Information Processing and Management: an International Journal, v.32 n.3, p.317-327, May 1996 [doi>10.1016/0306-4573(95)00058-5]
• 24: Thoma, G. and G. Ford. Automated Data Entry System: Performance Issues. Proc. SPIE Conference on Document Recognition and Retrieval IX, San Jose, 2002, pp. 181--190, (2002).
• 25: Tseng, Y. and D. Oard. Document Image Retrieval Techniques for Chinese. In Symposium on Document Image Understanding Technology, Columbia, MD, pp 151--158 (2001).
• 26: Ellen M. Voorhees, Evaluation by highly relevant documents, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.74-82, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383963]
• 27: Wu, H., G. Kazai, and M. Taylor, Book Search Experiments: Investigating IR Methods for the Indexing and Retrieval of Books. ECIR 2008: 234--245 (2008).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Experimentation, Measurement, Performance

Keywords:
OCR retrieval, book search