Enhanced web document summarization using hyperlinks

ABSTRACT

This paper addresses the issue of Web document summarization. As textual content of Web documents is often scarce or irrelevant and existing summarization techniques are based on it, many Web pages and websites cannot be suitably summarized. We consider the context of a Web document by the textual content of all the documents linking to it. To summarize a target Web document, a context-based summarizer has to perform a preprocessing task, during which it will be decided which pieces of information in the source documents are relevant to the content of the target. Then a context-based summarizer faces two issues: first, the selected elements may partially deal with the topic of the target, second they may be related to the target and yet not contain any clues about the content of the target.In this paper we put forward two new summarization by context algorithms. The first one uses both the content and the context of the document and the second one is based only on the elements of the context. It is shown that summaries taking into account the context are usually much more relevant than those made only from the content of the target document. Optimal conditions of the proposed algorithms with respect to the sizes of the content and the context of the document to summarize are studied.

REFERENCES

• 1: Einat Amitay , Cécile Paris, Automatically summarising Web sites: is there a way around it?, Proceedings of the ninth international conference on Information and knowledge management, p.173-179, November 06-11, 2000, McLean, Virginia, United States [doi>10.1145/354756.354816]
• 2: G. Attardi, M. S. Di, and D. Salvi. Categorisation by Context. J.UCS: Journal of Universal Computer Science, 4(9):719--736, 1998.
• 3: Adam L. Berger , Vibhu O. Mittal, OCELOT: a system for summarizing Web pages, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.144-151, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345565]
• 4: Bernadette Bouchon-Meunier , Maria Rifqi , Sylvie Bothorel, Towards general measures of comparison of objects, Fuzzy Sets and Systems, v.84 n.2, p.143-153, Dec. 9, 1996 [doi>10.1016/0165-0114(96)00067-X]
• 5: Orkut Buyukkokten , Hector Garcia-Molina , Andreas Paepcke, Seeing the whole in parts: text summarization for web browsing on handheld devices, Proceedings of the 10th international conference on World Wide Web, p.652-662, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372178]
• 6: Soumen Chakrabarti , Byron Dom , Prabhakar Raghavan , Sridhar Rajagopalan , David Gibson , Jon Kleinberg, Automatic resource compilation by analyzing hyperlink structure and associated text, Proceedings of the seventh international conference on World Wide Web 7, p.65-74, April 1998, Brisbane, Australia
• 7: Brian D. Davison, Topical locality in the Web, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.272-279, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345597]
• 8: J. Furnkranz. Using links for classifying Web-pages. Technical report, Austrian Research Institute for Artificial Intelligence, TR-OEFAI-98-29, 1998.
• 9: Jade Goldstein , Mark Kantrowitz , Vibhu Mittal , Jaime Carbonell, Summarizing text documents: sentence selection and evaluation metrics, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.121-128, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312665]
• 10: S. Johnson. Hierarchical clustering schemes. Psychometrika, 32:241--254, 1967.
• 11: S. Jones , S. Lundy , G. Paynter, Interactive Document Summarisation Using Automatically Extracted Keyphrases, Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS'02)-Volume 4, p.101.3, January 07-10, 2002
• 12: F. Menczer. Links tell us about lexical and semantic Web content. Technical report, Computer Science, abstract CS.IR/0108004, arXiv.org, Aug 2001.
• 13: G. Miller, R. Beckwith, C. Fellbaum, D. Gross, and K. Miller. Five papers on Wordnet. Technical report, Cognitive Science Laboratory, Princeton University, 1990.
• 14: L. Page, S. Brin, R. Motwani, and T. Winograd. The PageRank Citation Ranking: Bringing Order to the Web. Technical report, Stanford Digital Library Technologies Project, 1998.
• 15: G. Salton , A. Wong , C. S. Yang, A vector space model for automatic indexing, Communications of the ACM, v.18 n.11, p.613-620, Nov. 1975 [doi>10.1145/361219.361220]
• 16: H. Schmid. Probabilistic Part-of-Speech Tagging Using Decision Trees. In International Conference on New Methods in Language Processing, pages 44--49, Manchester, UK, Sep 1994.
• 17: K. Sparck-Jones and J. Galliers. Evaluating Natural Language Processing Systems. Springer, 32:241--254, 1996.
• 18: Mary Zajicek , Chris Powell , Chris Reeves, A Web navigation tool for the blind, Proceedings of the third international ACM conference on Assistive technologies, p.204-206, April 15-17, 1998, Marina del Rey, California, United States [doi>10.1145/274497.274534]
• 19: Y. Zhang. World Wide Web Site Summarization, Master thesis. Technical report, Faculty of Computer Science, Dalhousie University, Apr 2002.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.m Miscellaneous

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis

General Terms:
Algorithms, Experimentation

Keywords:
context, hyperlinks, summarization, web document