Site-based dynamic pruning for query processing in search engines

ABSTRACT

Web search engines typically index and retrieve at the page level. In this study, we investigate a dynamic pruning strategy that allows the query processor to first determine the most promising websites and then proceed with the similarity computations for those pages only within these sites.

REFERENCES

• 1: Ismail Sengor Altingovde , Engin Demir , Fazli Can , Özgür Ulusoy, Incremental cluster-based retrieval using compressed cluster-skipping inverted files, ACM Transactions on Information Systems (TOIS), v.26 n.3, p.1-36, June 2008 [doi>10.1145/1361684.1361688]
• 2: Vo Ngoc Anh , Alistair Moffat, Simplified similarity scoring using term ranks, Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, August 15-19, 2005, Salvador, Brazil [doi>10.1145/1076034.1076075]
• 3: Fazli Can , Ismail Sengör Altingövde , Engin Demir, Efficiency and effectiveness of query processing in cluster-based retrieval, Information Systems, v.29 n.8, p.697-717, December 2004 [doi>10.1016/S0306-4379(03)00062-0]
• 4: David Carmel , Doron Cohen , Ronald Fagin , Eitan Farchi , Michael Herscovici , Yoelle S. Maarek , Aya Soffer, Static index pruning for information retrieval systems, Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, p.43-50, September 2001, New Orleans, Louisiana, United States [doi>10.1145/383952.383958]
• 5: Wen-Syan Li , K. Selçuk Candan , Quoc Vu , Divyakant Agrawal, Retrieving and organizing web pages by “information unit”, Proceedings of the 10th international conference on World Wide Web, p.230-244, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372057]
• 6: Alistair Moffat , Justin Zobel, Self-indexing inverted files for fast text retrieval, ACM Transactions on Information Systems (TOIS), v.14 n.4, p.349-379, Oct. 1996 [doi>10.1145/237496.237497]
• 7: Silvestri, F. Sorting out the document identifier assignment problem. In ECIR'07, 101--112, 2007.
• 8: Trevor Strohman , W. Bruce Croft, Efficient document retrieval in main memory, Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval, July 23-27, 2007, Amsterdam, The Netherlands [doi>10.1145/1277741.1277774]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Clustering

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process; Query formulation

General Terms:
Experimentation, Performance

Keywords:
cluster-skipping, dynamic query pruning, inverted index