Adaptive query routing in peer web search

ABSTRACT

An unstructured peer network application was proposed to address the query forwarding problem of distributed search engines and scalability limitations of centralized search engines. Here we present novel techniques to improve local adaptive routing, showing they perform significantly better than a simple learning scheme driven by query response interactions among neighbors. We validate prototypes of our peer network application via simulations with 500 model users based on actual Web crawls. We finally compare the quality of the results with those obtained by centralized search engines, suggesting that our application can draw advantages from the context and coverage of the peer collective.

REFERENCES

• 1: R. Akavipat , L-S. Wu , F. Menczer, Small world peer networks in distributed web search, Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, May 19-21, 2004, New York, NY, USA [doi>10.1145/1013367.1013493]
• 2: Mayank Bawa , Roberto J. Bayardo, Jr. , Sridhar Rajagopalan , Eugene J. Shekita, Make it fresh, make it quick: searching a network of personal webservers, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary [doi>10.1145/775152.775234]
• 3: Sam Joseph, NeuroGrid: Semantically Routing Queries in Peer-to-Peer Networks, Revised Papers from the NETWORKING 2002 Workshops on Web Engineering and Peer-to-Peer Computing, p.202-214, May 19-24, 2002
• 4: Filippo Menczer , Gautam Pant , Padmini Srinivasan, Topical web crawlers: Evaluating adaptive algorithms, ACM Transactions on Internet Technology (TOIT), v.4 n.4, p.378-419, November 2004 [doi>10.1145/1031114.1031117]
• 5: Chunqiang Tang , Zhichen Xu , Sandhya Dwarkadas, Peer-to-peer information retrieval using self-organizing semantic overlay networks, Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, August 25-29, 2003, Karlsruhe, Germany [doi>10.1145/863955.863976]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Information networks; Performance evaluation (efficiency and effectiveness); Distributed systems

General Terms:
Performance

Keywords:
adaptive query routing, peer collaborative search, topical crawlers