AllInOneNews: development and evaluation of a large-scale news metasearch engine

ABSTRACT

AllInOneNews is the largest news metasearch engine in the world, connecting to over 1,000 news sites over 150 countries. Implementing a large-scale metasearch engine like AllInOneNews needs to overcome unique challenges not faced by building small metasearch engines such as developing highly scalable search engine selection techniques. In this paper, we discuss these unique challenges and our solutions to these challenges. We also discuss some novel features of AllInOneNews such as highly automated solution and semantic query match. This paper also reports the results of a comparative evaluation of three commercial news search systems, one search engine - Google News and two metasearch engines - Mamma News and AllInOneNews. Several measures such as effectiveness, diversity and time-sensitivity are used to perform the comparison. Another contribution of this paper is that we introduce a novel scheme to compare multiple news search systems in a combined measure that takes both relevance and time-sensitivity of retrieved information into consideration.

REFERENCES

• 1: C. Baumgarten. A probabilistic solutions to the selection and fusion problem in distributed information retrieval. ACM SIGIR Conference, 1999.
• 2: M. Bergman. The Deep Web: Surfacing Hidden Value. White Paper of CompletePlanet at http://brightplanet.com/pdf/deepwebwhitepaper.pdf, 2001.
• 3: L. Barbosa, J. Freire. Searching for hidden-web databases. 8th International Workshop on WebDB, 2005.
• 4: James P. Callan , Zhihong Lu , W. Bruce Croft, Searching distributed collections with inference networks, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.21-28, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215328]
• 5: Jared Cope , Nick Craswell , David Hawking, Automated discovery of search interfaces on the web, Proceedings of the 14th Australasian database conference, p.181-189, February 01, 2003, Adelaide, Australia
• 6: Daniel Dreilinger , Adele E. Howe, Experiences with selecting search engines using metasearch, ACM Transactions on Information Systems (TOIS), v.15 n.3, p.195-222, July 1997 [doi>10.1145/256163.256164]
• 7: Y. Fan, and S. Gauch. Adaptive Agents for Information Gathering from Multiple, Distributed Information Sources. 1999 AAAI Symposium on Intelligent Agents in Cyberspace, Stanford University, March 1999.
• 8: S. Gauch, G. Wang, and M. Gomez. ProFusion: Intelligent fusion from multiple, distributed search engines. Journal of Universal Computer Science, 1996.
• 9: Luis Gravano , Hector Garcia-Molina, Generalizing GlOSS to Vector-Space Databases and Broker Hierarchies, Proceedings of the 21th International Conference on Very Large Data Bases, p.78-89, September 11-15, 1995
• 10: Luis Gravano , Hector Garcia-Molina, Merging Ranks from Heterogeneous Internet Sources, Proceedings of the 23rd International Conference on Very Large Data Bases, p.196-205, August 25-29, 1997
• 11: D. Hawking, N. Craswell, and K. Griffiths. Which search engine is best at finding online services? WWW conference, poster, 2001.
• 12: David Hawking , Nick Craswell , Peter Bailey , Kathleen Griffihs, Measuring Search Engine Quality, Information Retrieval, v.4 n.1, p.33-59, April 2001 [doi>10.1023/A:1011468107287]
• 13: K. L. Liu, C. Yu, W. Meng, W. Wu, and N. Rishe. A Statistical Method for Estimating the Usefulness of Text Databases. IEEE TKDE, 2002.
• 14: Y. Lu, W. Meng, L. Shu, C. Yu, and K. L. Liu. Evaluation of Result Merging Strategies for Metasearch Engines. WISE Conference, pp.53--66, November 2005.
• 15: Y. Lu, W. Meng, W. Zhang, K. L. Liu, and C. Yu. Automatic Extraction of Publication Time from News Search Results. Int'l Workshop on Challenges in Web Information Retrieval and Integration (WIRI2006), April 2006.
• 16: Udi Manber , Peter A. Bigot, The search broker, Proceedings of the USENIX Symposium on Internet Technologies and Systems on USENIX Symposium on Internet Technologies and Systems, p.21-21, December 08-11, 1997, Monterey, California
• 17: W. Meng, K. L. Liu, C. Yu, X. Wang, Y. Chang and N. Rishe. Determining Text Databases to Search in the Internet. VLDB, 1998.
• 18: Weiyi Meng , Zonghuan Wu , Clement Yu , Zhuogang Li, A highly scalable and effective method for metasearch, ACM Transactions on Information Systems (TOIS), v.19 n.3, p.310-335, July 2001 [doi>10.1145/502115.502120]
• 19: Weiyi Meng , Clement Yu , King-Lup Liu, Building efficient and effective metasearch engines, ACM Computing Surveys (CSUR), v.34 n.1, p.48-89, March 2002 [doi>10.1145/505282.505284]
• 20: Yves Rasolofo , David Hawking , Jacques Savoy, Result merging strategies for a current news metasearcher, Information Processing and Management: an International Journal, v.39 n.4, p.581-609, July 2003 [doi>10.1016/S0306-4573(02)00122-X]
• 21: Zonghuan Wu , Weiyi Meng , Clement Yu , Zhuogang Li, Towards a highly-scalable and effective metasearch engine, Proceedings of the 10th international conference on World Wide Web, p.386-395, May 01-05, 2001, Hong Kong, Hong Kong [doi>10.1145/371920.372093]
• 22: Zonghuan Wu , Vijay Raghavan , Hua Qian , Vuyyuru Rama , Weiyi Meng , Hai He , Clement Yu, Towards Automatic Incorporation of Search Engines into a Large-Scale Metasearch Engine, Proceedings of the 2003 IEEE/WIC International Conference on Web Intelligence, p.658, October 13-17, 2003
• 23: C. Yu, W. Meng, K.L. Liu, W. Wu and N. Rishe. Efficient and Effective Metasearch for a Large Number of Text Databases. ACM CIKM, November 1999.
• 24: Clement Yu , King-Lup Liu , Weiyi Meng , Zonghuan Wu , Naphtali Rishe, A Methodology to Retrieve Text Documents from Multiple Databases, IEEE Transactions on Knowledge and Data Engineering, v.14 n.6, p.1347-1361, November 2002 [doi>10.1109/TKDE.2002.1047772]
• 25: C. Yu, and W. Meng. Web Search Technology. In The Internet Encyclopedia edited by Hossein Bidgoli, Wiley Publishers, pp.738--753, 2003.
• 26: B. Yuwono, and D. Lee. Server Ranking for Distributed Text Resource Systems on the Internet. DASFAA, 1997, pp.391--400.
• 27: Hongkun Zhao , Weiyi Meng , Zonghuan Wu , Vijay Raghavan , Clement Yu, Fully automatic wrapper generation for search engines, Proceedings of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan [doi>10.1145/1060745.1060760]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Selection process; Search process
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)
      H.3.5 On-line Information Services
          Subjects: Web-based services

General Terms:
Algorithms, Design, Measurement, Performance

Keywords:
metasearch engine, news search, search engine, time-sensitive ranking