Open source search: a data mining platform

ABSTRACT

Commercial search engines provide a quality service at no cost to consumers thanks to embedded targeted marketing. Despite this, I argue there are still reasons why an open source effort should be encouraged in the community: as part of broader open publishing initiatives, to allow quality subject specific search engines to develop, and "because we can," because search is one of the great grand challenge problems and we, the research community, cannot join in without an accessible, non-proprietary system. This talk outlines arguments and discusses some of the new technology that could go into such a system, as well as the infrastructure that would be required to make it work.

REFERENCES

• 1: K. Aberer, F. Klemm, M. Rajman, and J. Wu. An architecture for peer-to-peer information retrieval. In SIGIR workshop on Peer-to-Peer Information Retrieval, 2004.
• 2: Giuseppe Attardi , Antonio Cisternino, Reflection Support by Means of Template Metaprogramming, Proceedings of the Third International Conference on Generative and Component-Based Software Engineering, p.118-127, September 09-13, 2001
• 3: Luiz André Barroso , Jeffrey Dean , Urs Hölzle, Web Search for a Planet: The Google Cluster Architecture, IEEE Micro, v.23 n.2, p.22-28, March 2003 [doi>10.1109/MM.2003.1196112]
• 4: Jamie Callan , Norbert Fuhr, The SIGIR peer-to-peer information retrieval workshop, ACM SIGIR Forum, v.38 n.2, December 2004 [doi>10.1145/1041394.1041402]
• 5: O. Drori. How to display search results in digital libraries-user study. In NDDL 2003, pages 13--28, 2003.
• 6: Jie Lu , Jamie Callan, Content-based retrieval in hybrid peer-to-peer networks, Proceedings of the twelfth international conference on Information and knowledge management, November 03-08, 2003, New Orleans, LA, USA [doi>10.1145/956863.956903]
• 7: A. K. McCallum. Mallet: A machine learning for language toolkit. http://mallet.cs.umass.edu, 2002.
• 8: C. Nedellec. Ontologies and information extraction. In S. Staab and R. Studer, editors, Handbook on Ontologies in Information Systems. Springer Verlag, 2004.
• 9: W. Nejdl. How to build Google2Google - an (incomplete) recipe. In International Semantic Web Conference, pages 1--5, 2004.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.8 Database applications
          Subjects: Data mining

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process
      H.3.4 Systems and Software

          Nouns: World Wide Web (WWW)

General Terms:
Algorithms, Performance