GlOSS: text-source discovery over the Internet

ABSTRACT

The dramatic growth of the Internet has created a new problem for users: location of the relevant sources of documents. This article presents a framework for (and experimentally analyzes a solution to) this problem, which we call the text-source discovery problem. Our approach consists of two phases. First, each text source exports its contents to a centralized service. Second, users present queries to the service, which returns an ordered list of promising text sources. This article describes GlOSS, Glossary of Servers Server, with two versions: bGlOSS, which provides a Boolean query retrieval model, and vGlOSS, which provides a vector-space retrieval model. We also present hGlOSS, which provides a decentralized version of the system. We extensively describe the methodology for measuring the retrieval effectiveness of these systems and provide experimental evidence, based on actual data, that all three systems are highly effective in determining promising text sources for a given query.

REFERENCES

• 1: BARBAR , D. AND CLIFTON, C. 1992. Information brokers: Sharing knowledge in a heterogeneous distributed system. Tech. Rep. MITL-TR-31-92. Matsushita Information Technology Laboratory.
• 2: BOWMAN, C. M., DANZIG, P. B., HARDY, D. R., MANBER, U., AND SCHWARTZ, M. F. 1994. Harvest: A scalable, customizable discovery and access system. Tech. Rep. CU-CS-732-94. Dept. Computer Science, Univ. of Colorado, Boulder.
• 3: James P. Callan , Zhihong Lu , W. Bruce Croft, Searching distributed collections with inference networks, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.21-28, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215328]
• 4: CHAMIS, A.Y. 1988. Selection of online databases using switching vocabularies. J. Am. Soc. Inf. Sci. 39, 3, 217-218.
• 5: Kevin Chen-Chuan Chang , Hector Garcia-Molina , Andreas Paepcke, Boolean Query Mapping Across Heterogeneous Information Sources, IEEE Transactions on Knowledge and Data Engineering, v.8 n.4, p.515-521, August 1996 [doi>10.1109/69.536244]
• 6: Peter B. Danzig , Jongsuk Ahn , John Noll , Katia Obraczka, Distributed indexing: a scalable mechanism for distributed information retrieval, Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, p.220-229, October 13-16, 1991, Chicago, Illinois, United States [doi>10.1145/122860.122883]
• 7: DANZIG, P. B., LI, S. -H., AND OBRACZKA, K. 1992. Distributed indexing of autonomous internet services. Comput. Syst. 5, 4, 433-459.
• 8: R. Dolin , D. Agrawal , L. Dillon , A. E Abbadi, Pharos: A Scalable Distributed Architecture for Locating Heterogeneous Information Sources, University of California at Santa Barbara, Santa Barbara, CA, 1996
• 9: DUDA, A. AND SHELDON, M.A. 1994. Content routing in a network of WAIS servers. In Proceedings of the 14th IEEE International Conference on Distributed Computing Systems (Poznan, Poland, June). IEEE Computer Society Press, Los Alamitos, CA.
• 10: FLATER, D. W. AND YESHA, Y. 1993. An information retrieval system for network resources. In Proceedings of the International Workshop on Next Generation Information Technologies and Systems (June).
• 11: James C. French , Allison L. Powell , Charles L. Viles , Travis Emmitt , Kevin J. Prey, Evaluating database selection techniques: a testbed and experiment, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.121-129, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.290976]
• 12: FULLTON, J. AND WARNOCK, A. ET AL. 1993. Release Notes for Free WAIS 0.2.
• 13: Luis Gravano , Chen-Chuan K. Chang , Héctor García-Molina , Andreas Paepcke, STARTS: Stanford proposal for Internet meta-searching, Proceedings of the 1997 ACM SIGMOD international conference on Management of data, p.207-218, May 11-15, 1997, Tucson, Arizona, United States
• 14: Luis Gravano , Hector Garcia-Molina, Generalizing GlOSS to Vector-Space Databases and Broker Hierarchies, Proceedings of the 21th International Conference on Very Large Data Bases, p.78-89, September 11-15, 1995
• 15: Luis Gravano , Hector Garcia-Molina, Generalizing GlOSS to Vector-Space Databases and Broker Hierarchies, Stanford University, Stanford, CA, 1995
• 16: Luis Gravano , Hector Garcia-Molina, Merging Ranks from Heterogeneous Internet Sources, Proceedings of the 23rd International Conference on Very Large Data Bases, p.196-205, August 25-29, 1997
• 17: Luis Gravano , Hector Garcia-Molina , Anthony Tomasic, The Efficacy of GlOSS for the Text Database Discovery Problem, Stanford University, Stanford, CA, 1993
• 18: Luis Gravano , Héctor García-Molina , Anthony Tomasic, The effectiveness of GIOSS for the text database discovery problem, Proceedings of the 1994 ACM SIGMOD international conference on Management of data, p.126-137, May 24-27, 1994, Minneapolis, Minnesota, United States
• 19: Luis Gravano , Héctor García-Molina , Anthony Tomasic, Precision and recall of GIOSS estimators for database discovery, Proceedings of the third international conference on on Parallel and distributed information systems, p.103-107, October 1994, Autin, Texas, United States
• 20: Brewster Kahle , Art Medlar, An information system for corporate users: wide area information servers, Online, v.15 n.5, p.56-60, Sept. 1991
• 21: MORRIS, A., DRENTH, H., AND TSENG, G. 1993. The development of an expert system for online company database selection. Expert Syst. 10, 2 (May), 47-60.
• 22: Anne Morris , Gwyneth Tseng , Hilary Drenth, Expert systems for online business database selection: the problem of choosing online business sources, Library Hi Tech, v.10 n.1-2, p.65-68, 1992 [doi>10.1108/eb047845]
• 23: NEUMAN, B. C. 1992. The Prospero file system: A global file system based on the virtual system model. Comput. Syst. 5, 4, 407-432.
• 24: Katia Obraczka , Peter B. Danzig , Shih-Hao Li, Internet Resource Discovery Services, Computer, v.26 n.9, p.8-22, September 1993 [doi>10.1109/2.231277]
• 25: ORDILLE, J. J. AND MILLER, B. P. 1992. Distributed active catalogs and meta-data caching in descriptive name services. Tech. Rep. 1118. University of Wisconsin at Madison, Madison, WI.
• 26: Gerard Salton, Automatic text processing: the transformation, analysis, and retrieval of information by computer, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1989
• 27: Gerard Salton , Edward A. Fox , Ellen M. Voorhees, A Comparison of Two Methods for Boolean Query Relevance Feedback, Cornell University, Ithaca, NY, 1983
• 28: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 29: SCHWARTZ, M. F. 1990. A scalable, non-hierarchical resource discovery mechanism based on probabilistic protocols. Tech. Rep. CU-CS-474-90. Department of Computer Science, University of Colorado at Boulder, Boulder, CO.
• 30: Michael F. Schwartz, Internet Resource Discovery at the University of Colorado, Computer, v.26 n.9, p.25-35, September 1993 [doi>10.1109/2.231273]
• 31: SCHWARTZ, M. F., EMTAGE, A., KAHLE, B., AND NEUMAN, B. C. 1992. A comparison of Internet resource discovery approaches. Comput. Syst. 5, 4, 461-493.
• 32: SELBERG, E. AND ETZIONI, O. 1995. Multi-service search and comparison using the MetaCrawler. In Proceedings of the Fourth International Conference on World-Wide Web (Dec.).
• 33: Mark A. Sheldon , Andrzej Duda , Ron Weiss , James W. O'Toole, Jr. , David K. Gifford, Content routing for distributed information servers, Proceedings of the 4th international conference on extending database technology on Advances in database technology, p.109-122, May 1994, Cambridge, United Kingdom
• 34: SIMPSON, P. AND ALONSO, R. 1989. Querying a network of autonomous databases. Tech. Rep. CS-TR-202-89. Department of Computer Science, Princeton Univ., Princeton, NJ.
• 35: Anthony Tomasic , Luis Gravano , Calvin Lue , Peter Schwarz , Laura Haas, Data structures for efficient broker implementation, ACM Transactions on Information Systems (TOIS), v.15 n.3, p.223-253, July 1997 [doi>10.1145/256163.256165]
• 36: VOORHEES, E. M., GUPTA, N. K., AND JOHNSON-LAIRD, B. 1995. The collection fusion problem. In Proceedings of the Third Conference on Text Retrieval (TREC-3, Mar.).
• 37: YAN, T. W. AND GARC A-MOLINA, H. 1995. SIFT--a tool for wide-area information dissemination. In Proceedings of the 1995 USENIX Technical Conference (Jan.). USENIX Assoc., Berkeley, CA, 177-186.
• 38: ZAHIR, S. AND CHANG, C. L. 1992. Online-Expert: An expert system for online database selection. J. Am. Soc. Inf. Sci. 43, 5, 340-357.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Textual databases
  H.3 INFORMATION STORAGE AND RETRIEVAL

General Terms:
Measurement, Performance

Keywords:
Internet search and retrieval, digital libraries, distributed information retrieval, text databases