Building a distributed full-text index for the Web

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Building a distributed full-text index for the Web

Full text

Pdf (299 KB)

Source

International World Wide Web Conference archive
Proceedings of the 10th international conference on World Wide Web table of contents

Hong Kong, Hong Kong

Pages: 396 - 406

Year of Publication: 2001

ISBN:1-58113-348-0

Authors

Sergey Melnik	Computer Science Department, Stanford University, Stanford, CA
Sriram Raghavan	Computer Science Department, Stanford University, Stanford, CA
Beverly Yang	Computer Science Department, Stanford University, Stanford, CA
Hector Garcia-Molina	Computer Science Department, Stanford University, Stanford, CA

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web
SIGLINK: Hypertext, Hypermedia, and Web
IW3C2 : International World Wide Web Conference Committee

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 2, Downloads (12 Months): 55, Citation Count: 21

Additional Information:

references cited by index terms collaborative colleagues

Tools and Actions:	Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/371920.372095 What is a DOI?

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Eric W. Brown , James P. Callan , W. Bruce Croft , J. Eliot B. Moss, Supporting full-text information retrieval with a persistent object store, Proceedings of the 4th international conference on extending database technology on Advances in database technology, p.365-378, May 1994, Cambridge, United Kingdom
	2	Soumen Chakrabarti , S. Muthukrishnan, Resource scheduling for parallel database and scientific applications, Proceedings of the eighth annual ACM symposium on Parallel algorithms and architectures, p.329-335, June 24-26, 1996, Padua, Italy [doi>10.1145/237502.237577]
	3	Junghoo Cho , Hector Garcia-Molina, The Evolution of the Web and Implications for an Incremental Crawler, Proceedings of the 26th International Conference on Very Large Data Bases, p.200-209, September 10-14, 2000
	4	Chris Faloutsos , Stavros Christodoulakis, Signature files: an access method for documents and its analytical performance evaluation, ACM Transactions on Information Systems (TOIS), v.2 n.4, p.267-288, Oct. 1984 [doi>10.1145/2275.357411]
	5	Hector Garcia-Molina , Jennifer Widom , Jeffrey D. Ullman, Database System Implementation, Prentice-Hall, Inc., Upper Saddle River, NJ, 1999
	6	D. A. Gorssman and J. R. Driscoll. Structuring text within a relation system. In 3rd Intl. Conf. on Database and Expert System Applications, Sep 1992.
	7	D. Hawking and N. Craswell. Overview of TREC-7 very large collection track. In Proc. of the Seventh Text Retrieval Conf., pages 91-104, Nov 1998.
	8	Jun Hirai , Sriram Raghavan , Hector Garcia-Molina , Andreas Paepcke, WebBase: a repository of Web pages, Proceedings of the 9th international World Wide Web conference on Computer networks : the international journal of computer and telecommunications netowrking, p.277-293, June 2000, Amsterdam, The Netherlands
	9	Byeong-Soo Jeong , Edward Omiecinski, Inverted File Partitioning Schemes in Multiple Disk Systems, IEEE Transactions on Parallel and Distributed Systems, v.6 n.2, p.142-153, February 1995 [doi>10.1109/71.342125]
	10	S. Lawrence and C. L. Giles. Accessibility of information on the web. Nature, 400:107-109, 1999.
	11	Udi Manber , Gene Myers, Suffix arrays: a new method for on-line string searches, Proceedings of the first annual ACM-SIAM symposium on Discrete algorithms, p.319-327, January 22-24, 1990, San Francisco, California, United States
	12	Ian A. MacLeod , Patrick Martin , Brent Nordin, A design of a distributed full text retrieval system, Proceedings of the 9th annual international ACM SIGIR conference on Research and development in information retrieval, p.131-137, September 1986, Palazzo dei Congressi, Pisa, Italy [doi>10.1145/253168.253197]
	13	S. Melnik et al. Building a distributed full-text index for the web. Technical report, Stanford Digital Library Project, July 2000. Available at wwwdiglib.stanford.edu/cgi-bin/get/SIDL-WP-2000-0140.
	14	Mike Burrows. Personal Communication.
	15	Alistair Moffat , Justin Zobel, Self-indexing inverted files for fast text retrieval, ACM Transactions on Information Systems (TOIS), v.14 n.4, p.349-379, Oct. 1996 [doi>10.1145/237496.237497]
	16	Anh Ngoc Vo , Alistair Moffat, Compressed inverted files with reduced decoding overheads, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.290-297, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291011]
	17	M. Olson, K. Bostic, and M. Seltzer. Berkeley DB. In 1999 Summer Usenix Technical Conf., Jun 1999.
	18	Berthier A. Ribeiro-Neto , Ramurti A. Barbosa, Query performance for tightly coupled distributed digital libraries, Proceedings of the third ACM conference on Digital libraries, p.182-190, June 23-26, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276675.276695]
	19	Berthier Ribeiro-Neto , Edleno S. Moura , Marden S. Neubert , Nivio Ziviani, Efficient distributed algorithms to build inverted files, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.105-112, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312663]
	20	G. Salton. Information Retrieval: Data Structures and Algorithms. Addison-Wesley, Massachussetts, 1989.
	21	Anthony Tomasic , Hector Garcia-Molina, Query processing and inverted indices in shared: nothing text document information retrieval systems, The VLDB Journal — The International Journal on Very Large Data Bases, v.2 n.3, July 1993
	22	Anthony Tomasic , Héctor García-Molina , Kurt Shoens, Incremental updates of inverted lists for text document retrieval, Proceedings of the 1994 ACM SIGMOD international conference on Management of data, p.289-300, May 24-27, 1994, Minneapolis, Minnesota, United States
	23	Charles L. Viles , James C. French, Dissemination of collection wide information in a distributed information retrieval system, Proceedings of the 18th annual international ACM SIGIR conference on Research and development in information retrieval, p.12-20, July 09-13, 1995, Seattle, Washington, United States [doi>10.1145/215206.215327]
	24	Inktomi WebMap. www.inktomi.com/webmap/.
	25	Ian H. Witten , Alistair Moffat , Timothy C. Bell, Managing gigabytes (2nd ed.): compressing and indexing documents and images, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
	26	Justin Zobel , Alistair Moffat , Ron Sacks-Davis, An Efficient Indexing Technique for Full Text Databases, Proceedings of the 18th International Conference on Very Large Data Bases, p.352-362, August 23-27, 1992

CITED BY 21

	Arvind Arasu , Junghoo Cho , Hector Garcia-Molina , Andreas Paepcke , Sriram Raghavan, Searching the Web, ACM Transactions on Internet Technology (TOIT), v.1 n.1, p.2-43, Aug. 2001
	D. Zeinalipour-Yazti , Vana Kalogeraki , Dimitrios Gunopulos, Exploiting locality for scalable information retrieval in peer-to-peer networks, Information Systems, v.30 n.4, p.277-298, June 2005
	Kjetil Nørvåg, Supporting temporal text-containment queries in temporal document databases, Data & Knowledge Engineering, v.49 n.1, p.105-125, 1 April 2004
	Vana Kalogeraki , Dimitrios Gunopulos , D. Zeinalipour-Yazti, A local search mechanism for peer-to-peer networks, Proceedings of the eleventh international conference on Information and knowledge management, November 04-09, 2002, McLean, Virginia, USA
	Steffen Heinz , Justin Zobel, Efficient single-pass index construction for text databases, Journal of the American Society for Information Science and Technology, v.54 n.8, p.713-729, June 2003
	Lipyeow Lim , Min Wang , Sriram Padmanabhan , Jeffrey Scott Vitter , Ramesh Agarwal, Dynamic maintenance of web indexes using landmarks, Proceedings of the 12th international conference on World Wide Web, May 20-24, 2003, Budapest, Hungary
	Ronny Lempel , Shlomo Moran, Optimizing result prefetching in web search engines with segmented indices, ACM Transactions on Internet Technology (TOIT), v.4 n.1, p.31-59, February 2004
	GuangWen Yang , ShuMing Shi , DingXing Wang , QiFeng Huang , XueZheng Liu, DSI: distributed service integration for service grid, Journal of Computer Science and Technology, v.18 n.4, p.474-483, July 2003
	Xiaohui Long , Torsten Suel, Three-level caching for efficient query processing in large Web search engines, Proceedings of the 14th international conference on World Wide Web, May 10-14, 2005, Chiba, Japan
	Junghoo Cho , Hector Garcia-Molina , Taher Haveliwala , Wang Lam , Andreas Paepcke , Sriram Raghavan , Gary Wesley, Stanford WebBase components and applications, ACM Transactions on Internet Technology (TOIT), v.6 n.2, p.153-186, May 2006
	Wray Buntine , Jaakko Lofstrom , Jukka Perkio , Sami Perttu , Vladimir Poroshin , Tomi Silander , Henry Tirri , Antti Tuominen , Ville Tuulos, A Scalable Topic-Based Open Source Search Engine, Proceedings of the 2004 IEEE/WIC/ACM International Conference on Web Intelligence, p.228-234, September 20-24, 2004
	I. S. Nekrestyanov , N. V. Panteleeva, Text Retrieval Systems for the Web, Programming and Computing Software, v.28 n.4, p.207-225, July-August 2002
	Maxim Lifantsev , Tzi-cker Chiueh, Implementation of a modern web search engine cluster, Proceedings of the USENIX Annual Technical Conference 2003 on USENIX Annual Technical Conference, p.8-8, June 09-14, 2003, San Antonio, Texas
	Robert W. P. Luk , Wai Lam, Efficient in-memory extensible inverted file, Information Systems, v.32 n.5, p.733-754, July, 2007
	Marcus Fontoura , Engene Shekita , Jason Y. Zien , Sridhar Rajagopalan , Andreas Neumann, High performance index build algorithms for intranet search engines, Proceedings of the Thirtieth international conference on Very large data bases, p.1122-1133, August 31-September 03, 2004, Toronto, Canada
	Xiaohui Long , Torsten Suel, Optimized query execution in large search engines with global page ordering, Proceedings of the 29th international conference on Very large data bases, p.129-140, September 09-12, 2003, Berlin, Germany
	Ronny Lempel , Shlomo Moran, Optimizing result prefetching in web search engines with segmented indices, Proceedings of the 28th international conference on Very Large Data Bases, p.370-381, August 20-23, 2002, Hong Kong, China
	Maxim Lifantsev , Tzi-cker Chiueh, I/O-conscious data preparation for large-scale web search engines, Proceedings of the 28th international conference on Very Large Data Bases, p.382-393, August 20-23, 2002, Hong Kong, China
	Lipyeow Lim , Min Wang , Sriram Padmanabhan , Jeffrey Scott Vitter , Ramesh Agarwal, Efficient Update of Indexes for Dynamically Changing Web Documents, World Wide Web, v.10 n.1, p.37-69, March 2007
	Ronny Lempel , Yosi Mass , Shila Ofek-Koifman , Dafna Sheinwald , Yael Petruschka , Ron Sivan, Just in time indexing for up to the second search, Proceedings of the sixteenth ACM conference on Conference on information and knowledge management, November 06-10, 2007, Lisbon, Portugal
	Marcus Fontoura , Vanja Josifovski , Ravi Kumar , Christopher Olston , Andrew Tomkins , Sergei Vassilvitskii, Relaxation in text search using taxonomies, Proceedings of the VLDB Endowment, v.1 n.1, August 2008