Caching and database scaling in distributed shared-nothing information retrieval systems

ABSTRACT

A common class of existing information retrieval system provides access to abstracts. For example Stanford University, through its FOLIO system, provides access to the INSPECT database of abstracts of the literature on physics, computer science, electrical engineering, etc. In this paper this database is studied by using a trace-driven simulation. We focus on physical index design, inverted index caching, and database scaling in a distributed shared-nothing system. All three issues are shown to have a strong effect on response time and throughput. Database scaling is explored in two ways. One way assumes an “optimal” configuration for a single host and then linearly scales the database by duplicating the host architecture as needed. The second way determines the optimal number of hosts given a fixed database size.

REFERENCES

• 1: Forbes J. Burkowski, Retrieval performance of a distributed text database utilizing a parallel processor document server, Proceedings of the second international symposium on Databases in parallel and distributed systems, p.71-79, July 02-04, 1990, Dublin, Ireland [doi>10.1145/319057.319065]
• 2: Ann L. Chervenak, Performance Measurements of the First RAID Prototype, University of California at Berkeley, Berkeley, CA, 1990
• 3: J. K. Cringean , R. England , G. A. Manson , P. Willett, Parallel text searching in serial files using a processor farm, Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, p.429-453, September 05-07, 1990, Brussels, Belgium [doi>10.1145/96749.98249]
• 4: S. DeFazio and J. Hull. Toward servicing textual database transactions on symmetric shared memory multiprocessors. In Proceedings of the intern=tional Workshop on High Performance Transaction Systems, Asilomar, 1991.
• 5: P. A. Erarath. Page Indzzing Jot Teztual Inyorma#ion Retrieval Systems. PhD thesis, University of illinois at Urbane-Champaign, October 1983.
• 6: Chris Faloutsos, Access methods for text, ACM Computing Surveys (CSUR), v.17 n.1, p.49-74, March 1985 [doi>10.1145/4078.4080]
• 7: W. B. Frt#kes and R. Baeza-Yates. Information Retrieval: Data Structures and Algorithms. Prentice-Hall, 1992.
• 8: Jim Gray , Andreas Reuter, Transaction Processing: Concepts and Techniques, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1992
• 9: 1%. Jain. The Art of Computer Systems Performance Analysis. John Wiley and Sons, New York, 1991.
• 10: B.-S. Jeong and E. Omleclnskl. Inverted file partitioning schemes for a shared-everytl#ing multlprocessor. Tech#cal Report GIT-CC-92/39, Georgia Institute of Technology, College of Computing, 1992.
• 11: C. Stanfill, Partitioned posting files: a parallel inverted file structure for information retrieval, Proceedings of the 13th annual international ACM SIGIR conference on Research and development in information retrieval, p.413-428, September 05-07, 1990, Brussels, Belgium [doi>10.1145/96749.98247]
• 12: Harold S. Stone, Parallel querying of large databases: a case study, Computer, v.20 n.10, p.11-21, Oct. 1987 [doi>10.1109/MC.1987.1663384]
• 13: A. Tome#ic and H. Garcia-Mollna. Caching and database scaling in distributed shared-nothlng information retrieval systems. T~c2xrdcal Report STAN-CS-92-14#6, Stanford University, December 1992.
• 14: Anthony Tomasic , Hector Garcia-Molina, Performance of inverted indices in shared-nothing distributed text document informatioon retrieval systems, Proceedings of the second international conference on Parallel and distributed information systems, p.8-17, January 1993, San Diego, California, United States

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Distributed databases

Additional Classification:
  H. Information Systems
  H.2 DATABASE MANAGEMENT
      H.2.4 Systems
          Subjects: Query processing
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Design, Experimentation, Performance