Evaluation of an expert system for searching in full text

ABSTRACT

This paper presents a prototype expert system which provides online search assistance. The expert system automatically reformulates queries, using an online thesaurus as the source of domain knowledge, and a knowledge base of domain-independent search tactics. The expert system works with a full-text database which requires no syntactic or semantic pre-processing. In addition, the expert system ranks the retrieved passages in decreasing order of probable relevance. Users' search performance using the expert system was compared with their search performance on their own, and their search performance using the online thesaurus. The following conclusions were reached: 1) The expert system significantly reduced the number of queries necessary to find relevant passages compared with the user searching alone or with the thesaurus. 2) The expert system produced marginally significant improvements in precision compared with the user searching on their own. There was no significant difference in the recall achieved by the three system configurations. 3) Overall, the expert system ranked relevant passages above irrelevant passages.

REFERENCES

• 1: Bates, Marcia J., "Information Search Tactics", Journal of the ASIS, Vol. 30, No. 4, 1979. pp. 205-214.
• 2: David C. Blair , M. E. Maron, An evaluation of retrieval effectiveness for a full-text document-retrieval system, Communications of the ACM, v.28 n.3, p.289-299, March 1985 [doi>10.1145/3166.3197]
• 3: Blaauw G.A. and Brooks, Frederick P. Jr., Computer Architecture, Volume 1-Design Decisions, Draft, Spring 1987.
• 4: Borgman, Christine L., "Why Are Online Catalogs Hard to Use?", Journal of the ASIS, Vol. 37, No. 6, November 1986. pp. 387-400.
• 5: C. Borgman, Why do some people have more difficulty learning to use an information retrieval system than others?, Proceedings of the 10th annual international ACM SIGIR conference on Research and development in information retrieval, p.61-71, June 03-05, 1987, New Orleans, Louisiana, United States [doi>10.1145/42005.42014]
• 6: G. Brajnik , G. Guida , C. Tasso, IR-NLI II: applying man-machine interaction and artificial intelligence conceptsto information retrieval, Proceedings of the 11th annual international ACM SIGIR conference on Research and development in information retrieval, p.387-399, May 1988, Grenoble, France [doi>10.1145/62437.62479]
• 7: Y. Chiaramella , B. Defude, A prototype of an intelligent system for information retrieval: IOTA, Information Processing and Management: an International Journal, v.23 n.4, p.285-303, July 1987 [doi>10.1016/0306-4573(87)90019-7]
• 8: W. B. Croft , R. H. Thompson, I3R: a new approach to the design of document retrieval systems, Journal of the American Society for Information Science, v.38 n.6, p.389-404, Nov. 1987 [doi>10.1002/(SICI)1097-4571(198711)38:6<389::AID-ASI1>3.0.CO;2-4]
• 9: Fenichel, C. H., "Online Searching: Measures that Discriminate Among Users with Different Types of Experience", Journal of the ASIS, Vol. 32, No.: I, January 1981, pp. 23-32.
• 10: Fox, Edward A., Weaver, Marybeth T., Chen, Qi-Fan, and France, Robert K.,"Implementing a Distributed Expert-Based Information Retrieval System", Proceedings of RIAO 88, M.I.T., March 1988. pp. 708-726.
• 11: Susan Evalyn Gauch , John B. Smith, An expert system for searching in full-text, 1990
• 12: S. Gauch , J. B. Smith, An expert system for searching in full-text, Information Processing and Management: an International Journal, v.25 n.3, p.253-263, May 1989 [doi>10.1016/0306-4573(89)90043-5]
• 13: Gauch, Susan, and Smith, John B.,"Query Reformulation Strategies for an Intelligent Search Intermediary", Proceedings of the Annual AI Systems in Government Conference, Washington, D.C., March 1989.
• 14: Oldroyd, B. K., "Study of strategies used in online searching 5" differences between the experienced and the inexperienced searcher", Online Review, Vol. 8, No. 3, 1984. pp. 233- 244.
• 15: Pollitt, A.S., "A "Front-End' System: An Expert System as an Online Search Intermediary", ASLIB Proceedings, Vol. 36, No. 5, May 1984. pp. 229-234.
• 16: Peretz Shoval, Principles, procedures and rules in an expert system for information retrieval, Information Processing and Management: an International Journal, v.21 n.6, p.475-487, Dec. 1985 [doi>10.1016/0306-4573(85)90002-0]
• 17: J. Smith , S. Weiss , G. Ferguson, An advanced full-text retrieval and analysis system, Proceedings of the 10th annual international ACM SIGIR conference on Research and development in information retrieval, p.187-195, June 03-05, 1987, New Orleans, Louisiana, United States [doi>10.1145/42005.42027]
• 18: P. J. Smith , S. J. Shute , D. Galdes, In search of knowledge-based search tactics, Proceedings of the 12th annual international ACM SIGIR conference on Research and development in information retrieval, p.3-10, June 25-28, 1989, Cambridge, Massachusetts, United States
• 19: Teskey, Niall, "Extensions to the Advanced Interface Management Project", OCLC Research Review, July I987. pp. 1-3.
• 20: R. Tong , L. Appelbaum , V. Askman , J. Cunningham, Conceptual information retrieval using RUBRIC, Proceedings of the 10th annual international ACM SIGIR conference on Research and development in information retrieval, p.247-253, June 03-05, 1987, New Orleans, Louisiana, United States [doi>10.1145/42005.42033]
• 21: Vernimb, Carlo, "Automatic Query Adjustment in Document Retrieval", Information Processing & Management, Vol. 13, No. 6, 1977. pp. 339-353.
• 22: A. Vickery , H. M. Brooks, PLEXUS-The expert system for referral, Information Processing and Management: an International Journal, v.23 n.2, p.99-117, March 1987 [doi>10.1016/0306-4573(87)90052-5]
• 23: Williams, P.W., "A Model for an Expert System for Automated Information Retrieval", Proceedings of the 8th International Online Meeting, 1984. pp. 139-149.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.1 Applications and Expert Systems
          Subjects: Office automation

Additional Classification:
  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: C

  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Thesauruses
      H.3.3 Information Search and Retrieval
          Subjects: Query formulation; Search process

General Terms:
Design, Languages