Document filtering method using non-relevant information profile

ABSTRACT

Document filtering is a task to retrieve documents relevant to a user's profile from a flow of documents. Generally, filtering systems calculate the similarity between the profile and each incoming document, and retrieve documents with similarity higher than a threshold. However, many systems set a relatively high threshold to reduce retrieval of non-relevant documents, which results in the ignorance of many relevant documents. In this paper, we propose the use of a non-relevant information profile to reduce the mistaken retrieval of non-relevant documents. Results from experiments show that this filter has successfully rejected a sufficient number of non-relevant documents, resulting in an improvement of filtering performance.

REFERENCES

• 1: K Hoashi, K Matsumoto, N Inoue, K Hashimoto: "Experiments on the TREC-8 Filtering Track", (to be published in The 8th Text REtrieval Conference"), 2000.
• 2: Keiichiro Hoashi , Kazunori Matsumoto , Naomi Inoue , Kazuo Hashimoto, Query expansion method based on word contribution (poster abstract), Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.303-304, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312733]
• 3: D Hull: "The TREC-7 Filtering Track: Description and Analysis", The 7th Text REtrieval Conference, NIST SP 500-242, pp 33-56, 1999.
• 4: S Robertson, S Walker, S Jones, M Hancock- Beaulieu, and M Gatford, "Okapi at TREC-3", Overview of the Third Text REtrieval Conference, pp 109-125, 1994.
• 5: J Rocchio: "Relevance Feedback in Information Retrieval", in "The SMART Retrieval System - Experiments in Automatic Document Processing", Prentice Hall Inc., pp 313-323, 1971.
• 6: A Singhal, J Choi, D Hindle, D Lewis, and F Pereira: "AT&T at TREC-7", The Seventh Text REtrieval Conference, NIST SP 500-242, pp 239- 251, 1999.
• 7: E Voorhees, D Harman: "The 8th Text REtrieval Conference", (to be published), 2000.
• 8: C Zhai, P Jansen, N Roma, E Stoica, D Evans "Notes on Optimization in CLARIT Adaptive Filtering", (to be published in the 8th text REtrieval Conference"), 2000.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Information filtering

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Relevance feedback
      H.3.4 Systems and Software
          Subjects: User profiles and alert services

General Terms:
Algorithms, Experimentation, Human Factors