Classifying news stories using memory based reasoning

ABSTRACT

We describe a method for classifying news stories using Memory Based Reasoning (MBR) a k-nearest neighbor method), that does not require manual topic definitions. Using an already coded training database of about 50,000 stories from the Dow Jones Press Release News Wire, and SEEKER [Stanfill] (a text retrieval system that supports relevance feedback) as the underlying match engine, codes are assigned to new, unseen stories with a recall of about 80% and precision of about 70%. There are about 350 different codes to be assigned. Using a massively parallel supercomputer, we leverage the information already contained in the thousands of coded stories and are able to code a story in about 2 seconds. Given SEEKER, the text retrieval system, we achieved these results in about two person-months. We believe this approach is effective in reducing the development time to implement classification systems involving large number of topics for the purpose of classification, message routing etc.

REFERENCES

• 1: Biebricher, Peter; Fuhr, Norbert et al, "The Automatic Indexing System AIR/PHYS -- From Research to Application." Internal report, TH Darmstadt, Department of Computer Science, Darmstadt, Germany.
• 2: Robert H. Creecy , Brij M. Masand , Stephen J. Smith , David L. Waltz, Trading MIPS and memory for knowledge engineering, Communications of the ACM, v.35 n.8, p.48-64, Aug. 1992 [doi>10.1145/135226.135228]
• 3: Dasrathy B. V. Nearest Neighbor (NN) Norms: NN Pattern Classification Techniques. IEEE Computer Society Press, Los Alamitos, California (1991).
• 4: Marc Goodman, Prism: A Case-Based Telex Classifier, Proceedings of the The Second Conference on Innovative Applications of Artificial Intelligence, p.25-38, May 01-03, 1990
• 5: Philip J. Hayes , Steven P. Weinstein, CONSTRUE/TIS: A System for Content-Based Indexing of a Database of News Stories, Proceedings of the The Second Conference on Innovative Applications of Artificial Intelligence, p.49-64, May 01-03, 1990
• 6: P. J. Hayes , P. M. Andersen , I. B. Nirenburg , L. M. Schmandt, TCS: a shell for content-based text categorization, Proceedings of the sixth conference on Artificial intelligence applications, p.320-326, January 1990, Santa Barbara, California, United States
• 7: P. S. Jacobs , Lisa F. Rau, SCISOR: extracting information from on-line news, Communications of the ACM, v.33 n.11, p.88-97, Nov. 1990 [doi>10.1145/92755.92769]
• 8: Lewis, David D., "An Evaluation of Phrasal and Clustered Representation on a Text Categorization Task." University of Chicago, personal communication, manuscript in progress.
• 9: Lisa F. Rau , Paul S. Jacobs, Creating segmented databases from free text for text retrieval, Proceedings of the 14th annual international ACM SIGIR conference on Research and development in information retrieval, p.337-346, October 13-16, 1991, Chicago, Illinois, United States [doi>10.1145/122860.122894]
• 10: Craig Stanfill , Brewster Kahle, Parallel free-text search on the connection machine system, Communications of the ACM, v.29 n.12, p.1229-1239, Dec. 1986 [doi>10.1145/7902.7907]
• 11: Stanfill, C. and Waltz, D. L. "The Memory-Based Reasoning Paradigm?' Proc. Case-Based Reasoning Workshop, Clearwater Beach, FL (May 1988), pp. 414-424.
• 12: Craig Stanfill , David Waltz, Toward memory-based reasoning, Communications of the ACM, v.29 n.12, p.1213-1228, Dec. 1986 [doi>10.1145/7902.7906]
• 13: Young, Sheryl R., Hayes, Philip J., "Automatic Classification and Summarization of Banking Telexes." Proceedings of the Second IEEE Conference on AI Applications, 1985, Miami Beach, FL.
• 14: Waltz, D. L. "Memory-Based Reasoning." In M.A. Arbib and J.A. Robinson (eds), Natural and Artificial Parallel Computation, The MIT Press, Cambridge, Mass., (1990), pp. 251-276.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Text analysis
  I.7 DOCUMENT AND TEXT PROCESSING

General Terms:
Design, Performance