Combining semantic and syntactic document classifiers to improve first story detection

ABSTRACT

In this paper we describe a type of data fusion involving the combination of evidence derived from multiple document representations. Our aim is to investigate if a composite representation can improve the online detection of novel events in a stream of broadcast news stories. This classification process otherwise known as first story detection FSD (or in the Topic Detection and Tracking pilot study as online new event detection [1]), is one of three main classification tasks defined by the TDT initiative. Our composite document representation consists of a semantic representation (based on the lexical chains derived from a text) and a syntactic representation (using proper nouns). Using the TDT1 evaluation methodology, we evaluate a number of document representation combinations using these document classifiers.

REFERENCES

• 1: Ron Papka, James Allan, Topic Detection and Tracking: Event Clustering as a basis for first story detection, Kluwer Academic Publishers, 4:97-126, 2000.
• 2: Jane Morris , Graeme Hirst, Lexical cohesion computed by thesaural relations as an indicator of the structure of text, Computational Linguistics, v.17 n.1, p.21-48, March 1991
• 3: Christiane Fellbaum, WordNet: An Electronic Lexical Database, MIT Press, 1998.
• 4: Nicola Stokes, Paula Hatch, Joe Carthy, Topic Detection, a new application for lexical chaining?, In the Proceedings of BCS IRSG Colloquium 2000, pp. 94-103, 2000.
• 5: W. B. Croft, Combining Approaches to information retrieval, Advances in Information Retrieval, 1:1-36 Kluwer Academic Publishers, 2000.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.5 Document Capture
          Subjects: Document analysis

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.4 Knowledge Representation Formalisms and Methods
          Subjects: Semantic networks

General Terms:
Design, Experimentation, Performance