Mining temporal patterns of movement for video content classification

ABSTRACT

Scalable approaches to video content classification are limited by an inability to automatically generate representations of events that encode abstract temporal structure. This paper presents a method in which temporal information is captured by representing events using a lexicon of hierarchical patterns of movement that are mined from large corpora of unannotated video data. These patterns are then used as features for a discriminative model of event classification that exploits tree kernels in a Support Vector Machine. Evaluations show the method learns informative patterns on a 1450-hour video corpus of natural human activities recorded in the home.

REFERENCES

• 1: James F. Allen, Towards a general theory of action and time, Artificial Intelligence, v.23 n.2, p.123-154, July 1984 [doi>10.1016/0004-3702(84)90008-0]
• 2: Paul R. Cohen, Fluent Learning: Elucidating the Structure of Episodes, Proceedings of the 4th International Conference on Advances in Intelligent Data Analysis, p.268-277, September 13-15, 2001
• 3: Michael Collins , Nigel Duffy, New ranking algorithms for parsing and tagging: kernels over discrete structures, and the voted perceptron, Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, July 07-12, 2002, Philadelphia, Pennsylvania [doi>10.3115/1073083.1073128]
• 4: Fleischman, M. B. and Roy, D. Why Verbs are Harder to Learn than Nouns: Initial Insights from a Computational Model of Intention Recognition in Situated Word Learning. 27th Annual Meeting of the Cognitive Science Society, Stresa, Italy. July 2005.
• 5: Fern, A., Givan R., Siskind, J.: Specific-to-General Learning for Temporal Events with Application to Learning Event Definitions from Video. J. Artif. Intell. Res. (JAIR) 17: 379--449 (2002)

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Knowledge acquisition

General Terms:
Algorithms, Experimentation

Keywords:
support vector machine, temporal data mining, tree kernel, video content classification, video event recognition