Conditional random fields for activity recognition

ABSTRACT

Activity recognition is a key component for creating intelligent, multi-agent systems. Intrinsically, activity recognition is a temporal classification problem. In this paper, we compare two models for temporal classification: hidden Markov models (HMMs), which have long been applied to the activity recognition problem, and conditional random fields (CRFs). CRFs are discriminative models for labeling sequences. They condition on the entire observation sequence, which avoids the need for independence assumptions between observations. Conditioning on the observations vastly expands the set of features that can be incorporated into the model without violating its assumptions. Using data from a simulated robot tag domain, chosen because it is multi-agent and produces complex interactions between observations, we explore the differences in performance between the discriminatively trained CRF and the generative HMM. Additionally, we examine the effect of incorporating features which violate independence assumptions between observations; such features are typically necessary for high classification accuracy. We find that the discriminatively trained CRF performs as well as or better than an HMM even when the model features do not violate the independence assumptions of the HMM. In cases where features depend on observations from many time steps, we confirm that CRFs are robust against any degradation in performance.

REFERENCES

• 1: J. Hammersley and P. Clifford. Markov fields on finite graphs and lattices, 1971.
• 2: K. Han and M. Veloso. Automated robot behavior recognition applied to robotic soccer. In J. Hollerbach and D. Koditschek, editors, Robotics Research: the Ninth International Symposium, pages 199--204. Springer-Verlag, London, 2000.
• 3: RoboCup-97: Robot Soccer World Cup I, January 1998
• 4: Sanjiv Kumar , Martial Hebert, Discriminative Random Fields: A Discriminative Framework for Contextual Interaction in Classification, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.1150, October 13-16, 2003
• 5: John D. Lafferty , Andrew McCallum , Fernando C. N. Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, Proceedings of the Eighteenth International Conference on Machine Learning, p.282-289, June 28-July 01, 2001
• 6: L. Liao, D. Fox, and H. Kautz. Hierarchical Conditional Random Fields for GPS-based activity recognition. In Proc. of the International Symposium of Robotis Research (ISRR 2005). Springer Verlag, 2005.
• 7: D. C. Liu , J. Nocedal, On the limited memory BFGS method for large scale optimization, Mathematical Programming: Series A and B, v.45 n.3, p.503-528, Dec. 1989 [doi>10.1007/BF01589116]
• 8: A. Y. Ng and M. I. Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, pages 841--848, Cambridge, MA, 2002. MIT Press.
• 9: L. Rabiner. A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257--286, February 1989.
• 10: David A. Ross , Simon Osindero , Richard S. Zemel, Combining discriminative features to infer complex trajectories, Proceedings of the 23rd international conference on Machine learning, p.761-768, June 25-29, 2006, Pittsburgh, Pennsylvania [doi>10.1145/1143844.1143940]
• 11: P. Rybski and M. Veloso. Using sparse visual data to model human activities in meetings. In Modeling Other Agents from Observations Workshop (MOO 2004), 2004.
• 12: Fei Sha , Fernando Pereira, Shallow parsing with conditional random fields, Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, p.134-141, May 27-June 01, 2003, Edmonton, Canada [doi>10.3115/1073445.1073473]
• 13: Cristian Sminchisescu , Atul Kanaujia , Zhiguo Li , Dimitris Metaxas, Conditional Random Fields for Contextual Human Motion Recognition, Proceedings of the Tenth IEEE International Conference on Computer Vision, p.1808-1815, October 17-20, 2005 [doi>10.1109/ICCV.2005.59]
• 14: Gita Sukthankar , Katia Sycara, Robust recognition of physical team behaviors using spatio-temporal models, Proceedings of the fifth international joint conference on Autonomous agents and multiagent systems, May 08-12, 2006, Hakodate, Japan [doi>10.1145/1160633.1160746]
• 15: C. Sutton and A. McCallum. An introduction to conditional random fields for relational learning. In L. Getoor and B. Taskar, editors, Introduction to Statistical Relational Learning. MIT Press, 2006. To appear.
• 16: H. Wallach. Efficient training of conditional random fields. Master's thesis, University of Edinburgh, 2002.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

General Terms:
Algorithms, Design, Performance

Keywords:
activity recognition, conditional random fields (CRFs), hidden Markov models (HMMs)