Categorizing bi-object video activities using bag of segments and causality features

ABSTRACT

We address the recognition problem of video activities involving two interacting moving objects under a surveillance camera. We develop a novel video activity representation scheme --'bag of segments'. In this scheme, the video sessions are represented as a collection of independent segments, with memberships to each pre-learned visual patterns that we call codewords. To better represent the video segments with object interaction, we design a set of new features based on the prediction filter responses and the Granger Causality Test (GCT). These features capture the inter-relationship between moving objects and are combined with conventional features such as position and velocity. We validate the proposed method for the task of video activities classification with extensive experiments on a surveillance database with 867 video sessions.

REFERENCES

• 1: David M. Blei , Andrew Y. Ng , Michael I. Jordan, Latent dirichlet allocation, The Journal of Machine Learning Research, 3, p.993-1022, 3/1/2003 [doi>10.1162/jmlr.2003.3.4-5.993]
• 2: Oren Boiman , Michal Irani, Detecting Irregularities in Images and in Video, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.462-469, October 17-20, 2005 [doi>10.1109/ICCV.2005.70]
• 3: M. Brand , N. Oliver , A. Pentland, Coupled hidden Markov models for complex action recognition, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.994, June 17-19, 1997
• 4: Dorin Comaniciu , Visvanathan Ramesh , Peter Meer, Kernel-Based Object Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.5, p.564-575, May 2003 [doi>10.1109/TPAMI.2003.1195991]
• 5: Pedro Domingos , Michael Pazzani, On the Optimality of the Simple Bayesian Classifier under Zero-One Loss, Machine Learning, v.29 n.2-3, p.103-130, Nov./Dec. 1997 [doi>10.1023/A:1007413511361]
• 6: C.W.J. Granger. Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3):424--438, 1969.
• 7: Thomas Hofmann, Probabilistic latent semantic indexing, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.50-57, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312649]
• 8: Yuri A. Ivanov , Aaron F. Bobick, Recognition of Visual Activities and Interactions by Stochastic Parsing, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.8, p.852-872, August 2000 [doi>10.1109/34.868686]
• 9: Kuang-Chih Lee , Jeffrey Ho , Ming-Hsuan Yang , David Kriegman, Visual tracking and recognition using probabilistic appearance manifolds, Computer Vision and Image Understanding, v.99 n.3, p.303-331, September 2005 [doi>10.1016/j.cviu.2005.02.002]
• 10: David D. Lewis, Naive (Bayes) at Forty: The Independence Assumption in Information Retrieval, Proceedings of the 10th European Conference on Machine Learning, p.4-15, April 21-23, 1998
• 11: Fei-Fei Li , Pietro Perona, A Bayesian Hierarchical Model for Learning Natural Scene Categories, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.524-531, June 20-26, 2005 [doi>10.1109/CVPR.2005.16]
• 12: Gérard Medioni , Isaac Cohen , François Brémond , Somboon Hongeng , Ramakant Nevatia, Event Detection and Analysis from Video Streams, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.8, p.873-889, August 2001 [doi>10.1109/34.946990]
• 13: A.Y. Ng, M.I. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. Proceedings of the Neural Information Processing Systems Conference pages 849--856, 2001.
• 14: C.A. Sims. Money, income, and causality. American Economic Review 62(4):540--52, September 1972.
• 15: Josef Sivic , Bryan C. Russell , Alexei A. Efros , Andrew Zisserman , William T. Freeman, Discovering Objects and their Localization in Images, Proceedings of the Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1, p.370-377, October 17-20, 2005 [doi>10.1109/ICCV.2005.77]
• 16: C. Stauffer and W. Grimson. Adaptive background mixture models for real-time tracking. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition 02:252, 1999.
• 17: H. Zhong, J. Shi, and M. Visontai. Detecting unusual activity in video. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition 02:819--826, 2004.
• 18: Y. Zhou, S. Yan,and T. Huang. Pair-activity classification by bi-trajectory analysis. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition 2008.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.7 Feature Measurement

General Terms:
Algorithms, Design, Management, Theory

Keywords:
'bag of segments', classification, granger causality, video activity