Invariance in motion analysis of videos

ABSTRACT

In this paper, we propose an approach that retrieves motion of objects from the videos based on the dynamic time warping of view invariant characteristics. The motion is represented as a sequence of dynamic instants and intervals, which are automatically computed using the spatiotemporal curvature of the trajectory of moving object in the videos. Dynamic Time Warping (DTW) method matches trajectories using a view invariant similarity measure. Our system is able to incrementally learn different actions without any initialization mode, therefore it can work in an unsupervised manner. The retrieval of relevant videos can be easily performed by computing a simple distance metric. This paper makes two fundamental contribution to view invariant video retrieval: (1) Dynamic Instant detection in trajectories of moving objects acquired from video. (2) View-invariant Dynamic Time Warping to measure similarity between two trajectories of actions performed by different persons and from different viewpoints. Although the learning algorithm is relatively simple in our approach, we can achieve high recognition rate because of the view-invariant representation and the similarity measure using DTW.

REFERENCES

• 1: Cen Rao and Mubarak Shah, View invariance in action recognition. In IEEE International Conference on Computer Vision and Pattern Recognition, CVPR 2001, Kauai, Hawaii, Dec. 2001.
• 2: Tanveer Syeda-Mahmood, A. Vasilescu, and S. Sethi, Recognizing action events from multiple viewpoints. In IEEE Workshop on Detection and Recognition of Events in Video (EVENT'01), Vancouver, Canada, July 2001.
• 3: L. W. Campbell , D. A. Becker , A. Azarbayejani , A. F. Bobick , A. Pentland, Invariant features for 3-D gesture recognition, Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96), p.157, October 14-16, 1996
• 4: P. Perona , J. Malik, Scale-Space and Edge Detection Using Anisotropic Diffusion, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.12 n.7, p.629-639, July 1990 [doi>10.1109/34.56205]
• 5: Paul J. Besl , Ramesh C. Jain, Invariant surface characteristics for 3D object recognition in range images, Computer Vision, Graphics, and Image Processing, v.33 n.1, p.33-80, Jan. 1986 [doi>10.1016/0734-189X(86)90220-3]
• 6: R. Kjeldsen , J. Kender, Finding skin in color images, Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96), p.312, October 14-16, 1996
• 7: Jeffrey Mark Siskind , Quaid Morris, A Maximum-Likelihood Approach to Visual Event Classification, Proceedings of the 4th European Conference on Computer Vision-Volume II, p.347-360, April 15-18, 1996
• 8: James W. Davis and Aaron Bobick. "Action recognition using temporal templates", pages 125--146. CVPR-97, 1997.
• 9: Michael J. Black , Allan D. Jepson, EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation, International Journal of Computer Vision, v.26 n.1, p.63-84, Jan. 1998 [doi>10.1023/A:1007939232436]
• 10: M. Izumi A. Kojiama "Generating natural language description of human behavior from video images", ICPR-2000, 4: 728--731, 2000.
• 11: Steven M. Seitz , Charles R. Dyer, View-Invariant Analysis of Cyclic Motion, International Journal of Computer Vision, v.25 n.3, p.231-251, Dec. 1997 [doi>10.1023/A:1007928103394]
• 12: Joseph L. Mundy , Andrew Zisserman, Geometric invariance in computer vision, MIT Press, Cambridge, MA, 1992
• 13: Carlo Tomasi , Takeo Kanade, Shape and motion from image streams under orthography: a factorization method, International Journal of Computer Vision, v.9 n.2, p.137-154, Nov. 1992 [doi>10.1007/BF00129684]
• 14: J. Davis, A. Bobick, W. Richards, "Categorical Representation and Recognition of Oscillatory Motion Patterns", IEEE Conference on Computer Vision and Pattern Recognition, June 2000, pp. 628--635.
• 15: Yaser Yacoob , Michael J. Black, Parameterized Modeling and Recognition of Activities, Proceedings of the Sixth International Conference on Computer Vision, p.120, January 04-07, 1998
• 16: S. Niyogi and E.H. Adelson, "Analyzing and recognizing walking figures in XYT", cvpr 1994.
• 17: A. Nishikawa , A. Ohnishi , F. Miyazaki, Description and Recognition of Human Gestures Based on the Transition of Curvature from Motion Images, Proceedings of the 3rd. International Conference on Face & Gesture Recognition, p.552, April 14-16, 1998
• 18: R. Polana and R.C. Nelson, "Detecting activities", J. of Visual Communication and Image Representation", vol 5, P172--180, 1994.
• 19: M. -H. Yang , N. Ahuja, Extracting Gestural Motion Trajectories, Proceedings of the 3rd. International Conference on Face & Gesture Recognition, p.10, April 14-16, 1998
• 20: removed for blind review, Proc. IEEE Workshop on Applications of Computer Vision, WACV'98, 1998.
• 21: M. J. Black , Y. Yacoob, Tracking and recognizing rigid and non-rigid facial motions using local parametric models of image motion, Proceedings of the Fifth International Conference on Computer Vision, p.374, June 20-23, 1995
• 22: C. Bregler and A. Hertzmann and H. Biermann, "Recovering non-rigid 3d shape from image streams",CVPR, 2000.
• 23: M. Black and A. Jepson, "Eigentracking: Robust matching and tracking of articulated objects using a view-based representation",ECCV 1998.
• 24: Ismail Haritaoglu , Davis Harwood , Larry S. David, W4: Real-Time Surveillance of People and Their Activities, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.8, p.809-830, August 2000 [doi>10.1109/34.868683]
• 25: Y. Caspi and M. Irani, "A step towards sequence-to-sequence alignment",CVPR 2000
• 26: K.Yu, J.Mason, J.Oglesby, "Speaker recognition using hidden Markov models, dynamic time warping and vector quantisation", IEEE Proceedings- Vision, Image and Signal Processing Vol.142, Issue 5, pg. 313--318, Oct 1995.
• 27: J. Brian Burns , R. S. Weiss , E. M. Riseman, View Variation of Point-Set and Line-Segment Features, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.15 n.1, p.51-68, January 1993 [doi>10.1109/34.184774]
• 28: D. Moore, I. Essa, M. Hayes, "ObjectSpaces: Context Management for Action Recognition," Proceedings of the 2nd Annual Conference on Audio-Visual Biometric Person Authentication, Washington, D.C.,March 1999
• 29: Jesse Hoey and James J. Little, "Representation and recognition of complex human motion". In Proc. IEEE CVPR, Hilton Head, SC, June 2000
• 30: Nuria M. Oliver , Barbara Rosario , Alex P. Pentland, A Bayesian Computer Vision System for Modeling Human Interactions, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.8, p.831-843, August 2000 [doi>10.1109/34.868684]
• 31: G. R. Bradski. Computer vision face tracking for use in a perceptual user interface. Intel Tech J Q2, 1998
• 32: T. Syeda-Mahmood, "Segmenting Actions in Velocity Curve Space", ICPR 2002.
• 33: Vasu Parameswaran and Rama Chellappa, "Quasi-Invariants for Human Action Representation and Recognition".
• 34: Krishnan Rangarajan , Mubarak Shah, Establishing motion correspondence, CVGIP: Image Understanding, v.54 n.1, p.56-73, July 1991 [doi>10.1016/1049-9660(91)90075-Z]
• 35: John M. Rubin , W. A. Richards, Boundaries of Visual Motion, Massachusetts Institute of Technology, Cambridge, MA, 1985
• 36: Trevor J. Darrell , Irfan A. Essa , Alex P. Pentland, Task-Specific Gesture Analysis in Real-Time Using Interpolated Views, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.18 n.12, p.1236-1242, December 1996 [doi>10.1109/34.546259]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering
          Subjects: Algorithms

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering
          Subjects: Similarity measures

General Terms:
Algorithms, Measurement

Keywords:
human actions, learning, spatiotemporal curvature, view-invariant action representation, view-invariant dynamic time warping, view-invariant measure