People tracking and segmentation using spatiotemporal shape constraints

ABSTRACT

We present an efficient people tracking and segmentation algorithm for gait recognition. Even though most existing gait recognition algorithms assume that people have been tracked and that silhouettes are available for gait classification, tracking and segmentation are very difficult especially for articulated objects such as human beings. We improve the performance of tracking and segmentation based on spatiotemporal shape constraints. First of all, we track people using an adaptive mean-shift tracker which produces initial results consisting of bounding boxes and foreground likelihood images. The initial results, generally speaking, are not accurate enough to be applied in gait recognition directly. We refine the results by matching with silhouette templates sequences in a batch mode to find the optimal silhouette-based gait paths corresponding to the input. Since the process is computationally expensive, we propose a novel efficient distance computation method to accelerate the spatiotemporal silhouette matching. The spatiotemporal shape priors are embedded into the Min-Cut algorithm to segment people out. Experiments on indoor and outdoor sequences demonstrate the effectiveness of the proposed approach.

REFERENCES

• 1: Ankur Agarwal , Bill Triggs, Recovering 3D Human Pose from Monocular Images, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.28 n.1, p.44-58, January 2006 [doi>10.1109/TPAMI.2006.21]
• 2: Gunilla Borgefors, Distance transformations in digital images, Computer Vision, Graphics, and Image Processing, v.34 n.3, p.344-371, June 1986 [doi>10.1016/S0734-189X(86)80047-0]
• 3: Y. Boykov and M-P. Jolly. "Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in n-d images," in Proc. of Int'l Conf. on Computer Vision pp. 105--112, 2001.
• 4: Yuri Boykov , Vladimir Kolmogorov, An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.26 n.9, p.1124-1137, September 2004 [doi>10.1109/TPAMI.2004.60]
• 5: T. Brox, B. Rosenhahn, J. Weickert. "Three-dimensional shape knowledge for joint image segmentation and pose estimation," in Proc. 27th DAGM pp. 109--116, 2005.
• 6: M. Bray, P. Kohli, P. H. S. Torr. "PoseCut: Simultaneous Segmentation and 3D Pose Estimation of Humans Using Dynamic Graph-Cuts," in ECCV II, pp. 642--655, 2006.
• 7: Robert T. Collins , Yanxi Liu , Marius Leordeanu, Online Selection of Discriminative Tracking Features, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.10, p.1631-1643, October 2005 [doi>10.1109/TPAMI.2005.205]
• 8: Dorin Comaniciu , Visvanathan Ramesh , Peter Meer, Kernel-Based Object Tracking, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.5, p.564-575, May 2003 [doi>10.1109/TPAMI.2003.1195991]
• 9: A. Criminisi , G. Cross , A. Blake , V. Kolmogorov, Bilayer Segmentation of Live Video, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.53-60, June 17-22, 2006 [doi>10.1109/CVPR.2006.69]
• 10: A. Elgammal and C-S. Lee. "Inferring 3D body pose from silhouettes using activity manifold learning," in Proc. of IEEE Conf. on Computer Vision and Patten recognition pp. II-681-II-688, 2005. ,
• 11: Daniel Freedman , Tao Zhang, Interactive Graph Cut Based Segmentation with Shape Priors, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.755-762, June 20-26, 2005 [doi>10.1109/CVPR.2005.191]
• 12: V. Kolmogorov , A. Criminisi , A. Blake , G. Cross , C. Rother, Bi-Layer Segmentation of Binocular Stereo Video, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.407-414, June 20-26, 2005 [doi>10.1109/CVPR.2005.91]
• 13: David Serby , Esther Koller-Meier , Luc Van Gool, Probabilistic Object Tracking Using Multiple Features, Proceedings of the Pattern Recognition, 17th International Conference on (ICPR'04) Volume 2, p.184-187, August 23-26, 2004 [doi>10.1109/ICPR.2004.716]
• 14: R. Li, M-H Yang, S. Sclaroff, and T-P. Tian. "Monocular tracking of 3D human motion with a coordinated mixture of factor analyzers", in Proc. of European Conf. on Computer Vision pp. 323--330, 2006.
• 15: M. Marszalek, C. Schmid. "Accurate Object Localization with Shape Masks." in Proc. of IEEE Conf. on Computer Vision and Patten Recognition pp. 1--8, 2007.
• 16: Y. Makihara and R. Sagawa and Y. Mukaigawa and T. Echigo and Y. Yagi. "Gait Recognition Using a View Transformation Model in the Frequency Domain," in Proc. of European Conf. on Computer Vision pp. 151--163, 2006.
• 17: Lawrence Rabiner , Biing-Hwang Juang, Fundamentals of speech recognition, Prentice-Hall, Inc., Upper Saddle River, NJ, 1993
• 18: Yogesh Rathi , Namrata Vaswani , Allen Tannenbaum , Anthony Yezzi, Particle Filtering for Geometric Active Contours with Application to Tracking Moving and Deforming Objects, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 2, p.2-9, June 20-26, 2005 [doi>10.1109/CVPR.2005.271]
• 19: Hedvig Sidenbladh , Michael J. Black , David J. Fleet, Stochastic Tracking of 3D Human Figures Using 2D Image Motion, Proceedings of the 6th European Conference on Computer Vision-Part II, p.702-718, June 26-July 01, 2000
• 20: Cristian Sminchisescu , Allan Jepson, Generative modeling for continuous non-linearly embedded visual inference, Proceedings of the twenty-first international conference on Machine learning, p.96, July 04-08, 2004, Banff, Alberta, Canada [doi>10.1145/1015330.1015371]
• 21: Michael J. Swain , Dana H. Ballard, Color indexing, International Journal of Computer Vision, v.7 n.1, p.11-32, Nov. 1991 [doi>10.1007/BF00130487]
• 22: K. R. , Jr. Sloan , S. L. Tanimoto, Progressive Refinement of Raster Images, IEEE Transactions on Computers, v.28 n.11, p.871-874, November 1979 [doi>10.1109/TC.1979.1675269]
• 23: K. Toyama and A. Blake. "Probabilistic tracking in a metric space," in Proc. of Int'l Conf. on Computer Vision pp. 50--57, 2001.
• 24: Raquel Urtasun , David J. Fleet , Pascal Fua, 3D People Tracking with Gaussian Process Dynamical Models, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.238-245, June 17-22, 2006 [doi>10.1109/CVPR.2006.15]
• 25: Ashok Veeraraghavan , Amit K. Roy-Chowdhury , Rama Chellappa, Matching Shape Sequences in Video with Applications in Human Movement Analysis, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.12, p.1896-1909, December 2005 [doi>10.1109/TPAMI.2005.246]
• 26: J. Wang, Y. Yagi, "Integrating Color and Shape-texture Features for Adaptive Real-time Tracking", IEEE Trans. on Image Processing vol. 17, no. 2, 2008.
• 27: L. Wang, T. Tan, W. Hu, H. Ning. "Automaticgait recognition based on statistical shape analysis." IEEE Trans. on Image Processing 12(9), pp. 1120--1131, 2003.
• 28: P. Yin, A. Criminisi, J. Winn, and I. Essa. "Tree-based classifiers for bilayer video segmentation", in Proc. of Conf. on Computer Vision and Pattern Recognition pp. 1--8, 2007.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications

General Terms:
Algorithms

Keywords:
people segmentation, people tracking, spatiotemporal shape priors