Real-time 3d arm pose estimation from monocular video for enhanced HCI

ABSTRACT

In this paper an approach for 3D arm pose estimation from a monocular video is presented. Our proposal has been designed to provide real-time and realistic reconstruction of the user motion, as required by advanced Human Computer Interaction (HCI) applications. Both a 2D arm tracking and a 3D arm pose estimation algorithm are introduced and discussed. Tracking exploits fast and robust segmentation of the arm silhouette together with detection and tracking of skin colored regions. 3D pose estimation relies on a stick-figure arm model and the Analysis-by-Synthesis approach, but achieves real-time performance using geometrical constraints on tracking results to reduce the search space cardinality. Experiments on the animation of 3D avatars using off-the-shelf hardware demonstrate the effectiveness and real-time performance of our proposal.

REFERENCES

• 1: S. Askar, Y. Kondratyuk, K. Elazouzi, P. Kauff, and O. Schreer. Vision-based skin-colour segmentation of moving hands for real-time applications. Proc. of the CVMP pages 79--85, 15-16 March 2004.
• 2: Y. Azoz , L. Devi , R. Sharma, Reliable Tracking of Human Arm Dynamics by Multiple Cue Integration and Constraint Fusion, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.905, June 23-25, 1998
• 3: Dave Bullock , John Zelek, Towards real-time 3-D monocular visual tracking of human limbs in unconstrained environments, Real-Time Imaging, v.11 n.4, p.323-353, August 2005 [doi>10.1016/j.rti.2005.06.004]
• 4: L. Goncalves , E. Di Bernardo , E. Ursella , P. Perona, Monocular tracking of the human arm in 3D, Proceedings of the Fifth International Conference on Computer Vision, p.764, June 20-23, 1995
• 5: A. S. Micilotta, E.-J. Ong, and R. Bowden. Real-time upper body detection and 3d pose estimation in monoscopic images. In Proc. of the ECCV volume3, pages 139--150, 2006.
• 6: K. Mikolajczyk, C. Schmid, and A. Zisserman. Human detection based on a probabilistic assembly of robust part detectors. In Proc. of the ECCV volume I, pages 69--81, 2004.
• 7: Thomas B. Moeslund , Erik Granum, Modelling and estimating the pose of a human arm, Machine Vision and Applications, v.14 n.4, p.237-247, September 2003 [doi>10.1007/s00138-002-0090-1]
• 8: Thomas B. Moeslund , Adrian Hilton , Volker Krüger, A survey of advances in vision-based human motion capture and analysis, Computer Vision and Image Understanding, v.104 n.2, p.90-126, November 2006 [doi>10.1016/j.cviu.2006.08.002]
• 9: J. Rehg, D. D. Morris, and T. Kanade. Ambiguities in visual tracking of articulated objects using two-and three-dimensional models. International Journal of Robotics Research 22(6):393--418, June 2003.
• 10: O. Schreer, P. Eisert, P. Kauff, R. Tanger, and R. Englert. Towards robust intuitive vision-based user interfaces. Proc. of the ICME pages 69--72, 2006.
• 11: O. Schreer, R. Tanger, P. Eisert, P. Kauff, B. Kaspar, and R. Englert. Real-time avatar animation steered by live body motion. In ICIAP volume 1, pages 147--154, 2005.
• 12: Matheen Siddiqui , Gérard Medioni, Robust real-time upper body limb detection and tracking, Proceedings of the 4th ACM international workshop on Video surveillance and sensor networks, October 27-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178782.1178792]
• 13: L. Sigal, S. Bhatia, S. Roth, M. J. Black, and M. Isard. Tracking loose-limbed people. In CVPR (1) pages 421--428, 2004.
• 14: C. Sminchisescu and B. Triggs. Estimating articulated human motion with covariance scaled sampling. International Journal of Robotics Research 22(6):371--391, June 2003. Special issue on Visual Analysis of Human Movement.
• 15: P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. Proc. of CVPR 1:511--518, 2001.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Tracking

General Terms:
Design, Performance

Keywords:
3d arm pose estimation, human computer interaction