Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

Our context is Virtual Theater. Our aim is to put on a short play featuring a real actor and a virtual actor, who will communicate through movements and choreography with mutual synchronization. We currently work on a system that can recognize key-gestures made by a real actor.

In this paper, we cover a method for real-time recognition. Our idea is that the data for movements from any motion-capture system can be reduced to a single artificial signature. We use properties from Principal Component Analysis (PCA) to generate it. This artificial gesture representation is used in real-time by our multiagent systems to simultaneously perform segmentation (gesture's beginning and end) and recognition. We conducted various experiments which demonstrate our system and define its limitations.

REFERENCES

• 1: M. Aubry, F. Julliard, and S. Gibet. Apprentissage des paramètres d'un contrôleur pour la synthèse de mouvements réalistes. Journées l'Aber Wrac'h des 29 et 30 mai 2008, Jun 2008.
• 2: Jernej Barbič , Alla Safonova , Jia-Yu Pan , Christos Faloutsos , Jessica K. Hodgins , Nancy S. Pollard, Segmenting motion capture data into distinct behaviors, Proceedings of Graphics Interface 2004, p.185-194, May 17-19, 2004, London, Ontario, Canada
• 3: Donald J. Berndt , James Clifford, Finding patterns in time series: a dynamic programming approach, Advances in knowledge discovery and data mining, American Association for Artificial Intelligence, Menlo Park, CA, 1996
• 4: A. F. Bobick , A. D. Wilson, A state-based technique for the summarization and recognition of gesture, Proceedings of the Fifth International Conference on Computer Vision, p.382, June 20-23, 1995
• 5: S. Boukir and F. Cheneviere. Conception d'un systÃĺme de reconnaissance de gestes dansés = design of a dance gesture recognition system. TS. Traitement du signal (Trait. signal) ISSN 0765-0019, 21(3):195--203, 2004.
• 6: L. W. Campbell , A. F. Bobick, Recognition of human body motion using phase space constraints, Proceedings of the Fifth International Conference on Computer Vision, p.624, June 20-23, 1995
• 7: Jinxiang Chai , Jessica K. Hodgins, Performance animation from low-dimensional control signals, ACM SIGGRAPH 2005 Papers, July 31-August 04, 2005, Los Angeles, California [doi>10.1145/1186822.1073248]
• 8: Ajo Fod , Maja J. Matarić , Odest Chadwicke Jenkins, Automated Derivation of Primitives for Movement Classification, Autonomous Robots, v.12 n.1, p.39-54, January 2002 [doi>10.1023/A:1013254724861]
• 9: K. Forbes , E. Fiume, An efficient search algorithm for motion data using weighted PCA, Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 29-31, 2005, Los Angeles, California [doi>10.1145/1073368.1073377]
• 10: Isabelle Guyon , André Elisseeff, An introduction to variable and feature selection, The Journal of Machine Learning Research, 3, 3/1/2003
• 11: Odest Chadwicke Jenkins , Maja J. Mataric, Automated derivation of behavior vocabularies for autonomous humanoid motion, Proceedings of the second international joint conference on Autonomous agents and multiagent systems, July 14-18, 2003, Melbourne, Australia [doi>10.1145/860575.860612]
• 12: Daehwan Kim , Jinyoung Song , Daijin Kim, Simultaneous gesture segmentation and recognition based on forward spotting accumulative HMMs, Pattern Recognition, v.40 n.11, p.3012-3026, November, 2007 [doi>10.1016/j.patcog.2007.02.010]
• 13: Hyeon-Kyu Lee , Jin H. Kim, An HMM-Based Threshold Model Approach for Gesture Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.10, p.961-973, October 1999 [doi>10.1109/34.799904]
• 14: N. Magnenat-Thalmann and A. Egges. Interactive virtual humans in real-time virtual environments. The International Journal of Virtual Reality, 5(2):15--24, June 2006.
• 15: N. Rezzoug, P. Gorce, A. Heloir, S. Gibet, N. Courty, J.-F. Kamp, F. Multon, and C. Pelachaud. Virtual humanoids endowed with expressive communication gestures: the hugex project. In Proc. of IEEE Systems, Man and Cybernetics, Taipei, Taiwan, October 2006. IEEE CS.
• 16: Charles Rose , Michael F. Cohen , Bobby Bodenheimer, Verbs and Adverbs: Multidimensional Motion Interpolation, IEEE Computer Graphics and Applications, v.18 n.5, p.32-40, September 1998 [doi>10.1109/38.708559]
• 17: A. Sandberg. Gesture recognition using neural networks, 1997. Master's thesis TRITA-NA-E9727, dept.
• 18: J. Shlens. A tutorial on principal component analysis, December 2005.
• 19: M. Alex O. Vasilescu, Human Motion Signatures: Analysis, Synthesis, Recognition, Proceedings of the 16 th International Conference on Pattern Recognition (ICPR'02) Volume 3, p.30456, August 11-15, 2002
• 20: Andrew D. Wilson , Aaron F. Bobick , Justine Cassell, Recovering the Temporal Structure of Natural Gesture, Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96), p.66, October 14-16, 1996
• 21: J. Yamato, J. Ohya, and K. Ishii. Recognizing human action in time-sequential images using hidden markov model. Computer Vision and Pattern Recognition, 1992. Proceedings CVPR '92., 1992 IEEE Computer Society Conference on, pages 379--385, 15--18 Jun 1992.
• 22: H. Zhou and H. Hu. A survey--human movement tracking and stroke rehabilitation, 2004. Technical Report: CSM-420.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications

Keywords:
gesture command, gesture recognition, motion-capture, virtual theatre