Walk to here: a voice driven animation system

ABSTRACT

We present a novel interface for directing the actions of computer animated characters and camera movements. Our system takes spoken input in combination with mouse pointing to generate desired character animation based on motion capture data. The aim is to achieve a more natural animation interface by supporting the types of dialogue and pointing that might be used when one person is explaining a desired motion to another person. We compare our voice-driven system with a button-driven animation interface that has equivalent capabilities. An informal user study indicates that for the test scenarios, the voice-user interface (VUI) is faster than an equivalent graphical user interface (GUI). Potential applications include storyboarding for film or theatre, directing characters in video games, and scene reconstruction.

REFERENCES

• 1: Okan Arikan , D. A. Forsyth, Interactive motion generation from examples, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 2: {Apa02} Apaydin O.: Networked humanoid animation driven by human voice using extensible 3d (x3d), h-anim and java speech open standards. Master's Thesis at Naval Postgraduate School (2002).
• 3: {Bel05} Beletsky A.: Wave player and recorder library. http://www.codeguru.com/Cpp/G-M/multimedia/audio/article.php/c9305/ (2005).
• 4: Richard A. Bolt, “Put-that-there”: Voice and gesture at the graphics interface, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.262-270, July 14-18, 1980, Seattle, Washington, United States
• 5: T. W. Calvert , J. Chapman , A. Patla, The integration of subjective and objective data in the animation of human movement, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.198-203, July 14-18, 1980, Seattle, Washington, United States
• 6: Jinxiang Chai , Jessica K. Hodgins, Performance animation from low-dimensional control signals, ACM Transactions on Graphics (TOG), v.24 n.3, July 2005
• 7: Jauvane C. de Oliveira , Shervin Shirmohammadi , Nicolas D. Georganas, A Collaborative Virtual Environment for Industrial Training, Proceedings of the IEEE Virtual Reality 2000 Conference, p.288, March 18-22, 2000
• 8: Li-wei He , Michael F. Cohen , David H. Salesin, The virtual cinematographer: a paradigm for automatic real-time camera control and directing, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.217-224, August 1996 [doi>10.1145/237170.237259]
• 9: Moon Ryul Jung , Jugal K. Kalita , Norman I. Badler , Wallace Ching, Simulating human tasks using simple natural language instructions, Proceedings of the 23rd conference on Winter simulation, p.1049-1057, December 08-11, 1991, Phoenix, Arizona, United States
• 10: Jehee Lee , Jinxiang Chai , Paul S. A. Reitsma , Jessica K. Hodgins , Nancy S. Pollard, Interactive control of avatars animated with human motion data, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 11: A. Bryan Loyall , W. Scott Neal Reilly , Joseph Bates , Peter Weyhrauch, System for authoring highly interactive, personality-rich interactive characters, Proceedings of the 2004 ACM SIGGRAPH/Eurographics symposium on Computer animation, August 27-29, 2004, Grenoble, France [doi>10.1145/1028523.1028532]
• 12: Michael Neff , Eugene Fiume, AER: aesthetic exploration and refinement for expressive character animation, Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation, July 29-31, 2005, Los Angeles, California [doi>10.1145/1073368.1073391]
• 13: Sharon Oviatt , Rachel Coulston , Rebecca Lunsford, When do we interact multimodally?: cognitive load and multimodal communication patterns, Proceedings of the 6th international conference on Multimodal interfaces, October 13-15, 2004, State College, PA, USA [doi>10.1145/1027933.1027957]
• 14: Sharon Oviatt, Ten myths of multimodal interaction, Communications of the ACM, v.42 n.11, p.74-81, Nov. 1999 [doi>10.1145/319382.319398]
• 15: Alexander I. Rudnicky, The design of voice-driven interfaces, Proceedings of the workshop on Speech and Natural Language, p.120-124, February 21-23, 1989, Philadelphia, Pennsylvania [doi>10.3115/100964.100972]
• 16: Matthew Thorne , David Burke , Michiel van de Panne, Motion doodles: an interface for sketching character motion, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.7 Three-Dimensional Graphics and Realism
          Subjects: Animation

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Animations

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Motion

General Terms:
Algorithms, Design