Synthesizing pose, unconscious movement, and gesture for mental behavior expression of interactive characters

ABSTRACT

Mental communication with characters is important for many entertainment applications. Human's mental statuses are complex, and mental behaviors typically consist of multiple components, such as poses and various unconscious movements. In this paper, we propose the layered behavior synthesis technique for integrating mental behavior and conversational gestures. We compose various mental behavior representations by considering the spatial consistency and co-occurrence probability in interactive games. We also show an example of a game that incorporates the mental behavior synthesis technique.

REFERENCES

• 1: Matthew Stone , Doug DeCarlo , Insuk Oh , Christian Rodriguez , Adrian Stere , Alyssa Lees , Chris Bregler, Speaking with hands: creating animated conversational characters from recordings of human performance, ACM Transactions on Graphics (TOG), v.23 n.3, August 2004
• 2: Justine Cassell , Hannes Högni Vilhjálmsson , Timothy Bickmore, BEAT: the Behavior Expression Animation Toolkit, Proceedings of the 28th annual conference on Computer graphics and interactive techniques, p.477-486, August 2001 [doi>10.1145/383259.383315]
• 3: Stefan Kopp , Ipke Wachsmuth, Synthesizing multimodal utterances for conversational agents: Research Articles, Computer Animation and Virtual Worlds, v.15 n.1, p.39-52, March 2004 [doi>10.1002/cav.v15:1]
• 4: Butterworth, B. & U. Hadar,: Gesture, speech, and computational stages: A reply to McNeill. Psychological Review, 96, pp. 168--174 (1989).
• 5: Kita, S. & A. Ozyurek, Semantic coordination between speech and gesture cross-linguistically: Evidence for the interface representation of spatial thinking and speaking. Journal of Memory and Language, under review.
• 6: Kendon, A., How gesture can become like words. InPoyatos, F. (Ed.), Cross-cultural Perspective in Nonverbal Communication. pp. 131--141, Tronto: Hogrefe, 1988.
• 7: Norman, Donald, "The Design of Everyday Things", Currency/Doubleday
• 8: Sang Il Park , Hyun Joon Shin , Tae Hoon Kim , Sung Yong Shin, On-line motion blending for real-time locomotion generation: Research Articles, Computer Animation and Virtual Worlds, v.15 n.3-4, p.125-138, July 2004 [doi>10.1002/cav.v15:3/4]
• 9: Charles Rose , Michael F. Cohen , Bobby Bodenheimer, Verbs and Adverbs: Multidimensional Motion Interpolation, IEEE Computer Graphics and Applications, v.18 n.5, p.32-40, September 1998 [doi>10.1109/38.708559]
• 10: Peter Bull, "Posture and Gesture", 1987.
• 11: McNeill, D. Catchments and contexts: Non-modular factors in speech and gesture production. In D. McNeill (Ed.), Language and gesture. Cambridge University Press. pp. 312--328, 2000.
• 12: Berlyne, D. E., Aesthetics and psychobiology, Appleton-Century-Crofts, 1971.
• 13: Mehrabian, A., Inference of attitude from the posture, orientation and distance of a communicator, Journal of Consulting and Clinical Psychology 32, pp. 296--308, 1968.
• 14: Bond, M. H., Shiraishi, D., The effect of body lean and status of an interviewer on the non-verbal behavior of Japanese interviewees, International Journal of Psychology 9, pp. 117--128, 1974.
• 15: Mehrabian, A., Relationship of attitude to seated posture, orientation and distance, Journal of Personality and Social Psychology 10, pp. 359--372, 1968.
• 16: Mehrabian, A., Friar, J. T., Encoding of attitude by a seated communicator via posture and position cues, Journal of Consulting and Clinical Psychology 33, pp. 330--336, 1969.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.8 Applications

General Terms:
Design, Experimentation, Human Factors

Keywords:
interactive character, mental behavior