Take a look at the new version of this page: [ beta version ]. Tell us what you think.

ABSTRACT

The human face is an independent communication channel that conveys emotional and conversational signals encoded as facial displays. Facial displays can be viewed as communicative signals that help coordinate conversation. We are attempting to introduce facial displays into computer-human interaction as a new modality. This will make the interaction tighter and more efficient while lessening the cognitive load. As the first step, a speech dialogue system was selected to investigate the power of communicative facial displays. We analyzed the conversations between users and the speech dialogue system, to which facial displays had been added. We found that conversation with the system featuring facial displays was more successful than that with a system without facial displays.

REFERENCES

• 1: Blatmer, M. Multimedia and Multimodal User Interface Design: CHI'92 Tutorial Course Note 4. ACM Press, 1992.
• 2: Chovil, N. Communicative Functions of Facial Displays in Conversation. Ph.D. Thesis, University of Victoria, 1989.
• 3: Darwin, C. The Expression of Emotion in Man and Animals. University of Chicago Press, Chicago, 1965.
• 4: Abbe Don , Susan Brennan , Brenda Laurel , Ben Shneiderman, Anthropomorphism: from Eliza to Terminator 2, Proceedings of the SIGCHI conference on Human factors in computing systems, p.67-70, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142760]
• 5: Abbe Don , Tim Oren , Brenda Laurel, Guides 3.0, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.447-448, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108996]
• 6: Ekman, P. and Friesen, W. V. The repertoire of nonverbal behavior - categories, origins, usage, and coding, Semiotica 1 (1969), pp. 49-98.
• 7: Ekman, P. and Friesen, W. V. Facial Action Coding System. Consulting Psychologists Press, Palo Alto, California, 1978.
• 8: Ekman, P. and Friesen, W. V. Unmasking the Face. Consulting Psychologists Press, Inc., Palo Alto, California, 1984.
• 9: Fridlund, A. J. and Gilbert, A. N. Emotions and facial expression, Science, 230 (1985), pp. 607-608.
• 10: Hindus, D. and Brennan, S. Conversational Paradigms in User Interfaces: CHI'92 Tutorial Course Note 11. ACM Press, 1992.
• 11: Itou, K. and Hayamizu, S. and Tanaka, H. Continuous speech recognition by context-dependent phonetic HMM and an efficient algorithm for finding N-best sentence hypotheses, in Proc. ICASSP'92, IEEE Press, pp. 1 21-I 24.
• 12: Katashi Nagao, A preferential constraint satisfaction technique for natural language analysis, Proceedings of the 10th European conference on Artificial intelligence, p.523-527, August 1992, Vienna, Austria
• 13: Nagao, K. and Osawa, E. A Logic-Based Approach to Plan Recognition and Belief Revision. Tech. Report. SCSL-TR-92-007, Sony Computer Science Laboratory, Inc., Tokyo, 1992.
• 14: Perret, D. I. et al. Neurones responsive to faces in the temporal cortex: studies of functional organization sensitivity and relation to perception. Human Neurobiology, 3 (1984) 197-208.
• 15: Sherer, K. R. The functions of nonverbal signs in conversation, in The Social and Psychological Contexts of Language, St. Clair, R. N. and Giles, H. ( Eds.), Lawrence Erlbaum, Hillsdale, NJ, 1980, pp. 225-244.
• 16: Takeuchi, A. and Franks, S. A Rapid Face Construction Lab. Tech. Report. SCSL-TR-92-010, Sony Computer Science Laboratory, Inc., Tokyo, 1992.
• 17: Masaru Tomita, An efficient augmented-context-free parsing algorithm, Computational Linguistics, v.13 n.1-2, p.31-46, January-June 1987
• 18: Keith Waters, A muscle model for animation three-dimensional facial expression, ACM SIGGRAPH Computer Graphics, v.21 n.4, p.17-24, July 1987

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Animations
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Theory and methods; Interaction styles (e.g., commands, menus, forms, direct manipulation)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.1 Applications and Expert Systems
          Subjects: Natural language interfaces
      I.2.7 Natural Language Processing
          Subjects: Speech recognition and synthesis

General Terms:
Design, Experimentation, Human Factors, Languages, Theory

Keywords:
anthropomorphism, conversational interfaces, facial expression, multimodal interfaces, user interface design