Emoticons convey emotions without cognition of faces: an fMRI study

ABSTRACT

In this paper, we describe the brain activities that are associated with emoticons by using functional MRI (fMRI). In communication over a computer network, we use abstract faces such as computer graphics (CG) avatars and emoticons. These faces convey users' emotions and enrich their communications. In particular, when we see some abstract faces, we feel that they are more vivid and lively than photorealistic faces. However, the manner in which these faces influence the mental process is as yet unknown. In this research, we conducted an experiment by using fMRI for the most abstract faces?emoticons. The experimental results show that emoticons convey emotions without the cognition of faces. This result is very important in order to promote an understanding of how abstract faces affect our behaviors.

REFERENCES

• 1: Harashima-Naemura Lab., University of Tokyo. http://www.hc.ic.i.u-tokyo.ac.jp/index.php.
• 2: Hasegawa, O. and Sakaue, K. A CG tool for constructing anthropomorphic interface agents. In Proc. IJCAI-97, (1997), 23--26.
• 3: Kanwisher, N., McDermott, J. and Chun, M.M. The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience 17, 11 (1997), 4302--4311.
• 4: Kawashima, R., Itoh, M., Hatazawa, J., Miyazawa, H., Yamada, K., Matsuzawa, T. and Fukuda, H. Changes of regional cerebral blood flow during listening to an unfamiliar spoken language. Neuroscience Letters 161 (1993), 69--72.
• 5: Koda, T. Agents with Faces: A Study on the Effects of Personification of Software Agents. http://alumni.media.mit.edu/~tomoko/home-page.html.
• 6: McCloud, S. Understanding Comics. HarperPerennial, (1993).
• 7: Monsters of Photorealism.http://www.wired.com/news/culture/0,1284,69739,00.html.
• 8: Nakamura, K. Activation of the right inferior frontal cortex during assessment of facial emotion. Advances in Neurological Sciences 43, 4 (1999), 519--527.
• 9: Takehara, T., Kuribayashi, Y., Mukawa, N., Mizuoka, I. and Takinami, E. The effect of emotionally inconsistent face marks embedded in emotional messages. Journal of Japanese Academy of Facial Studies 5, 1 (2005), 75--89.
• 10: The Uncanny Valley. http://www.arclight.net/~pdb/nonfiction/uncanny-valley.html.
• 11: Tong, F., Nakayama, K., Moscovitch, M., Weinrib, O. and Kanwisher, N. Response properties of the human fusiform face area. Cognitive Neuropsychology 17 (1/2/3), (2000), 257--279.
• 12: Yuasa, M., Yasumura, Y. and Nitta, K. A tool for animated agents in network-based negotiation. In Proc. RO-MAN'2003. 4B3, (2003), 259--264.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Keywords:
brain activity, emoticon, face mark, facial expression, functional MRI, life-like character, nonverbal communication, personified agent