E-tree: emotionally driven augmented reality art

ABSTRACT

In this paper, we describe an Augmented Reality Art installation, which reacts to user behaviour using Multimodal analysis of affective signals. The installation features a virtual tree, whose growth is influenced by the perceived emotional response from spectators. The system implements a 'magic mirror' paradigm (using a large-screen display or projection system) and is based on the ARToolkit with extended representations for scene graphs. The system relies on a PAD dimensional model of affect to support the fusion of different affective modalities, while also supporting the representation of affective responses that relate to aesthetic impressions. The influence of affective input on the visual component is achieved by mapping affective data to an L-System governing virtual tree behaviour. We have performed an early evaluation of the system, both from the technical perspective and in terms of user experience. Post-hoc questionnaires were generally consistent with data from multimodal affective processing, and users rated the overall experience as positive and enjoyable, regardless of how proactive they were in their interaction with the installation.

REFERENCES

• 1: Marc Cavazza , Jean-Luc Lugrin , Simon Hartley , Paolo Libardi , Matthew J. Barnes , Mikael Le Bras , Marc Le Renard , Louis Bec , Alok Nandi, New ways of worldmaking: the Alterne platform for VR art, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027542]
• 2: Stephen W. Gilroy, Marc Cavazza, Rémi Chaignon, Satu-Marja Mäkelä, Markus Niiranen, Elisabeth André, Thurid Vogt, Mark Billinghurst, Hartmut Seichter, and Maurice Benayoun. 2007. An Emotionally Responsive AR Art Installation. Proceedings of 6th International Symposium on Mixed and Augmented Reality (ISMAR 2007), Workshop on Mixed Reality, Entertainment and Art, pp.61--66.
• 3: Maurice Benayoun. http://www.benayoun.com/works.php?id=01000
• 4: Przemyslaw Prusinkiewicz , Aristid Lindenmayer, The algorithmic beauty of plants, Springer-Verlag New York, Inc., New York, NY, 1996
• 5: Mehrabian, A., & Russell, J. A. 1974. An approach to environmental psychology. Cambridge, MA: MIT. J. A. Russell. 1989. Measures of Emotion. In Emotion: Theory, Research and Experience, ch.4, pp.83--111. Academic Press, Inc.
• 6: J. A. Russell. 1989. Measures of Emotion. In Emotion: Theory, Research and Experience, ch.4, pp.83--111. Academic Press, Inc.
• 7: Mehrabian, A. 1995. Framework for a comprehensive description and measurement of emotional states. Genetic, Social, and General Psychology Monographs, 121, 339--361.
• 8: Russell. J. A. 1980. A circumplex model of affect. Journal of personality and social psychology, 39. 1161--1178
• 9: J. Looser, R. Grasset, H. Seichter, M. Billinghurst. 2006. OSGART - A Pragmatic Approach to MR. In Industrial Workshop at ISMAR 2006, Santa Barbara, California, USA, October, 2006.
• 10: Hirokazu Kato , Mark Billinghurst, Marker Tracking and HMD Calibration for a Video-Based Augmented Reality Conferencing System, Proceedings of the 2nd IEEE and ACM International Workshop on Augmented Reality, p.85, October 20-21, 1999
• 11: Don Burns , Robert Osfield, Open Scene Graph A: Introduction, B: Examples and Applications, Proceedings of the IEEE Virtual Reality 2004, p.265, March 27-31, 2004 [doi>10.1109/VR.2004.57]
• 12: Rosalind W. Picard, Affective computing, MIT Press, Cambridge, MA, 1997
• 13: Bryant, J., Vorderer,. P. (Eds.) 2006. Psychology of Entertainment. Lawrence Erlbaum Associates, Inc.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

General Terms:
Design, Experimentation, Theory

Keywords:
affective computing, augmented reality, interactive digital arts, multimodal interaction