A classification scheme for multi-sensory augmented reality

ABSTRACT

We present a new classification framework for describing augmented reality (AR) applications based on where the mixing of real and computer-generated stimuli takes place. In addition to "classical" visual AR techniques, such as optical-see-through and video-see-through AR, our framework encompasses AR directed at the other senses as well. This "axis of mixing location" is a continuum ranging from the physical environment to the human brain. There are advantages and disadvantages of mixing at different points along the continuum, and while there is no "best" location, we present sample usage scenarios that illustrate the expressiveness of this classification approach.

REFERENCES

• 1: Mark Billinghurst , Hirokazu Kato, Collaborative augmented reality, Communications of the ACM, v.45 n.7, July 2002 [doi>10.1145/514236.514265]
• 2: Jochen Ehnes , Koichi Hirota , Michitaka Hirose, Projected Augmentation - Augmented Reality using Rotatable Video Projectors, Proceedings of the 3rd IEEE/ACM International Symposium on Mixed and Augmented Reality, p.26-35, November 02-05, 2004 [doi>10.1109/ISMAR.2004.47]
• 3: Steven Feiner , Blair MacIntyre , Marcus Haupt , Eliot Solomon, Windows on the world: 2D windows for 3D augmented reality, Proceedings of the 6th annual ACM symposium on User interface software and technology, p.145-155, December 1993, Atlanta, Georgia, United States [doi>10.1145/168642.168657]
• 4: Masaaki Fukumoto , Yoshinobu Tonomura, Whisper: a wristwatch style wearable handset, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.112-119, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303009]
• 5: Yuki Hashimoto , Naohisa Nagaya , Minoru Kojima , Satoru Miyajima , Junichiro Ohtaki , Akio Yamamoto , Tomoyasu Mitani , Masahiko Inami, Straw-like user interface: virtual experience of the sensation of drinking using a straw, Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology, June 14-16, 2006, Hollywood, California [doi>10.1145/1178823.1178873]
• 6: Hirose, M., Tanikawa, T., Ishida, K. A Study of Olfactory Display, Proc. of the Virtual Reality Soc. of Japan 2nd Annual Conf. 1997 (in Japanese), 155--158.
• 7: Masahiko Inami , Naoki Kawakami , Dairoku Sekiguchi , Yasuyuki Yanagida , Taro Maeda , Susumu Tachi, Visuo-Haptic Display Using Head-Mounted Projector, Proceedings of the IEEE Virtual Reality 2000 Conference, p.233, March 18-22, 2000
• 8: Hiroo Iwata , Hiroaki Yano , Takahiro Uemura , Tetsuro Moriya, Food Simulator: A Haptic Interface for Biting, Proceedings of the IEEE Virtual Reality 2004, p.51, March 27-31, 2004 [doi>10.1109/VR.2004.40]
• 9: W. Lindeman , Yasuyuki Yanagida , Haruo Noma , Kenichi Hosaka, Wearable vibrotactile systems for virtual contact and information display, Virtual Reality, v.9 n.2, p.203-213, January 2006 [doi>10.1007/s10055-005-0010-6]
• 10: Maynes-Aminzade, D. Edible Bits: Seamless Interfaces Between People, Data, and Food, ACM CHI 2005 Extended Abstracts, 2207--2210.
• 11: Milgram, P., Takemura, H., Utsumi, A., Kishino, F. Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum, SPIE Vol. 2351, Telemanipulator and Telepresence Technologies, 1994, 282--292.
• 12: Mori, M. The Uncanny Valley, Energy, 7(4), 1970, 33--35 (translated by K. F. MacDorman and T. Minato)
• 13: Nakamoto T., Min, P. H. D. Improvement of Olfactory Display Using Solenoid Valves, Proc. of IEEE Virtual Reality 2007, 179--186.
• 14: Pryor, H. L., Furness, T. A., Viirre, E. The Virtual Retinal Display: A New Display Technology Using Scanned Laser Light, Proc. of Human Factors and Ergonomics Soc., 42nd Annual Meeting, 1998, 1570--1574.
• 15: Tomoya Yamada , Satoshi Yokoyama , Tomohiro Tanikawa , Koichi Hirota , Michitaka Hirose, Wearable Olfactory Display: Using Odor in Outdoor Environment, Proceedings of the IEEE conference on Virtual Reality, p.199-206, March 25-29, 2006 [doi>10.1109/VR.2006.147]
• 16: Yasuyuki Yanagida , Shinjiro Kawato , Haruo Noma , Akira Tomono , Nobuji Tetsutani, Projection-Based Olfactory Display with Nose Tracking, Proceedings of the IEEE Virtual Reality 2004, p.43, March 27-31, 2004 [doi>10.1109/VR.2004.62]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

Keywords:
audio, augmented reality, gustatory, haptics, olfactory, video