Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality

Subscribe (Full Service)


	Search: The ACM Digital Library The Guide

	Feedback

Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality

Full text

Pdf (4.26 MB)

Source

ACM Transactions on Applied Perception (TAP) archive
Volume 6 , Issue 2 (February 2009) table of contents

Article No. 7

Year of Publication: 2009

ISSN:1544-3558

Authors

Bernhard E. Riecke	Max Planck Institute for Biological Cybernetics
Aleksander Väljamäe	Chalmers University of Technology, Gothenburg, Sweden
Jörg Schulte-Pelkum	Max Planck Institute for Biological Cybernetics

Publisher

ACM New York, NY, USA

Bibliometrics

Downloads (6 Weeks): 40, Downloads (12 Months): 271, Citation Count: 0

Additional Information:

abstract references index terms collaborative colleagues

Tools and Actions:	Request Permissions Review this Article Save this Article to a Binder Display Formats: BibTeX EndNote ACM Ref

DOI Bookmark:	Use this link to bookmark this Article: http://doi.acm.org/10.1145/1498700.1498701 What is a DOI?

ABSTRACT

While rotating visual and auditory stimuli have long been known to elicit self-motion illusions (“circular vection”), audiovisual interactions have hardly been investigated. Here, two experiments investigated whether visually induced circular vection can be enhanced by concurrently rotating auditory cues that match visual landmarks (e.g., a fountain sound). Participants sat behind a curved projection screen displaying rotating panoramic renderings of a market place. Apart from a no-sound condition, headphone-based auditory stimuli consisted of mono sound, ambient sound, or low-/high-spatial resolution auralizations using generic head-related transfer functions (HRTFs). While merely adding nonrotating (mono or ambient) sound showed no effects, moving sound stimuli facilitated both vection and presence in the virtual environment. This spatialization benefit was maximal for a medium (20° × 15°) FOV, reduced for a larger (54° × 45°) FOV and unexpectedly absent for the smallest (10° × 7.5°) FOV. Increasing auralization spatial fidelity (from low, comparable to five-channel home theatre systems, to high, 5° resolution) provided no further benefit, suggesting a ceiling effect. In conclusion, both self-motion perception and presence can benefit from adding moving auditory stimuli. This has important implications both for multimodal cue integration theories and the applied challenge of building affordable yet effective motion simulators.

REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

	1	Algazi, V. R., Duda, R. O., Thompson, D. M., and Avendano, C. 2001. The CIPIC HRTF database. In Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics. IEEE, Los Alamitos, CA, 99--102.
	2	Best, V., Van Schaik, A., Jin, C., and Carlile, S. 2005. Auditory spatial perception with sources overlapping in frequency and time. Acta Acustica 91, 421--428.
	3	Boer, E. R., Girshik, A. R., Yamamura, T., and Kuge, N. 2000. Experiencing the same road twice: A driver-centred comparison between simulation and reality. In Proceedings of the Driving Simulation Conference (DSC'00). Elsevier, New York.
	4	Brandt, T., Dichgans, J., and Held, R. 1973. Optokinesis affects body posture and subjective visual vertical. Pflugers Archiv-European J. Physiol. 339, 97--97.
	5	Brandt, T., Dichgans, J., and Koenig, E. 1973. Differential effects of central versus peripheral vision on egocentric and exocentric motion perception. Exp. Brain Res. 16, 476--491.
	6	Brungart, D., Simpson, B., and Kordik, A. 2005. Localization in the presence of multiple simultaneous sounds. Acta Acustica 91, 3, 471--479.
	7	Burki-Cohen, J., Go, T. H., Chung, W. Y., Schroeder, J., Jacobs, S., and Longridge, T. 2003. Simulator fidelity requirements for airline pilot training and evaluation continued: An update on motion requirements research. In Proceedings of the 12th International Symposium on Aviation Psychology. 182--189.
	8	K. Cater , A. Chalmers , G. Ward, Detail to attention: exploiting visual tasks for selective rendering, Proceedings of the 14th Eurographics workshop on Rendering, June 25-27, 2003, Leuven, Belgium
	9	Sarah S. Chance , Florence Gaunet , Andrew C. Beall , Jack M. Loomis, Locomotion Mode Affects the Updating of Objects Encountered During Travel: The Contribution of Vestibular and Proprioceptive Inputs to Path Integration, Presence: Teleoperators and Virtual Environments, v.7 n.2, p.168-178, April 1998 [doi>10.1162/105474698565659]
	10	Dichgans, J. and Brandt, T. 1978. Visual-vestibular interaction: Effects on self-motion perception and postural control. In Perception. Handbook of Sensory Physiology, vol. VIII. Springer, New York, 756--804.
	11	Dodge, R. 1923. Thresholds of rotation. J. Exp. Psychol. 6, 107--137.
	12	Durgin, F. H., Pelah, A., Fox, L. F., Lewis, J. Y., Kane, R., and Walley, K. A. 2005. Self-motion perception during locomotor recalibration: More than meets the eye. J. Exp. Psychol. 31, 3, 398--419.
	13	Durlach, N. I. and Mavor, A. S., Eds. 1995. Virtual Reality: Scientific and Technological Challenges. National Academy Press, Washington, DC.
	14	Ernst, M. O. and Bülthoff, H. H. 2004. Merging the senses into a robust percept. Trends Cog. Sci. 8, 4, 162--169.
	15	Fischer, M. H. and Kornmüller, A. E. 1930. Optokinetisch ausgelöste Bewegungswahrnehmung und optokinetischer Nystagmus {Optokinetically induced motion perception and optokinetic nystagmus}. Journal für Psychologie und Neurologie, 273--308.
	16	Flanagan, M. B., May, J. G., and Dobie, T. G. 2004. The role of vection, eye movements and postural instability in the etiology of motion sickness. J. Vestibular Res. Equilibrium Orientation 14, 4, 335--346.
	17	Gardner, W. and Martin, K. 1995. Hrtf measurements of a KEMAR. J. Acoust. Soc. Am. 97, 3907--3908.
	18	Gekhman, B. 1991. Audiokinetic nystagmus. Sensornye Sistemy 5, 2, 71--78. (In Russian.)
	19	Hendrix, C. and Barfield, W. 1996. The sense of presence within auditory virtual environments. Presence Teleoperators Virtual Environ. 5, 3, 290--301.
	20	Hennebert, P. E. 1960. Audiokinetic nystagmus. J. Auditory Res. 1, 1, 84--87.
	21	Hettinger, L. J. 2002. Illusory self-motion in virtual environments. In Handbook of Virtual Environments, K. M. Stanney, Ed. Lawrence Erlbaum, Mahwah, NJ, 471--492.
	22	Hollerbach, J. M. 2002. Locomotion interfaces. In Handbook of Virtual Environments, K. M. Stanney, Ed. Lawrence Erlbaum, Mahwah, NJ, 239--254.
	23	Howard, I. P. 1982. Human Visual Orientation. J. Wiley, Hoboken, NJ.
	24	Howard, I. P. 1986. The perception of posture, self motion, and the visual vertical. In Handbook of Human Perception and Performance, vol. 1, K. R. Boff, L. Kaufman, and J. P. Thomas, Eds. Wiley, Hoboken, NJ, 18.1--18.62.
	25	Kapralos, B., Zikovitz, D., Jenkin, M., and Harris, L. 2004. Auditory cues in the perception of self-motion. In Proceedings of the 116th AES Convention.
	26	Kleiner, M. and Dalenbäck, B-I. Andsvensson, P. 1993. Auralization -- an overview. J. Audio Engin. Soc. 41, 11, 861--875.
	27	Lackner, J. R. 1977. Induction of illusory self-rotation and nystagmus by a rotating sound-field. Aviat. Space Environ. Med. 48, 2, 129--131.
	28	Langendijk, E. H. A. and Bronkhorst, A. W. 2000. Fidelity of three-dimensional-sound reproduction using a virtual auditory display. Acoustical Soc. Am. J. 107, 528--537.
	29	Larsson, P., Väljamäe, A., Västfjäll, D., Tajadura-Jiménez, A., and Kleiner, M. 2009. Auditory-induced presence in mixed presence reality environments and related technology. In The Engineering of Mixed Reality Systems. E. Dubois, L. Nigay, and P. Gray, Eds. Springer.
	30	Larsson, P., Västfjäll, D., Tajadura and Kleiner, M. 2004. Perception of self-motion and presence in auditory virtual environments. In Proceedings of 7th Annual Workshop of Presence. 252--258. http://www.kyb.mpg.de/publication.html?publ=2953.
	31	Lepecq, J. C., Giannopulu, I., and Baudonniere, P. M. 1995. Cognitive effects on visually induced body motion in children. Perception 24, 4, 435--449.
	32	Lombard, M. and Ditton, T. 1999. At the heart of it all: The concept of presence. J. Comput. Mediated Comm. 3, 2.
	33	Luciani, A. 2004. Dynamics as a common criterion to enhance the sense of presence in virtual environments. In Proceedings of 7th Annual Workshop Presence 2004. 96--103.
	34	Mach, E. 1875. Grundlinien der Lehre von der Bewegungsempfindung. Engelmann, Leipzig, Germany.
	35	Marmekarelse, A. M. and Bles, W. 1977. Circular vection and human posture ii: Does the auditory-system play a role. Agressologie 18, 6, 329--333.
	36	McFarland, W. and Weber, B. 1969. An investigation of ocular response to various forms of sound field auditory stimulation. J. Auditory Res. 9, 3, 236--239.
	37	Thomas Moeck , Nicolas Bonneel , Nicolas Tsingos , George Drettakis , Isabelle Viaud-Delmon , David Alloza, Progressive perceptual audio rendering of complex scenes, Proceedings of the 2007 symposium on Interactive 3D graphics and games, April 30-May 02, 2007, Seattle, Washington [doi>10.1145/1230100.1230133]
	38	Max Mulder , M. M. van Paassen , Erwin R. Boer, Exploring the roles of information in the manual control of vehicular locomotion: from kinematics and dynamics to cybernetics, Presence: Teleoperators and Virtual Environments, v.13 n.5, p.535-548, October 2004 [doi>10.1162/1054746042545256]
	39	Ozawa, K., Chujo, Y., Suzuki, Y., and Sone, T. 2004. Psychological factors involved in auditory presence. Acoustical Sci. Tech. 24, 42--44.
	40	Pope, J. and Chalmers, A. 1999. Multi-sensory rendering: Combining graphics and acoustics. In Proceedings of the 7th International Conference in Central Europe on Computer Graphics. 233--242.
	41	Riecke, B. E., Feuereissen, D., and Rieser, J. J. 2009. Auditory self-motion simulation is facilitated by haptic and vibrational cues suggesting the possibility of actual motion. ACM Trans. Appl. Percept. Submitted.
	42	Bernhard E. Riecke , Daniel Feuereissen , John J. Rieser, Auditory self-motion illusions ("circular vection") can be facilitated by vibrations and the potential for actual motion, Proceedings of the 5th symposium on Applied perception in graphics and visualization, August 09-10, 2008, Los Angeles, California [doi>10.1145/1394281.1394309]
	43	Riecke, B. E., Feuereissen, D., and Rieser, J. J. 2008b. Contribution and interaction of auditory and biomechanical cues for self-motion illusions (“circular vection”). In Proceedings of the CyberWalk Workshop.
	44	Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., and Bülthoff, H. H. 2004. Enhancing the visually induced self-motion illusion (vection) under natural viewing conditions in virtual reality. In Proceedings of 7th Annual Workshop Presence 2004. 125--132. www.kyb.mpg.de/publication.html?publ=2864.
	45	Bernhard E. Riecke , Jörg Schulte-Pelkum , Marios N. Avraamides , Markus Von Der Heyde , Heinrich H. Bülthoff, Cognitive factors can influence self-motion perception (vection) in virtual reality, ACM Transactions on Applied Perception (TAP), v.3 n.3, p.194-216, July 2006 [doi>10.1145/1166087.1166091]
	46	Riecke, B. E., Schulte-Pelkum, J., and Bülthoff, H. H. 2005. Perceiving simulated ego-motions in virtual reality: Comparing large screen displays with HMDs. Proc. SPIE, Vol. 5666. SPIE, Bellingham, WA, 344--355.
	47	Riecke, B. E., Schulte-Pelkum, J., Caniard, F., and Bülthoff, H. H. 2005a. Influence of auditory cues on the visually-induced self-motion illusion (circular vection) in virtual reality. In Proceedings of 8th Annual Workshop Presence 2005. ACM, New York, 49--57.
	48	Bernhard E. Riecke , Jorg Schulte-Pelkum , Franck Caniard , Heinrich H. Bulthoff, Towards Lean and Elegant Self-Motion Simulation in Virtual Reality, Proceedings of the 2005 IEEE Conference 2005 on Virtual Reality, p.131-138, March 12-16, 2005 [doi>10.1109/VR.2005.83]
	49	Riecke, B. E., Västfjäll, D., Larsson, P., and Schulte-Pelkum, J. 2005. Top-down and multi-modal influences on self-motion perception in virtual reality. In Proceedings of the Internation Human-Computer Interaction Conference. HCI, Toronto, Ontario, 1--10.
	50	Riecke, B. E. and Wiener, J. M. 2007. Can people not tell left from right in vr? point-to-origin studies revealed qualitative errors in visual path integration. In Proceedings of IEEE Virtual Reality 2007. IEEE, Los Alamitos, CA, 3--10.
	51	Rumsey, F. 2002. Spatial quality evaluation for reproduced sound: Terminology, meaning, and a scene-based paradigm. J. Audio Engin. Soc. 50, 9, 651.
	52	Sakamoto, S., Osada, Y., Suzuki, Y., and Gyoba, J. 2004. The effects of linearly moving sound images on selfmotion perception. Acoustical Sci. Tech. 25, 100--102.
	53	Schinauer, T., Hellmann, A., and Höger, R. 1993. Dynamic acoustical stimulation affects self-motion perception. In Contributions to Psychological Acoustics. Results of the 6th Oldenburg Symposium on Psychological Acoustics, A. Schick, Ed. BIS, Oldenburg, Germany, 373--385.
	54	Thomas Schubert , Frank Friedmann , Holger Regenbrecht, The Experience of Presence: Factor Analytic Insights, Presence: Teleoperators and Virtual Environments, v.10 n.3, p.266-281, June 2001 [doi>10.1162/105474601300343603]
	55	Schulte-Pelkum, J. and Riecke, B. E. 2009. An integrative approach to presence and self-motion perception research. http://www.sfu.ca/~ber1/web/Shuite-PelkumRiecke_HOP_chapter.pdf.
	56	Schulte-Pelkum, J., Riecke, B. E., and Bülthoff, H. H. 2004. Vibrational cues enhance believability of ego-motion simulation. In International Multisensory Research Forum (IMRF). http://www.kyb.mpg.de/publication.html?publ=2766.
	57	Schulte-Pelkum, J., Riecke, B. E., von der Heyde, M., and Bülthoff, H. H. 2004. Influence of display device and screen curvature on perceiving and controlling simulated ego-rotations from optic flow. Tech. rep. 122, MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=2574.
	58	Soto-Faraco, S., Kingstone, A., and Spence, C. 2003. Multisensory contributions to the perception of motion. Neuropsychologia 41, 13, 1847--1862.
	59	Veronica Sundstedt , Kurt Debattista , Alan Chalmers, Selective rendering using task-importance maps, Proceedings of the 1st Symposium on Applied perception in graphics and visualization, August 07-08, 2004, Los Angeles, California [doi>10.1145/1012551.1012597]
	60	Desney S. Tan , Darren Gergle , Peter Scupelli , Randy Pausch, Physically large displays improve performance on spatial tasks, ACM Transactions on Computer-Human Interaction (TOCHI), v.13 n.1, p.71-99, March 2006 [doi>10.1145/1143518.1143521]
	61	Desney S. Tan , Darren Gergle , Peter G. Scupelli , Randy Pausch, Physically large displays improve path integration in 3D virtual navigation tasks, Proceedings of the SIGCHI conference on Human factors in computing systems, p.439-446, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985748]
	62	Väljamäe, A., Larsson, P., Tajadura-Jiménez, A., Västfjäll, D., and Kleiner, M. 2008. Auditory cues for multisensory enhancement of self-motion perception. IEEE Multimedia.
	63	Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2004. Auditory presence, individualized head-related transfer functions, and illusory ego-motion in virtual environments. In Proceedings of 7th Annual Workshop of Presence. 141--147.
	64	Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2005. Travelling without moving: Auditory scene cues for translational self-motion. In Proceedings of the 11th Meeting of International Conference on Auditory Display (ICAD 2005).
	65	Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2006. Vibrotactile enhancement of auditory induced self-motion and spatial presence. J. Acoustic Engin. Soc. 54, 10, 954--963.
	66	Väljamäe, A., Larsson, P., Västfjäll, D., and Kleiner, M. 2008a. Auditory landmarks enhance multimodal circular vection simulations in virtual reality. J. Acoustic Engin. Soc. Submitted.
	67	Aleksander Väljamäe , Pontus Larsson , Daniel Västfjäll , Mendel Kleiner, Sound representing self-motion in virtual environments enhances linear vection, Presence: Teleoperators and Virtual Environments, v.17 n.1, p.43-56, February 2008 [doi>10.1162/pres.17.1.43]
	68	Väljamäe, A., Västfjäll, D., Larsson, P., and Kleiner, M. 2008. Perceived sound in mediated environments. In Immersed in Media Experiences: Presence Psychology and Design (Handbook of Presence). Lawrence Erlbaum, Mahwah, NJ.
	69	Van Der Steen, F. A. M. and Brockhoff, P. T. M. 2000. Induction and impairment of saturated yaw and surge vection. Percep. Psychophys. 62, 1, 89--99.
	70	Von Helmholtz, H. 1896. Handbuch der physiologischen Optik. L. Voss, Hamburg-Leipzig.
	71	Warren, R. and Wertheim, A. H., Eds. 1990. Perception & Control of Self-Motion. Erlbaum, Mahwah, NJ.
	72	Watson, J. E. 1968. Evaluation of audiokinetic nystagmus as as test of auditory sensitivity. J. Auditory Res. 8, 161--165.
	73	Wright, W. G., Dizio, P., and Lackner, J. R. 2006. Perceived self-motion in two visual contexts: dissociable mechanisms underlie perception. J. Vestib. Res. 16, 1--2, 23--28.