The spatial resolution of crossmodal attention: Implications for the design of multimodal interfaces

ABSTRACT

Previous research on crossmodal attentional orienting has reported speeded reaction times (RT) when the stimuli from the different modalities are in the same spatial location and slowed RTs when the stimuli are presented in very different locations (e.g., opposite sides of the body). However, little is known about what occurs for spatial interactions between these two extremes. We systematically varied the separation between cues and targets to quantify the spatial distribution of crossmodal attention. The orthogonal cueing paradigm [Spence et al. 1998] was used. Visual targets presented above or below the forearm were preceded by either vibrotactile cues presented on the forearm, auditory cues presented below the forearm, or visual cues presented on the forearm. The presentation of both unimodal and crossmodal cues led to a roughly monotonic increase in RT as a function of the cue-target separation. Unimodal visual cueing resulted in an attentional focus that was significantly narrower than that produced by crossmodal cues: the distribution of visual attention for visual cues had roughly half of the lateral extent of that produced by tactile cueing and roughly one fourth of the lateral extent as that produced by auditory cueing. This occurred when both seven (Experiment 1) and three (Experiment 2) cue locations were used suggesting that the effects are not primarily due to differences in the ability to localize the cues. These findings suggest that the location of tactile and auditory warning signals does not have to be controlled as precisely as the location of visual warning signals to facilitate a response to the critical visual event.

REFERENCES

• 1: Chastain, G. 1992. Analog versus discrete shifts of attention across the visual field. Psych. Resear. 54, 175--181.
• 2: Colonius, H. and Adndt, P. 2001. A two-stage model for visual-auditory interaction in saccadic latencies. Percep. Psychophy. 63, 126--147.
• 3: Diederich, A. and Colonius, H. 2007. Modeling spatial effects in visual-tactile saccadic reaction time. Percep. Psychophy.
• 4: Diederich, A., Colonius, H., Bockhorst, D., and Tabeling, S. 2003. Visual-tactile spatial interaction in saccade generation. Exper. Brain Resear. 148, 328--337.
• 5: Downing, C. J. and Pinker, S. 1985. The spatial structure of visual attention. In M. I. Posner and O. S. M. Martin, Eds. Attention and Performance XI. Erlbaum, Hillsdale, NJ, 171--188.
• 6: Driver, J. and Spence, C. 1998. Crossmodal links in spatial attention. Philosoph. Trans. the Royal Society Series B 353, 1319--1331.
• 7: Edworthy, J. and Adams, A. 1996. Warning Design: A Research Prospective. Taylor & Francis, Bristol, PA.
• 8: Edworthy, J., Loxley, S., and Dennis, I. 1991. Improving auditory warning design: relationship between warning sound parameters and perceived urgency. Human Factors 33, 205--231.
• 9: Eimer, M., and Van Velzen, J. 2005. Spatial tuning of tactile attention modulates visual processing within hemifields: an ERP investigation of crossmodal attention. Exper. Brain Resear. 166, 402--410.
• 10: Farah, M. J., Wong, A. B., Monheit, M. A., and Morrow, L. A. 1989. Parietal lobe mechanisms of spatial attention: Modality-specific or supramodal? Neuropsychologica 27, 461--470.
• 11: Ferris, T., Penfold, R., Hameed, S., and Sarter, N. 2006. The implications of crossmodal links in attention for the design of multimodal interfaces: a driving simulation study. In Proceedings of the 50th Annual Meeting on Human Factors and Ergonomics Society. HFES, Santa Monica, CA, 406--409.
• 12: Groh, J. M. and Sparks, D. L. 1996. Saccades to somatosensory targets II. Motor convergence in the primate superior colliculus. J. Neurophysiology 75, 428--438.
• 13: Ho, C., and Spence, C. 2005a. Assessing the effectiveness of various auditory cues in capturing a driver's visual attention. J. Exper. Psych.: Applied 11, 157--174.
• 14: Ho, C., Tan, H. Z., and Spence, C. 2005. Using spatial vibrotactile cues to direct visual attention in driving scenes. Transport. Resear. Part F: Traffic Psych. Behav. 8, 397--412.
• 15: Ho, C., Tan, H. Z., and Spence, C. 2006. The differential effect of vibrotactile and auditory cues on visual spatial attention. Ergonomics 49, 724--738.
• 16: Chanon M. Jones , J. Jay Young , Rob Gray , Charles Spence , Hong Z. Tan, An Eyetracker Study of the Haptic Cuing of Visual Attention, Proceedings of the Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.557-558, March 22-24, 2007 [doi>10.1109/WHC.2007.22]
• 17: Kennett, S., Spence, C., and Driver, J. 2002. Visuo-tactile links in covert exogenous spatial attention remap across changes in unseen hand posture. Percep. Psychophy. 64, 1083--1094.
• 18: Rupert, A. H. 2000a. An instrumentation solution for reducing spatial disorientation mishaps—A more “natural” approach to maintaining spatial orientation. IEEE Engineering Med. Biology Mag. 19, 71--80.
• 19: Rupert, A. H. 2000b. Tactile situation awareness system: Proprioceptive prostheses for sensory deficiencies. Aviation Space Environ. Medicine 71, A92--A99.
• 20: Shepherd, M. and Muller, H. J. 1989. Movement versus focusing of visual attention. Percept. Psychophy. 46, 146--154.
• 21: Shulman, G. L., Remington, R. W., and McLean, J. P. 1979. Moving attention through physical space. J. Exper. Psych.-Hum. Percep. Perform. 5, 522--526.
• 22: Shulman, G. L., Wilson, J., and Sheehey, J. B. 1985. Spatial determinants of the distribution of attention. Percep. Psychophy. 37, 59--65.
• 23: Spence, C. 2001. Crossmodal attentional capture: A controversy resolved? In C. Folk and B. Gibson, Eds. Attention, distraction and action: Multiple perspectives on attentional capture. Advances in Psychology, 133. Elsevier Science, Amsterdam, 231--262.
• 24: Spence, C. and Driver, J. 1996. Audiovisual links in endogenous covert spatial attention. J. Exper. Psych. Hum. Percep. Perform. 22, 1005--1030.
• 25: Spence, C. and Driver, J. 1997. Cross-modal links in attention between audition, vision, and touch: Implications for interface design. Int. J. Cognitive Ergonomics 1, 351--373.
• 26: Spence, C. Nicholls, M. E. R., Gillespie, N., and Driver, J. 1998. Cross-modal links in exogenous covert spatial orienting between touch, audition, and vision. Percep. Psychophy. 60, 544--557.
• 27: Spence, C., Pavaini, F., and Driver, J. 2000. Crossmodal links between vision and touch incovert endogenous spatial attention. J. Exper. Psych.-Hum. Percep. Perform. 26, 1298--1319.
• 28: Stein, B. M. and Meredith, M. A. 1993. The Merging of the Senses. MIT Press, Cambridge, MA
• 29: Tan, H. Z., Gray, R., Young, J. J., and Traylor, R. 2003. A haptic back display for attentional and directional cueing. Haptics-e: Electron. J. Haptics Resear. 3.
• 30: Wickens, C. D. 1980. The structure of attentional resources. In Nickerson, R. S., Ed. Attention and Performance VIII. Lawrence Erlbaum Associates, Hillsdale, NJ, 239--254.
• 31: Wiener, E. L. and Nagel, D. C. 1988. Human Factors in Aviation. Academic Press, New York

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Experimentation, Human Factors

Keywords:
Attention, warnings