A comparative psychophysical and EEG study of different feedback modalities for HRI

ABSTRACT

This paper presents a comparison between six different ways to convey navigational information provided by a robot to a human. Visual, auditory, and tactile feedback modalities were selected and designed to suggest a direction of travel to a human user, who can then decide if he agrees or not with the robot's proposition. This work builds upon a previous research on a novel semi-autonomous navigation system in which the human supervises an autonomous system, providing corrective monitoring signals whenever necessary.

We recorded both qualitative (user impressions based on selected criteria and ranking of their feelings) and quantitative (response time and accuracy) information regarding different types of feedback. In addition, a preliminary analysis of the influence of the different types of feedback on brain activity is also shown. The result of this study may provide guidelines for the design of such a human-robot interaction system, depending on both the task and the human user.

REFERENCES

• 1: Antonis Argyros , Pantelis Georgiadis , Panos Trahanias , Dimitris Tsakiris, Semi-autonomous Navigation of a Robotic Wheelchair, Journal of Intelligent and Robotic Systems, v.34 n.3, p.315-329, July 2002 [doi>10.1023/A:1016371922451]
• 2: J. D. Bayliss and D. H. Ballard A virtual reality testbed for brain-computer interface research. IEEE Trans Rehab Eng, 8(2):188--190, 2000.
• 3: Stephen Brewster , Lorna M. Brown, Tactons: structured tactile messages for non-visual information display, Proceedings of the fifth conference on Australasian user interface, p.15-23, January 01, 2004, Dunedin, New Zealand
• 4: Stephen A. Brewster , Peter C. Wright , Alistair D. N. Edwards, An evaluation of earcons for use in auditory human-computer interfaces, Proceedings of the INTERCHI '93 conference on Human factors in computing systems, p.222-227, May 1993, Amsterdam, The Netherlands
• 5: E. Donchin, K. Spencer, and R. Wijesinghe. The mental prosthesis: assessing the speed of a P300-based brain-computer interface. IEEE Trans Rehab Eng, 8(2):174--179, 2000.
• 6: EAI. Engineering Acoustics, INC. http://www.eaiinfo.com, 2005.
• 7: P. Ferrez and J. d. R. Millán. You Are Wrong!--Automatic Detection of Interaction Errors from Brain Waves. In Proc of the 19th Int Joint Conf on Artificial Intelligence, Edinburgh, UK, 2005.
• 8: P. Ferrez and J. d. R. Millán. Error-Related EEG Potentials Generated During Simulated Brain-Computer Interaction. IEEE Trans Biomed Eng, To appear.
• 9: Thilo Hinterberger , Gerold Baier, Parametric Orchestral Sonification of EEG in Real Time, IEEE MultiMedia, v.12 n.2, p.70-79, April 2005 [doi>10.1109/MMUL.2005.36]
• 10: T. Hinterberger, G. Baier, J. Mellinger, and N. Birbaumer. Auditory feedback of human EEG for direct brain-computer communication. In Proc of the 10th Meeting of the Int Conf on Auditory Display (ICAD 04), 2004.
• 11: T. Hinterberger, N. Neumann, M. Pham, A. Kübler, A. Grether, N. Hofmayer, B. Wilhelm, H. Flor, and N. Birbaumer. A multimodal brain-based feedback and communication system. Experimental Brain Research, 154(4):521--526, 2004.
• 12: L. Kauhanen, T. Palomäki, P. Jylänki, F. Aloise, M. Nuttin, and J. d. R. Millán. Haptic Feedback Compared With Visual Feedback For BCI. In Proc of the 3rd Int Brain-Computer Interface Workshop & Training Course, Graz, 2006. Verlag der TU Graz.
• 13: Robert W. Lindeman , John L. Sibert , Erick Mendez-Mendez , Sachin Patil , Daniel Phifer, Effectiveness of directional vibrotactile cuing on a building-clearing task, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055010]
• 14: D. McFarland, L. McCane, and J. Wolpaw. EEG-based communication and control: short-term role of feedback. IEEE Trans Rehab Eng, 6(1):7--11, 1998.
• 15: J. d. R. Millán, F. Renkens, J. Mourino, and W. Gerstner. Non-invasive brain-actuated control of a mobile robot by human EEG. IEEE Trans Biomed Eng, 51(6):1026--1033, 2004.
• 16: L. C. Parra, C. D. Spence, A. D. Gerson and P. Sajda. Recipes for the linear analysis of EEG. Neuroimage, 28(2):326--341, 2005.
• 17: X. Perrin, R. Chavarriaga, R. Siegwart, and J. d.. Millán. Bayesian Controller for a Novel Semi-Autonomous Navigation Concept. In 3rd European Conference on Mobile Robots (ECMR 2007), Freiburg, Germany, 2007.
• 18: B. Rebsamen, E. Burdet, C. Guan, H. Zhang, C. L. Teo, Q. Zeng, M. Ang, and C. Laugier. A Brain-Controlled Wheelchair Based on P300 and Path Guidance. In IEEE Int Conf on Biomedical Robotics and Biomechatronics (BioRob'06), pages 1101--1106, 2006.
• 19: E. W. Sellers and E. Donchin. A P300-based brain-computer interface: Initial tests by ALS patients. Clinical Neurophysiology, 117(3):538--548, 2006.
• 20: R. Simpson and S. Levine. Adaptative Shared Control of a Smart Wheelchair Operated by Voice Control. In Proc of the IEEE/RSJ Int Conf on Intelligent Robots and Systems (IROS '97), volume 2, pages 622--626, 1997.
• 21: H. J. Spiers and E. A. Maguire. Decoding human brain activity during real-world experiences. Trends in Cognitive Sciences, 11(8):356--365, 2007.
• 22: J. Wolpaw, N. Birbaumer, D. McFarland, G. Pfurtscheller, and T. M. Vaughan. Brain-Computer Interfaces for Communication and Control. Clinical Neurophysiology, 113(6):767--791, 2002.
• 23: J. Wolpaw, H. Ramoser, D. McFarland, and G. Pfurtscheller. EEG-based communication: improved accuracy by response verification. IEEE Trans. on Neural Systems and Rehabilitation, 6(3):326--333, 1998.
• 24: Holly A. Yanco, Wheelesley: A Robotic Wheelchair System: Indoor Navigation and User Interface, Assistive Technology and Artificial Intelligence, Applications in Robotics, User Interfaces and Natural Language Processing, p.256-268, January 1998

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.9 Robotics

General Terms:
Human Factors

Keywords:
auditory feedback, brain-computer interface, multimodal interaction, robot navigation, vibro-tactile feedback, visual feedback