Spoken words: activating text-to-speech through eye closure

ABSTRACT

Writing is a predominantly visual task. In the following, we describe a system that adds a new modality to computer-supported text editing: Listening with closed eyes.

We present a prototype, for which we combined a text-to-speech software, Microsoft Word and a face tracking system. It reads out the current sentence in Microsoft Word as soon as the user closes both eyes.

In this paper we discuss further details of the concept and its background, describe the implementation of the prototype and disclose first user testing results, before we conclude our findings with an outlook on future research.

REFERENCES

• 1: Ergonomic Problems of Eye Strain, http://www.safecomputingtips.com/ergonomic-problems-eye.html
• 2: Dealing With Eyestrain, http://www.upmc.com/HealthManagement/ManagingYourHealth/PersonalHealth/Men/~chunkiid=14167
• 3: Evinger C, Manning KA, Sibony PA, Eyelid Movements: Mechanisms and Normal Data. Investigative Ophthalmology and Visual Science (32), pp. 387--400, 1991
• 4: Meronk F, Atlas of Eyelid and Facial Anatomy, http://www.drmeronk.com/blephnews/anatomy.html
• 5: Davis et al., Human Brain Potentials During the Onset of Sleep. Journal of Neurophysiology (1), pp. 24--38, 1938
• 6: Buddhism in a Nutshell: Anapana Sati, http://www.accesstoinsight.org/lib/authors/narada/nutshell.html
• 7: Ways of Christ: Christian Meditation, http://www.ways-of-christ.net/topics/meditation.htm
• 8: Shema Israel: Jewish Prayer and Ritual, http://www.jewishmag.com/49mag/shema/shema.htm
• 9: Tresniowski A, Arias R, The Boy Who Sees With Sound. People Magazine (6), pp. , 2006.
• 10: Gougoux et al., Pitch discrimination in the early blind. Nature (430), p. 309, 2004.
• 11: Stephen Brewster , Joanna Lumsden , Marek Bell , Malcolm Hall , Stuart Tasker, Multimodal 'eyes-free' interaction techniques for wearable devices, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642694]
• 12: Yacine Bellik , Dominique Burger, Multimodal interfaces: new solutions to the problem of computer accessibilty for the blind, Conference companion on Human factors in computing systems, p.267-268, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/259963.260482]
• 13: Opera Multimodal Browser, http://www.opera.com/products/devices/multimodal/
• 14: Multimodal Web Applications with XHTML+Voice, http://www-306.ibm.com/software/pervasive/multimodal/
• 15: Tobii Technology AB, http://www.tobii.com/
• 16: SR Research Ltd., http://www.sr-research.com/
• 17: Morimoto CH, Koons D, Amir A, Flickner M, Pupil detection and tracking using multiple light sources. Image and Vision Computing (18), pp. 331--335, 2000.
• 18: Rszkin A, Visual Feedback: Why Modes Kill, http://www.humanized.com/weblog/2006/12/07/is_visual_feedback_enough_why_modes_kill/
• 19: NaturalSoft Ltd., Natural Reader, http://www.naturalreaders.com/
• 20: Microsoft Corporation, Microsoft Word, http://office.microsoft.com/en-us/word/default.aspx
• 21: Froeba B, Kueblbeck C, Face Detection and Tracking using Edge Orientation Information, SPIE Visual Communications and Image Processing (4310), pp. 583--594, 2001.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Design, Human Factors

Keywords:
audiovisual, eye closure, face tracking, multimodal interfaces, text editing