Supporting medical conversations between deaf and hearing individuals with tabletop displays

ABSTRACT

This paper describes the design and evaluation of Shared Speech Interface (SSI), an application for an interactive multitouch tabletop display designed to facilitate medical conversations between a deaf patient and a hearing, non-signing physician. We employ a participatory design process involving members of the deaf community as well as medical and communication experts. We report results from an evaluation that compares conversation when facilitated by: (1) a digital table, (2) a human sign language interpreter, and (3) both a digital table and an interpreter. Our research reveals that tabletop displays have valuable properties for facilitating discussion between deaf and hearing individuals as well as enhancing privacy and independence. The contributions of this work include initial guidelines for cooperative group work technology for users with varying hearing abilities, discussion of benefits of participatory design with the deaf community, and lessons about using dictated speech on shared displays.

REFERENCES

• 1: Anna Cavender , Richard E. Ladner , Eve A. Riskin, MobileASL:: intelligibility of sign language video as constrained by mobile phone technology, Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility, October 23-25, 2006, Portland, Oregon, USA [doi>10.1145/1168987.1169001]
• 2: Clark, H. (2003). Pointing and Placing. In "Pointing: Where Language, Culture, and Cognition Meet." Edited by Sotaro Kita. Mihwah, NJ: Lawrence Erlbaum Associates, pp. 243--68.
• 3: CloudGarden. 2008. http://www.cloudgarden.com/
• 4: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 5: Dragon NaturallySpeaking Medical. 2008. www.nuance.com/dictaphone/emr
• 6: Fischer, G. and Sullivan, J. Human-centered public transportation systems for persons with cognitive disabilities -- Challenges and insights for participatory design. Proceedings of the Participatory Design Conference 2002, 194--198.
• 7: iCommunicator. 2008. www.myicommunicator.com
• 8: Java Speech API. 2008. java.sun.com/products/java-media/speech
• 9: Jefferson, G. (2004). Glossary of Transcript Symbols with an Introduction. In "Conversation Analysis: Studies from the First Generation." Jesse Benjamins Publishing Company, pp. 13--32.
• 10: Matthews, T., Carter, S., Pai, C., Fong, J., Mankoff, J. Scribe4Me: Evaluating a mobile sound transcription tool for the deaf. Proceedings of UbiComp 2006, 159--176.
• 11: Tara Matthews , Janette Fong , Jennifer Mankoff, Visualizing non-speech sounds for the deaf, Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, October 09-12, 2005, Baltimore, MD, USA [doi>10.1145/1090785.1090797]
• 12: McNeill, D. (1992). Hand & Mind: What Gestures Reveal about Thought. Chicago: University of Chicago Press.
• 13: Dorian Miller , Karl Gyllstrom , David Stotts , James Culp, Semi-transparent video interfaces to assist deaf persons in meetings, Proceedings of the 45th annual southeast regional conference, March 23-24, 2007, Winston-Salem, North Carolina [doi>10.1145/1233341.1233431]
• 14: Karyn Moffatt , Joanna McGrenere , Barbara Purves , Maria Klawe, The participatory design of a sound and image enhanced daily planner for people with aphasia, Proceedings of the SIGCHI conference on Human factors in computing systems, p.407-414, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985744]
• 15: Meredith Ringel Morris , Dan Morris , Terry Winograd, Individual audio channels with single display groupware: effects on communication and task strategy, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031646]
• 16: Meredith Ringel Morris , Kathy Ryall , Chia Shen , Clifton Forlines , Frederic Vernier, Beyond "social protocols": multi-user coordination policies for co-located groupware, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031648]
• 17: NIDCD. 2008. www.nidcd.nih.gov/health/hearing
• 18: Anne Marie Piper , Eileen O'Brien , Meredith Ringel Morris , Terry Winograd, SIDES: a cooperative tabletop computer game for social skills development, Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work, November 04-08, 2006, Banff, Alberta, Canada [doi>10.1145/1180875.1180877]
• 19: Meredith Ringel , Kathy Ryall , Chia Shen , Clifton Forlines , Frederic Vernier, Release, relocate, reorient, resize: fluid techniques for document sharing on multi-user interactive tables, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986085]
• 20: Douglas Schuler , Aki Namioka, Participatory Design: Principles and Practices, L. Erlbaum Associates Inc., Hillsdale, NJ, 1993
• 21: Jonathan Schull, An extensible, scalable browser-based architecture for synchronous and asynchronous communication and collaboration systems for deaf and hearing individuals, Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility, October 23-25, 2006, Portland, Oregon, USA [doi>10.1145/1168987.1169057]
• 22: Stacey D. Scott , M. Sheelagh , T. Carpendale , Kori M. Inkpen, Territoriality in collaborative tabletop workspaces, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031655]
• 23: Chia Shen , Frédéric D. Vernier , Clifton Forlines , Meredith Ringel, DiamondSpin: an extensible toolkit for around-the-table interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.167-174, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985714]
• 24: Starner, T., J. Weaver, and A. Pentland, A Wearable Computing Based American Sign Language Recognizer. In Personal and Ubiquitous Computing, Volume 1 (4), 1997.
• 25: Tse, E., Greenberg, S., Shen, C. and Forlines, C. Multimodal Multiplayer Tabletop Gaming. Proceedings of PerGames 2006, 139--148
• 26: Edward Tse , Chia Shen , Saul Greenberg , Clifton Forlines, Enabling interaction with single user applications through speech and gestures on a multi-user tabletop, Proceedings of the working conference on Advanced visual interfaces, May 23-26, 2006, Venezia, Italy [doi>10.1145/1133265.1133336]
• 27: Murray Turoff, Computerized conferencing for the deaf and handicapped, ACM SIGCAPH Computers and the Physically Handicapped [doi>10.1145/964121.964123]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Computer-supported cooperative work

General Terms:
Design, Experimentation, Human Factors

Keywords:
assistive technology, computer-supported cooperative health care, deafness, multimodal interfaces, multitouch, speech recognition, tabletop groupware