MobileASL:: intelligibility of sign language video as constrained by mobile phone technology

ABSTRACT

For Deaf people, access to the mobile telephone network in the United States is currently limited to text messaging, forcing communication in English as opposed to American Sign Language (ASL), the preferred language. Because ASL is a visual language, mobile video phones have the potential to give Deaf people access to real-time mobile communication in their preferred language. However, even today's best video compression techniques can not yield intelligible ASL at limited cell phone network bandwidths. Motivated by this constraint, we conducted one focus group and one user study with members of the Deaf Community to determine the intelligibility effects of video compression techniques that exploit the visual nature of sign language. Inspired by eyetracking results that show high resolution foveal vision is maintained around the face, we studied region-of-interest encodings (where the face is encoded at higher quality) as well as reduced frame rates (where fewer, better quality, frames are displayed every second). At all bit rates studied here, participants preferred moderate quality increases in the face region, sacrificing quality in other regions. They also preferred slightly lower frame rates because they yield better quality frames for a fixed bit rate. These results show promise for realtime access to the current cell phone network through signlanguage-specific encoding techniques.

REFERENCES

• 1: 3GNewsroom.com. 3UK disgraced by BBC watchdog programme. http://www.3gnewsroom.com/, October 22, 2003.
• 2: 3GToday. http://www.3gtoday.com/, 2006.
• 3: D. Agrafiotis, C. N. Canagarajah, D. R. Bull, M. Dye, H. Twyford, J. Kyle, and J. T. Chung-How. Optimized sign language video coding based on eye-tracking analysis. In VCIP, pages 1244--1252, 2003.
• 4: L. Aimar, L. Merritt, E. Petit, M. Chen, J. Clay, M. Rullgrd, C. Heine, and A. Izvorski. x264 - a free h264/AVC encoder. http://www.videolan.org/x264.html, 2005.
• 5: J. Bangham, S. J. Cox, M. Lincoln, I. Marshall, M. Tutt, and M. Wells. Signing for the Deaf Using Virtual Humans. In IEE Colloquium on Speech and Language Processing for Disabled and Elderly, 2000.
• 6: F. Ciaramello, A. Cavender, S. Hemami, E. Riskin, and R. Ladner. Predicting intelligibility of compressed american sign language video with objective quality metrics. In 2006 International Workshop on Video Processing and Quality Metrics for Consumer Electronics, 2006.
• 7: GSMA. General packet radio service. http://www.gsmworld.com/technology/gprs/class.shtml, 2006.
• 8: N. Habili, C. C. Lim, and A. Moini. Segmentation of the face and hands in sign language video sequences using color and motion cues. IEEE Trans. Circuits Syst. Video Techn., 14(8):1086--1097, 2004.
• 9: Christina L. James , Kelly M. Reischel, Text input for mobile devices: comparing model prediction to actual performance, Proceedings of the SIGCHI conference on Human factors in computing systems, p.365-371, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365300]
• 10: E. Keating and G. Mirus. American sign language in virtual space: Interactions between deaf users of computer-mediated video communication and the impact of technology on language practices. In Language in Society, volume 32, pages 693--714, 2003.
• 11: R. Mitchell. How many deaf people are there in the United States? http://gri.gallaudet.edu/Demographics/deaf-US.php, 2005.
• 12: L. Muir and I. Richardson. Perception of sign language and its application to visual communications for deaf people. In Journal of Deaf Studies and Deaf Education, volume 10, pages 390--401, 2005.
• 13: I. Richardson. vocdex : H.264 tutorial white papers. http://www.vcodex.com/h264.html, 2004.
• 14: R. Schumeyer, E. Heredia, and K. Barner. Region of Interest Priority Coding for Sign Language Videoconferencing. In IEEE First Workshop on Multimedia Signal Processing, pages 531--536, 1997.
• 15: I. T. S. Sector. Draft application profile: Sign language and lip reading real time conversation usage of low bit rate video communication. 1998.
• 16: Thad Starner , Alex Pentland , Joshua Weaver, Real-Time American Sign Language Recognition Using Desk and Wearable Computer Based Video, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.20 n.12, p.1371-1375, December 1998 [doi>10.1109/34.735811]
• 17: Christian Vogler , Dimitris Metaxas, A framework for recognizing the simultaneous aspects of American sign language, Computer Vision and Image Understanding, v.81 n.3, p.358-384, March 2001 [doi>10.1006/cviu.2000.0895]

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

General Terms:
Design, Experimentation, Human Factors

Keywords:
american sign language (ASL), deaf community, mobile telephone use, video compression