Facilitating mobile communication with multimodal access to email messages on a cell phone

ABSTRACT

This paper reports on a user trial (N=17) that compares the use of two systems for accessing email messages on a telephone handset. The first system uses graphic output and telephone keypad input, while the second system has both graphic and speech output, with keypad and speech as input. To our knowledge, this trial represents the first evaluation of a fully functioning multimodal system that uses natural language understanding on a phone, and was dependent on the 3G network currently available in Australia. Participants saw significantly greater value in the multimodal interaction, and rated their experience with the multimodal system significantly more positively than the unimodal system. They were also significantly more inclined to use and recommend the multimodal system over the current unimodal product offering. While we expected to see some mixed usage of modalities in the multimodal system, participants used speech predominantly, falling back to GUI selection only after encountering multiple speech recognition failures in a row.

REFERENCES

• 1: Lai, J., Mitchell, S., Viveros, M., Wood, D., Lee, K.M., Ubiquitous Access to Unified Messaging: A study of Usability, and the Limits of Pervasive Computing. International Journal of Human Computer Interaction, Volume 14 (3-4) 2002
• 2: Sharon Oviatt, Ten myths of multimodal interaction, Communications of the ACM, v.42 n.11, p.74-81, Nov. 1999 [doi>10.1145/319382.319398]
• 3: Sharon Oviatt, Multimodal interfaces, The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2002

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.3 Communications Applications

General Terms:
Design

Keywords:
mobile communication, mobile interface design, multimodal interaction, pervasive computing, spoken interfaces