User expectations and user experience with different modalities in a mobile phone controlled home entertainment system

ABSTRACT

Home environment is an exciting application domain for multimodal mobile interfaces. Instead of multiple remote controls, personal mobile devices could be used to operate home entertainment systems. This paper reports a subjective evaluation of multimodal inputs and outputs for controlling a home media center using a mobile phone. A within-subject evaluation with 26 participants revealed significant differences on user expectations on and experiences with different modalities. Speech input was received extremely well, even surpassing expectations in some cases, while gestures and haptic feedback were almost failing to meet the lowest expectations. The results can be applied for designing similar multimodal applications in home environments.

REFERENCES

• 1: Ferscha, A., Vogl, S., Emsenhuber, B., and Wally, B. 2007. Physical shortcuts for media remote controls. In Proceedings of the 2nd international Conference on intelligent Technologies For interactive Entertainment. ICST, Brussels, Belgium, 1--8.
• 2: Hartikainen, M., Salonen, E.-P., Turunen, M., 2004. Subjective evaluation of spoken dialogue systems using SERVQUAL method. In Proceedings of the 7th International Conference on Spoken Language Processing (ICSLP'04), 2273--2276.
• 3: Hoggan, E. and Brewster, S. A. 2007. Designing Audio and Tactile Crossmodal Icons for Mobile Devices. In Proceedings of ACM International Conference on Multimodal Interfaces (ICMI'07). ACM Press, 162--169.
• 4: Ibrahim, A., and Johansson, P. 2003. Multimodal Dialogue Systems: A Case Study for Interactive TV. Carbonell, Noelle; Stephanidis, Constantine (Eds.) Universal Access. Theoretical Perspectives, Practice, and Experience, 7th ERCIM International Workshop on User Interfaces for All, Revised Papers. Springer, LNCS, Vol. 2615. 209--218.
• 5: Melto, A., Turunen, M., Kainulainen, A., Hakulinen, J., Heimonen, T., and Antila, V. 2008. Evaluation of predictive text and speech inputs in a multimodal mobile route guidance application. In Proceedings of the 10th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '08). ACM, New York, NY, 355--358.
• 6: Schlömer, T., Poppinga, B., Henze, N., and Boll, S. 2008. Gesture recognition with a Wii controller. In Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (TEI '08). ACM, New York, NY, 11--14.
• 7: Soronen, H., Turunen, M., and Hakulinen, J. 2008. Voice Commands in Home Environment - a Consumer Survey. In Proceedings of Interspeech 2008, 2078--2081.
• 8: Turunen, M., Hakulinen, J., Melto, A., Heimonen, T., Laivo, T., and Hella, J. 2009. SUXES -- User Experience Evaluation Method for Spoken and Multimodal Interaction. In Proceedings of Interspeech 2009.
• 9: Turunen, M., Hakulinen, J., Melto, A., Hella, J., Rajaniemi, J.-P., Mäkinen, E., Rantala, J., Heimonen, T., Laivo, T., Soronen, H., Hansen, M., Valkama, P., Miettinen, T., Raisamo, R. 2009. Speech-based and Multimodal Media Center for Different User Groups. In Proceedings of Interspeech 2009.
• 10: Wittenburg, K., Lanning, T., Schwenke, D., Shubin, H., and Vetro, A. 2006. The prospects for unrestricted speech input for TV content search. In Proceedings of the Working Conference on Advanced Visual interfaces (AVI '06). ACM, New York, NY, 352--359.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Voice I/O; Interaction styles (e.g., commands, menus, forms, direct manipulation); Haptic I/O

General Terms:
Experimentation, Human Factors, Measurement, Performance

Keywords:
SUXES, gestures, haptic feedback, speech recognition