Design of communication in multimodal web interfaces

ABSTRACT

In this paper we present our considerations for the design, development and evaluation of Web multimodal interfaces using as example the analysis and the results of three Case Studies. Two usability methods were applied in these Case Studies as part of our approach for Web multimodal interfaces design. The objective is to smooth the progress of the design activities provided by the Multimodal Web Approach (MMWA) recovering Design Rationale from previous projects and improving the usability and user experience in the multimodal interaction. From the experience got performing three case studies with the MMWA, we derive a general framework for design of communication in Web Multimodal Interfaces.

REFERENCES

• 1: Abrams, M.; Phanouriou, C.; Batongbacal, A.; Williams, S.; Shuster, J.; 1994. UIML: An Appliance-Independent XML User Interface Language, Proceedings of the 8th WWW.
• 2: Almor, A., Noun-phrase anaphora and focus: The informational load hypothesis. Psychological Review, 1999.
• 3: Chang, J. and Bourguet, M. 2008. Usability framework for the design and evaluation of multimodal interaction. In Proceedings of the 22nd British CHI Group Annual Conference on HCI 2008: People and Computers Xxii: Culture, Creativity, interaction -- Volume 2 (Liverpool, United Kingdom, September 01 -- 05, 2008). British Computer Society Conference on Human--Computer Interaction. British Computer Society, Swinton, UK, 123--126.
• 4: Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman,J., and Smith, I. et al. QuickSet: multimodal interaction for simulation set-up and control. Proceedings of the Fifth Conference on Applied Natural Language Processing (1997) 20--24.
• 5: Coutaz, J.; Nigay, L.; Salber, D.; Blandford, A.; May, J. and Young, R. M. "Four easy pieces for assessing the usability of multimodal interaction: the CARE properties". Proceedings of INTERACT (1995) 115--120.
• 6: Desurvire, H. (1994). Faster, cheaper!! Are usability inspection methods as effective as empirical testing? In Nielsen, J. and Mack, R. (eds.), Usability inspection methods, John Wiley & Sons, Inc., New York, 173--202.
• 7: Limbourg, Q.; Vanderdonckt, J.; Michotte, B.; Bouillon, L.; Lopez, V. USIXML: a Language Supporting Multi-Path Development of User Interfaces. In Proc. of 9th IFIP Working Conference on Engineering for Human-Computer Interaction (Hamburg, July 11--13, 2004). Kluwer Academic Press, Dordrecht, 2004.
• 8: Nielsen, J. Usability Engineering. Academic Press, Cambridge, 1993.
• 9: Oviatt, S., Cohen, P. R., Wu, L., Vergo, J., Duncan, L., and Suhm, B. et al. Designing the user interface for multimodal speech and pen-based gesture applications: state-of-the-art systems and future research directions. Human-Computer Interaction, 15 (2000) 263--322.
• 10: Oviatt, S. Multimodal Interfaces. In: J. Jacko and A. Sears (eds) The Human-Computer Interaction Handbook. Mahwah, NJ: Lawrence Erlbaum and Associates, 2002.
• 11: Oviatt, S., Coulston, R., and Lunsford, R. 2004. When do we interact multimodally? Cognitive load and multimodal communication patterns. In Proceedings of the 6th international Conference on Multimodal interfaces (State College, PA, USA, October 13 -- 15, 2004). ICMI '04. ACM, New York, NY, 129--136.
• 12: Paternò. F.; Giammarino, F. Multimodal interaction: research papers: Authoring interfaces with combined use of graphics and voice for both stationary and mobile devices. Proceedings of the working conference on advanced visual interfaces AVI '06.
• 13: Puerta, A.; Eisenstein, J. XIML: A Common Representation for Interaction Data, Proceedings ACM IUI'01, pp.214--215.
• 14: Stanciulescu A.; Limbourg Q.; Vanderdonckt J.; Michotte B.; Montero F.; A Transformational Approach for Multimodal Web User Interfaces based on USIXML. Proceedings of the 7th international conference on Multimodal interfaces ICMI 2005, pages 259--266, ACM Press.
• 15: Talarico Neto, A.; Fortes, R.P.M ; Silva Filho, A.G . Multimodal interfaces design issues: the fusion of well-designed voice and graphical user interfaces. In: 26th ACM International Conference on Design of Communication, 2008, Lisbon. 26th ACM International Conference on Design of Communication, 2008. v. 1. p. 277--278.
• 16: Talarico Neto, A.; Bittar, T.J. ; Fortes, R.P.M ; Felizardo K.R . Abordagem para o desenvolvimento e avaliação de interfaces multimodais Web pautada em princípios de usabilidade. In: VIII Simpósio sobre Fatores Humanos em Sistemas Computacionais, Porto Alegre. v. 1. p. 21--30.
• 17: Talarico Neto, A., Bittar, T.J., Fortes, R.P.M. and Felizardo, K. 2009. Developing and evaluating web multimodal interfaces -- a case study with usability principles. In Proceedings of the 2009 ACM Symposium on Applied Computing (Honolulu, Hawaii).ACM, NY, 116--12.
• 18: Zdrahal, Z., Mulholland, P., Valasek, M., and Bernardi, A. 2007. Worlds and transformations: Supporting the sharing and reuse of engineering design knowledge. Int. J. Hum.-Comput. Stud. 65, 12 (Dec. 2007), 959--982. DOI= http://dx.doi.org/10.1016/j.ijhcs.2007.07.004

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Design, Documentation, Human Factors, Theory

Keywords:
multimodal interfaces design, multimodal web approach