Migratory MultiModal interfaces in MultiDevice environments

ABSTRACT

This paper describes an environment able to support migratory multimodal interfaces in multidevice environments. We introduce the software architecture and the device-independent languages used by our tool, which provides services enabling users to freely move about, change device and continue the current task from the point where they left off in the previous device. Our environment currently supports interaction with applications through graphical and vocal modalities, either separately or together. Such applications are implemented in Web-based languages. We discuss how the features of the device at hand, desktop or mobile, are considered when generating the multimodal user interface.

REFERENCES

• 1: Marc Abrams , Constantinos Phanouriou , Alan L. Batongbacal , Stephen M. Williams , Jonathan E. Shuster, UTML: an appliance-independent XML user interface language, Proceeding of the eighth international conference on World Wide Web, p.1695-1708, May 1999, Toronto, Canada
• 2: Balme, L. Demeure, A., Barralon, N., Coutaz, J., Calvary, G. CAMELEON-RT: a Software Architecture Reference Model for Distributed, Migratable, and Plastic User Interfaces. In Proceedings the Second European Symposium on Ambient Intelligence (EUSAI '04), LNCS 3295, Markopoulos et al. Springer-Verlag, Berlin Heidelberg, 2004, 291--302.
• 3: Bandelloni, R., Berti, S., Paternò F., "Mixed-Initiative, Trans-Modal Interface Migration", Proceedings Mobile HCI 2004, Glasgow, September 2004, Lecture Notes Computer Science 3160, pp.216--227, Sprinter Verlag..
• 4: Jullien Bouchet , Laurence Nigay, ICARE: a component-based approach for the design and development of multimodal interfaces, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986055]
• 5: Calvary, G., Coutaz, J., Thevenin, D., Limbourg, Q., Bouillon, L., Vanderdonckt, J. A Unifying Reference Framework for Multi-Target User Interfaces. Interacting with Computers. Vol. 15, No. 3, June 2003, pp. 289--308.
• 6: Coutaz J., Nigay L., Salber D.,.Blandford A, May J., Young R., 1995. Four Easy Pieces for Assessing the Usability of Multmodal Interaction: the CARE properties. Proceedings INTERACT 1995, pp.115--120
• 7: Channarukul S., McRoy S. W.,. Ali S. S. MULTIFACE: Multimodal Content Adaptations for Heterogeneous Devices.
• 8: David Garlan , Dan Siewiorek , Asim Smailagic , Peter Steenkiste, Project Aura: Toward Distraction-Free Pervasive Computing, IEEE Pervasive Computing, v.1 n.2, p.22-31, April 2002 [doi>10.1109/MPRV.2002.1012334]
• 9: Limbourg, Q., Vanderdonckt, J., UsiXML: A User Interface Description Language Supporting Multiple Levels of Independence, in Matera, M., Comai, S. (Eds.), 'Engineering Advanced Web Applications', Rinton Press, Paramus, 2004.
• 10: Bandelloni, R., Paternò, F. Migratory User Interfaces Able to Adapt to Various Interaction Platforms. In International Journal of Human Computer Studies 60, pp. 621--639, Elsevier, 2004.
• 11: Jeffrey Nichols , Brad A. Myers , Michael Higgins , Joseph Hughes , Thomas K. Harris , Roni Rosenfeld , Mathilde Pignol, Generating remote control interfaces for complex appliances, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.572008]
• 12: Obrenovic, Z., Starcevic D., Selic B., A Model-Driven Approach to Content Repurposing, IEEE Mutimedia, Januray March 2004, pp.62--71.
• 13: Fabio Paterno, Model-Based Design and Evaluation of Interactive Applications, Springer-Verlag, London, 1999
• 14: Pérez-Quiñoes, M, A., Capra, R, G., Shao, Z.. The Ears Have It: A Task by Information Structure Taxonomy for Voice Access to Web Pages. In Proceedings of INTERACT 2003, IOS Press.
• 15: Angel Puerta , Jacob Eisenstein, XIML: a common representation for interaction data, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502763]
• 16: Szekely, P., 1996. Retrospective and Challenges for Model-Based Interface Development. 2nd International Workshop on Computer-Aided Design of User Interfaces, Namur, Namur University Press.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Prototyping

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

General Terms:
Design, Human Factors, Languages

Keywords:
MultiModal user interfaces, migratory interfaces, model-based design, multi-device environments, ubiquitous systems