Direct manipulation of user interfaces for migration

ABSTRACT

From a topological model of a working environment, MigriXML automatically generates a virtual reality environment for controlling the run-time migration of a graphical user interface from one computing platform to another one (e.g., from a desktop to a pocket computer), from one interaction surface to another (e.g., from a laptop to a wall screen) at run-time. For this purpose, any user interface subject to migration is described in USer Interface eXtensible Markup Language regarding its look & feel as well as the platforms and the surfaces involved in the migration. Each interface, in part or in whole, can be attached to a platform or a surface, detached from it, and migrated across platforms or interaction surfaces. Instead of communicating data and code during the migration, the description of the user interface of concern is wirelessly passed from one platform to another one to be regenerated on the target platform. To ensure a continuous control of the run-time migration, MigriXML automatically generates a world model representing the context of use where the source/target platforms/interaction surfaces are represented. Finally, migrating a user interface becomes as natural as its direct manipulation from one platform to another exactly in the same way as it is done on a single platform.

REFERENCES

• 1: Balme, L., Demeure, A., Barralon, N., Coutaz, J., and Calvary, G. Ethylene: a Software Architecture Reference Model for Distributed, Migrable, Plastic User Interfaces. In Proc. of Conf. on Ambient Intelligence EUSAI'04. Springer-Verlag, Berlin, 2004, 291--302.
• 2: Bandelloni, R., Berti, S., and Paternò, F. Mixed-Initiative, Trans-modal Interface Migration. In Proc. of MobileHCI'2004. Springer-Verlag, Berlin, 2004, 216--227.
• 3: Bandelloni, R. and Paternò, F. Migratory User Interfaces Able to Adapt to Various Interaction Platforms. International Journal of Human Computer Studies 60 (2004), 621--639.
• 4: Baudish, P., Cutrell, E., Czerwinski, M., Tandler, P., Bederson, B., and Zierlinger, A. Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-Operated Systems. In Proc. of Interact'2003. IOS Press, Amsterdam, 2003, 57--64.
• 5: Krishna A. Bharat , Luca Cardelli, Migratory applications, Proceedings of the 8th annual ACM symposium on User interface and software technology, p.132-142, November 15-17, 1995, Pittsburgh, Pennsylvania, United States [doi>10.1145/215585.215711]
• 6: Jacob T. Biehl , Brian P. Bailey, ARIS: an interface for application relocation in an interactive space, Proceedings of the 2004 conference on Graphics interface, p.107-116, May 17-19, 2004, London, Ontario, Canada
• 7: Coutaz, J., Lachenal, C., Calvary, G., and Thevenin, D. Software Architecture Adaptivity for Multi-surface Interaction and Plasticity. In Proc. of IFIP WG2.7 Workshop on Software Architecture Requirements for CSCW-CSCW'2000, ACM Press, 2000.
• 8: Coutaz, J., Lachenal, C., and Dupuy-Chessa, S. Ontology for Multisurface Interaction. In Proc. of Interact'2003, IOS Press, Amsterdam, 2003, 447--453.
• 9: João Pedro Sousa , David Garlan, Aura: an Architectural Framework for User Mobility in Ubiquitous Computing Environments, Proceedings of the IFIP 17th World Computer Congress - TC2 Stream / 3rd IEEE/IFIP Conference on Software Architecture: System Design, Development and Maintenance, p.29-43, August 25-30, 2002
• 10: Grolaux, D., Van Roy, P., Vanderdonckt, J. Migratable User Interfaces: Beyond Migratory User Interfaces. In Proc. of MOBIQUITOUS'04. IEEE Computer Society Press, Los Alamitos, 2004, 422--430.
• 11: Jonathan Grudin, Partitioning digital worlds: focal and peripheral awareness in multiple monitor use, Proceedings of the SIGCHI conference on Human factors in computing systems, p.458-465, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365312]
• 12: Grundy, J.C. and Hosking, J.G. Developing Adaptable User Interfaces for Component-based Systems. Interacting with Computers 14 (2002), 175--194.
• 13: François Guimbretière , Maureen Stone , Terry Winograd, Fluid interaction with high-resolution wall-size displays, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502353]
• 14: Limbourg, Q., Vanderdonckt, J., Michotte, B., Bouillon, L., Lopez, V., UsiXML: a Language Supporting Multi-Path Development of User Interfaces, Proc. of 9th IFIP Working Conference on Engineering for Human-Computer Interaction jointly with 11th Int. Workshop on Design, Specification, and Verification of Interactive Systems EHCI-DSVIS'2004 (Hamburg, July 11-13, 2004). LNCS, Vol. 3425, Springer-Verlag, Berlin, 2005, pp. 200--220.
• 15: Dejan S. Milojičić , Fred Douglis , Yves Paindaveine , Richard Wheeler , Songnian Zhou, Process migration, ACM Computing Surveys (CSUR), v.32 n.3, p.241-299, Sept. 2000 [doi>10.1145/367701.367728]
• 16: José Pascual Molina Massó , Jean Vanderdonckt , Francisco Montero Simarro , Pascual González López, Towards virtualization of user interfaces based on UsiXML, Proceedings of the tenth international conference on 3D Web technology, March 29-April 01, 2005, Bangor, United Kingdom [doi>10.1145/1050491.1050516]
• 17: Montero, F., López-Jaquero, V., Vanderdonckt, J., Gonzalez, P., Lozano, M.D., and Limbourg, Q. Solving the Mapping Problem in User Interface Design by Seamless Integration in IdealXML. In Proc. of 12th Int. Work-shop on Design, Specification, and Verification of Interactive Systems DSV-IS'2005 (Newcastle upon Tyne, July 13-15, 2005), Lecture Notes in Computer Science, Springer-Verlag, Berlin, 2005.
• 18: Angel R. Puerta, A Model-Based Interface Development Environment, IEEE Software, v.14 n.4, p.40-47, July 1997 [doi>10.1109/52.595902]
• 19: Jun Rekimoto , Masanori Saitoh, Augmented surfaces: a spatially continuous work space for hybrid computing environments, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.378-385, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303113]
• 20: Jun Rekimoto, Pick-and-drop: a direct manipulation technique for multiple computer environments, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.31-39, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263505]
• 21: Rekimoto, J., Ayatsuka, Y., Kohno, M., and Oba, H. Proximal Interactions: A Direct Manipulation Technique for Wireless Networking. In Proc. of Interact'2003. IOS Press, Amsterdam, 2003, 511--518.
• 22: Kai Schäfer , Volker Brauer , Willi Bruns, A new approach to human-computer interaction—synchronous modelling in real and virtual spaces, Proceedings of the conference on Designing interactive systems: processes, practices, methods, and techniques, p.335-344, August 18-20, 1997, Amsterdam, The Netherlands [doi>10.1145/263552.263638]
• 23: Schmalstieg, D. and Hesina, G. Application Migration for Virtual Work Environments. Vienna Univ. for Virtual Work Environments, Vienna, 2001.
• 24: Song, H., Chu, H., and Kurakake, S. Browser Session Preservation and Migration. In Poster Session of WWW'2002.
• 25: Tan, D.S. and Czerwinski, M. Effects of Visual Separation and Physical Discontinuities when Distributing Information across Multiple Displays. In Proc. of Interact'2003. IOS Press, Amsterdam, 2003, 252--255.
• 26: A metamodel for the runtime architecture of an interactive system: the UIMS tool developers workshop, ACM SIGCHI Bulletin, v.24 n.1, p.32-37, Jan. 1992 [doi>10.1145/142394.142401]
• 27: Vanderdonckt, J. A MDA-Compliant Environment for Developing User Interfaces of Information Systems. In Proc. of 17th Conf. on Advanced Information Systems Engineering CAiSE'05 (Porto, 13-17 June 2005). LNCS, Vol. 3520, Springer-Verlag, Berlin, 2005, 16--31.
• 28: VRML Widgets Working Group Website. URL: http://zing. ncsl.nist.gov/~gseidman/vrml/wwg/
• 29: Web3D Consortium Website. URL: http://www.web 3D.org

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Design, Languages

Keywords:
migration, virtual environment