The MaggLite post-WIMP toolkit: draw it, connect it and run it

ABSTRACT

This article presents MaggLite, a toolkit and sketch-based interface builder allowing fast and interactive design of post-WIMP user interfaces. MaggLite improves design of advanced UIs thanks to its novel <i>mixed-graph</i> architecture that dynamically combines scene-graphs with interaction-graphs. <i>Scene-graphs</i> provide mechanisms to describe and produce rich graphical effects, whereas <i>interaction-graphs</i> allow expressive and fine-grained description of advanced interaction techniques and behaviors such as multiple pointers management, toolglasses, bimanual interaction, gesture, and speech recognition. Both graphs can be built interactively by sketching the UI and specifying the interaction using a dataflow visual language. Communication between the two graphs is managed at runtime by components we call <i>Interaction Access Points</i>. While developers can extend the toolkit by refining built-in generic mechanisms, UI designers can quickly and interactively design, prototype and test advanced user interfaces by applying the MaggLite principle: "draw it, connect it and run it".

REFERENCES

• 1: Rafael Ballagas , Meredith Ringel , Maureen Stone , Jan Borchers, iStuff: a physical user interface toolkit for ubiquitous computing environments, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642705]
• 2: Rémi Bastide , David Navarre , Philippe Palanque, A model-based tool for interactive prototyping of highly interactive applications, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506457]
• 3: Michel Beaudouin-Lafon , Wendy E. Mackay , Peter Andersen , Paul Janecek , Mads Jensen , Henry Michael Lassen , Kasper Lund , Kjeld Høyer Mortensen , Stephanie Munck , Anne V. Ratzer , Katrine Ravn , Søren Christensen , Kurt Jensen, CPN/Tools: A Post-WIMP Interface for Editing and Simulating Coloured Petri Nets, Proceedings of the 22nd International Conference on Application and Theory of Petri Nets, p.71-80, June 25-29, 2001
• 4: Michel Beaudouin-Lafon, Instrumental interaction: an interaction model for designing post-WIMP user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.446-453, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332473]
• 5: Michel Beaudouin-Lafon, Novel interaction techniques for overlapping windows, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502371]
• 6: Benjamin B. Bederson , James D. Hollan , Allison Druin , Jason Stewart , David Rogers , David Proft, Local tools: an alternative to tool palettes, Proceedings of the 9th annual ACM symposium on User interface software and technology, p.169-170, November 06-08, 1996, Seattle, Washington, United States [doi>10.1145/237091.237116]
• 7: Benjamin B. Bederson , Jon Meyer , Lance Good, Jazz: an extensible zoomable user interface graphics toolkit in Java, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.171-180, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354754]
• 8: Eric A. Bier , Steven Freeman, MMM: a user interface architecture for shared editors on a single screen, Proceedings of the 4th annual ACM symposium on User interface software and technology, p.79-86, November 11-13, 1991, Hilton Head, South Carolina, United States [doi>10.1145/120782.120791]
• 9: Alan Hamilton Borning, Thinglab--a constraint-oriented simulation laboratory., 1979
• 10: Dragicevic, P. and Fekete, J.D. Input Device Selection and Interaction Configuration with ICON. In Proc. of IHM-HCI 2001, People and Computers XV - Interaction without Frontiers (September 2001, Lille, France), Springer Verlag, 2001, pp. 543--448.
• 11: Allison Druin , Jason Stewart , David Proft , Ben Bederson , Jim Hollan, KidPad: a design collaboration between children, technologists, and educators, Proceedings of the SIGCHI conference on Human factors in computing systems, p.463-470, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258866]
• 12: Esteban, O., Chatty, S. and Palanque, P. Whizz'ed : a visual environment for building highly interactive software. In Proc. of INTERACT'95: 5 th IFIP International Conference on Human-Computer Interaction (June 1995, Lillehammer, Norway), IOS Press, 1995, pp. 121--126.
• 13: Jean-Daniel Fekete, The InfoVis Toolkit, Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'04), p.167-174, October 10-12, 2004 [doi>10.1109/INFOVIS.2004.64]
• 14: Saul Greenberg , Chester Fitchett, Phidgets: easy development of physical interfaces through physical widgets, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502388]
• 15: Tyson R. Henry , Scott E. Hudson , Gary L. Newell, Integrating gesture and snapping into a user interface toolkit, Proceedings of the 3rd annual ACM SIGGRAPH symposium on User interface software and technology, p.112-122, October 03-05, 1990, Snowbird, Utah, United States [doi>10.1145/97924.97938]
• 16: Jason I. Hong , James A. Landay, SATIN: a toolkit for informal ink-based applications, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.63-72, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354412]
• 17: Hourcade, J.P. and Bederson, B.B. Architecture and Implementation of a Java Package for Multiple Input Devices (MID), Human-Computer Interaction Laboratory, University of Maryland, College Park, MD 20742, USA, 1999.
• 18: Scott E. Hudson , Ian Smith, Ultra-lightweight constraints, Proceedings of the 9th annual ACM symposium on User interface software and technology, p.147-155, November 06-08, 1996, Seattle, Washington, United States [doi>10.1145/237091.237112]
• 19: Huot, S., Dumas, C. and Hegron, G. Svalabard: A Virtual Drawing Table for 3D Modeling. In Proc. of IHM'04: 16th French-Speaking Conference on Human-Computer Interaction (August 2004, Namur, Belgium), ACM press, 2004.
• 20: Dan Ingalls , Scott Wallace , Yu-Ying Chow , Frank Ludolph , Ken Doyle, Fabrik: a visual programming environment, ACM SIGPLAN Notices, v.23 n.11, p.176-190, Nov. 1988
• 21: Robert J. K. Jacob , Leonidas Deligiannidis , Stephen Morrison, A software model and specification language for non-WIMP user interfaces, ACM Transactions on Computer-Human Interaction (TOCHI), v.6 n.1, p.1-46, March 1999 [doi>10.1145/310641.310642]
• 22: Glenn E. Krasner , Stephen T. Pope, A cookbook for using the model-view controller user interface paradigm in Smalltalk-80, Journal of Object-Oriented Programming, v.1 n.3, p.26-49, Aug./Sept. 1988
• 23: James A. Landay , Brad A. Myers, Interactive sketching for the early stages of user interface design, Proceedings of the SIGCHI conference on Human factors in computing systems, p.43-50, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223910]
• 24: James A. Landay, SILK: sketching interfaces like krazy, Conference companion on Human factors in computing systems: common ground, p.398-399, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/257089.257396]
• 25: Eric Lecolinet, A molecular architecture for creating advanced GUIs, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.135-144, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964711]
• 26: Brad A. Myers, A new model for handling input, ACM Transactions on Information Systems (TOIS), v.8 n.3, p.289-320, July 1990 [doi>10.1145/98188.98204]
• 27: Puckette, M. Combining Event and Signal Processing in the MAX Graphical Programming Environment, Computer Music Journal, 15(3):50--57, 1991.
• 28: Daniel Salber , Anind K. Dey , Gregory D. Abowd, The context toolkit: aiding the development of context-enabled applications, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.434-441, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303126]
• 29: Brad Vander Zanden , Brad A. Myers, Demonstrational and constraint-based techniques for pictorially specifying application objects and behaviors, ACM Transactions on Computer-Human Interaction (TOCHI), v.2 n.4, p.308-356, Dec. 1995 [doi>10.1145/212430.212443]
• 30: Virtools dev. Virtools SA, 2001. http://www.virtools.com/.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
        D.2.11 Software Architectures

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen); Prototyping; User interface management systems (UIMS); Interaction styles (e.g., commands, menus, forms, direct manipulation)

Keywords:
GUI architectures, GUI toolkits, ICON, MaggLite, interaction design, interaction techniques