Visually prototyping perceptual user interfaces through multimodal storyboarding

ABSTRACT

We are applying our knowledge in designing informal prototyping tools for user interface design to create an interactive visual prototyping tool for perceptual user interfaces. Our tool allows a designer to quickly map out certain types of multimodal, cross-device user interface scenarios. These sketched designs form a multimodal storyboard that can then be executed, quickly testing the interaction and collecting feedback about refinements necessary for the design. By relying on visual prototyping, our multimodal storyboarding tool simplifies and speeds perceptual user interface prototyping and opens up the challenging space of perceptual user interface design to non-programmers.

REFERENCES

• 1: Moran, L. B., et al., Multimodal user interfaces in the open agent architecture. Knowledge-Based Systems, 1998. 10(5): p. 295--304.
• 2: Philip R. Cohen , Michael Johnston , David McGee , Sharon Oviatt , Jay Pittman , Ira Smith , Liang Chen , Josh Clow, QuickSet: multimodal interaction for distributed applications, Proceedings of the fifth ACM international conference on Multimedia, p.31-40, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266328]
• 3: Myers, B. A., et al., CMU Command Post of The Future: MultiModal Command and Control, Carnegie Mellon University. http://www.cs.cmu.edu/~cpof/index.html.
• 4: Laurence Nigay , Joëlle Coutaz, A design space for multimodal systems: concurrent processing and data fusion, Proceedings of the INTERCHI '93 conference on Human factors in computing systems, p.172-178, May 1993, Amsterdam, The Netherlands
• 5: James A. Landay , Brad A. Myers, Sketching Interfaces: Toward More Human Interface Design, Computer, v.34 n.3, p.56-64, March 2001
• 6: James Lin , Mark W. Newman , Jason I. Hong , James A. Landay, DENIM: finding a tighter fit between tools and practice for Web site design, Proceedings of the SIGCHI conference on Human factors in computing systems, p.510-517, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332486]
• 7: Sharon Oviatt, Mutual disambiguation of recognition errors in a multimodel architecture, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.576-583, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303163]
• 8: Allen Cypher , Daniel C. Halbert , David Kurlander , Henry Lieberman , David Maulsby , Brad A. Myers , Alan Turransky, Watch what I do: programming by demonstration, MIT Press, Cambridge, MA, 1993
• 9: Jason I. Hong , James A. Landay, SATIN: a toolkit for informal ink-based applications, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.63-72, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354412]

INDEX TERMS

Keywords:
design, informal prototyping, multimodal