Dialogue management for gestural interfaces

ABSTRACT

Gestural interfaces are electronic analogues to pencil and paper. Since the effectiveness of such interfaces depends heavily on enduser familiarity with pencil markup of printed documents, the interface must conform to the user's behaviour and not rely on educating the enduser. Spatial relationships among the gestural forms partially determine the syntactic interpretation of gestures, along with information about the context in the neighborhood of the gesture. Temporal grouping of gestural forms is more important than their temporal sequence. Such characteristics suggest a form of dialogue recognition in which rules do not specify temporal ordering of forms, and in which multiple parses are carried out in parallel.

REFERENCES

• 1: Alan Borning, The Programming Language Aspects of ThingLab, a Constraint-Oriented Simulation Laboratory, ACM Transactions on Programming Languages and Systems (TOPLAS), v.3 n.4, p.353-387, Oct. 1981 [doi>10.1145/357146.357147]
• 2: [2] Buxton, W., Sniderman, R., Reeves, W., Patel, S. and Baecker, R. The evolution of the SSSP score editing tools, Computer Music Journal 3, 4 (1979), 14-25.
• 3: [3] Buxton, W., Fiume, E., Hill, R., Lee, A. and Woo, C. Continuous hand-gesture driven input. Proceedings of Graphics Interface'83 (1983), 191-195.
• 4: Luca Cardelli , Rob Pike, Squeak: a language for communicating with mice, Proceedings of the 12th annual conference on Computer graphics and interactive techniques, p.199-204, July 1985
• 5: R. A. Duisberg, Animated graphical interfaces using temporal constraints, Proceedings of the SIGCHI conference on Human factors in computing systems, p.131-136, April 13-17, 1986, Boston, Massachusetts, United States
• 6: Adele Goldberg , David Robson, Smalltalk-80: the language and its implementation, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1983
• 7: Mark Green, The University of Alberta user interface management system, Proceedings of the 12th annual conference on Computer graphics and interactive techniques, p.205-213, July 1985
• 8: Paul R. Hanau , David R. Lenorovitz, Prototyping and simulation tools for user/computer dialogue design, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.271-278, July 14-18, 1980, Seattle, Washington, United States
• 9: D. A. Henderson, Jr., The Trillium user interface design environment, Proceedings of the SIGCHI conference on Human factors in computing systems, p.221-227, April 13-17, 1986, Boston, Massachusetts, United States
• 10: [10] Hutchins, E. L., Hollan, J. D. and Norman, D. A. Direct manipulation interfaces. In User centered system design, Norman, D. A. and Draper, S. W. (eds.), Lawrence Erlbaum Associates, Hillsdale, NJ, 1986, 87-124.
• 11: David J. Kasik, A user interface management system, Proceedings of the 9th annual conference on Computer graphics and interactive techniques, p.99-106, July 26-30, 1982, Boston, Massachusetts, United States
• 12: [12] Kay, A. and Goldberg A. Personal Dynamic Media. IEEE Computer 10, 3 (1977), 31-41.
• 13: [13] Kay, A. Microelectronics and the Personal Computer. Scientific American 237, 3, (Sept. 1977), 231.
• 14: [14] Konneker, L. K. A Graphical Interaction Technique Which Uses Gestures. Proceedings of the First International Conference on Office Automation, (1984), IEEE, 51-55.
• 15: Daniel E. Lipkie , Steven R. Evans , John K. Newlin , Robert L. Weissman, Star graphics: An object-oriented implementation, Proceedings of the 9th annual conference on Computer graphics and interactive techniques, p.115-124, July 26-30, 1982, Boston, Massachusetts, United States
• 16: Greg Nelson, Juno, a constraint-based graphics system, Proceedings of the 12th annual conference on Computer graphics and interactive techniques, p.235-243, July 1985
• 17: Dan R Olsen, Jr. , Elizabeth P. Dempsey, SYNGRAPH: A graphical user interface generator, Proceedings of the 10th annual conference on Computer graphics and interactive techniques, p.43-50, July 25-29, 1983, Detroit, Michigan, United States
• 18: David L. Parnas, On the use of transition diagrams in the design of a user interface for an interactive computer system, Proceedings of the 1969 24th national conference, p.379-385, August 26-28, 1969 [doi>10.1145/800195.805945]
• 19: Gregory F. Pfister, A high level language extension for creating and controlling dynamic pictures, ACM SIGGRAPH Computer Graphics, v.10 n.1, p.1-9, Spring 1976
• 20: [20] J. R. Rhyne and C. G. Wolf. Gestural Interfaces for Information Processing Applications. T. J. Watson Research Center, IBM Corporation, 1986, IBM Research Report RC-12179.
• 21: [21] Rhyne, J. R. Dialogue Management for Gestural Interfaces. IBM Research Report RC-12244 (An extended version of this article), 1986.
• 22: Andrew J. Schulert , George T. Rogers , James A. Hamilton, ADM — a dialog manager, Proceedings of the SIGCHI conference on Human factors in computing systems, p.177-183, April 1985, San Francisco, California, United States
• 23: [23] Shneiderman, B. Direct manipulation: a step beyond programming languages, IEEE Computer 16, 8 (1983), 57-69.
• 24: [24] Tappert, C. C., Fox, A. S., Kim, J., Levy, S. E., and Zimmerman, L. L. Handwriting recognition of transparent tablet over flat display. SID Digest of Technical Papers XVII, (May 1986), 308-312.
• 25: [25] Ward, J. R. and Blesser, B. Interactive Recognition of Handprinted Characters for Computer Input, IEEE Computer Graphics and Applications 5, 9, (Sept. 1985), 24-37.
• 26: Catherine G. Wolf, CAN PEOPLE USE GESTURE COMMANDS?, ACM SIGCHI Bulletin, v.18 n.2, p.73-74, Oct. 1986 [doi>10.1145/15683.1044102]

INDEX TERMS

Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

General Terms:
Design, Human Factors, Management