Creating highly-interactive and graphical user interfaces by demonstration

ABSTRACT

It is very time-consuming and expensive to create the graphical, highly-interactive styles of user interfaces that are increasingly common. User Interface Management Systems (UIMSs) attempt to make the creation of user interfaces easier, but most existing UIMSs cannot create the low-level interaction techniques (pop-up pull-down and fixed menus, on-screen "light buttons", scroll-bars, elaborate feedback mechanisms and animations, etc.) that are frequently used. This paper describes Peridot, a system that automatically creates the code for these user interfaces while the designer demonstrates to the system how the interface should look and work. Peridot uses rule-based inferencing so no programming by the designer is required, and Direct Manipulation techniques are used to create Direct Manipulation interfaces, which can make full use of a mouse and other input devices. This allows extremely rapid prototyping of user interfaces.

REFERENCES

• Apple 85: Apple Computer Inc., Inside MacIntosh, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1994
• Bauer 78: Michael Anthony Bauer. A Basis for the Acquisition of Procedures. PhD Thesis, Department of Computer Science, University of'Toronto. 1978. 310 pages.
• Biermann 76a: Alan W. Biermann. '~Approaches to Automatic Programming," Advanc~ in Computers, Morris Rubinoff and Marshall C. Yovitz, eds. Vol. 15. New York: Academic Press, 1976. pp. 1-63.
• Biermann 76b: Alan W. Biermann and Ramachandran Krishnaswamy. "Constructing Programs from Example Computations," IEEE Transactions on Software Engineering. Vol. SE-2, no. 3. Sept. 1976. pp. i41-153.
• Borning 79: Alan Borning. ThingIab--A Constraint-Oriented Simulation Laboratory. Xerox Pale Alto Research Center Technical Report SSL-79-3. July, 1979. 100 pages.
• Buxton 83: W. Buxton , M. R. Lamb , D. Sherman , K. C. Smith, Towards a comprehensive user interface management system, Proceedings of the 10th annual conference on Computer graphics and interactive techniques, p.35-42, July 25-29, 1983, Detroit, Michigan, United States
• Buxton 86: W. Buxton , B. Myers, A study in two-handed input, Proceedings of the SIGCHI conference on Human factors in computing systems, p.321-326, April 13-17, 1986, Boston, Massachusetts, United States
• Cardelli 85: Luca Cardelli , Rob Pike, Squeak: a language for communicating with mice, ACM SIGGRAPH Computer Graphics, v.19 n.3, p.199-204, Jul. 1985
• Foley 84: James D. Foley. "Managing the Design of User- Computer Interfaces," Proceedings of the Fifth Annual NCGA Conference and Exposition. Anaheim, CA. Vol. II. May 13-17, 1984. pp. 436-451.
• Gould 84: Laura Gould and William Finzer. Programming by Rehearsal. Xerox Pale Alto Research Center Technical Report SCL-84-1. May, 1984. 133 pages. A short version appears in Byte. Vol. 9, no. 6. June, 1984.
• Halbert 81: Daniel Conrad Halbert, Programming by example, 1984
• Halbert 84: Daniel C. Halhert. Programming by Example. PhD Thesis. Computer Science Division, Dept. of EE&CS, University of California, Berkeley. 1984. Also: Xerox Office Systems Division, Systems Development Department, TR OSD-T8402, December, 1984. 83 pages.
• Hayes 85: Philip J. Hayes , Pedro A. Szekely , Richard A. Lerner, Design alternatives for user interface management sytems based on experience with COUSIN, Proceedings of the SIGCHI conference on Human factors in computing systems, p.169-175, April 1985, San Francisco, California, United States
• Henderson 86: D. A. Henderson, Jr., The Trillium user interface design environment, Proceedings of the SIGCHI conference on Human factors in computing systems, p.221-227, April 13-17, 1986, Boston, Massachusetts, United States
• Hutchins 86: Edwin L. Hutchins, James D. Hollan, and Donald A. Norman. "Direct Manipulation Interfaces," User Centered System Design, Donald A. Norman and Stephen W. Draper, eds. Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1986. pp. 87-124.
• Jacob 85: Robert J.K. Jacob. "A State Transition Diagram Language for Visual Programming," IEEE Computer. VoI. 18, no. 8. Aug. 1985. pp. 51-59.
• Lieberman 82: Henry Lieberman. "Constructing Graphical User Interfaces by Example," Graphics Interface, '82, Toronto, Ontario, March 17-21, 1982. pp. 295-302.
• Myers 84: Brad A. Myers. "The User Interface for Sapphire," IEEE Computer Graphics and Applications. Vol. 4, no~ 12, December, 1984. pp. 13-23.
• Myers 86: B. A. Myers, Visual programming, programming by example, and program visualization: a taxonomy, Proceedings of the SIGCHI conference on Human factors in computing systems, p.59-66, April 13-17, 1986, Boston, Massachusetts, United States
• Myers prep: Brad A. Myers. Applying Visual Programming with Programming by Example and Constraints to User Interface Management Systems. PhD Thesis, Department of Computer Science, University of Toronto, Toronto, Ontario, Canada. In progress.
• Nix 86: Robert P. Nix, Editing by example, ACM Transactions on Programming Languages and Systems (TOPLAS), v.7 n.4, p.600-621, Oct. 1985 [doi>10.1145/4472.4476]
• Olsen 83: Dan R Olsen, Jr. , Elizabeth P. Dempsey, SYNGRAPH: A graphical user interface generator, Proceedings of the 10th annual conference on Computer graphics and interactive techniques, p.43-50, July 25-29, 1983, Detroit, Michigan, United States
• Olsen 84: Dan R. Olsen, Jr., William Buxton, Roger Ehrich, David J. Kasik, James R. Rhyne, and John Sibert. "A Context for User Interface Management," IEEE Computer Graphics and Applications. Vol. 4, no. 2. Dec. 1984. pp. 33- 42.
• Olsen 85: Dan R. Olsen, Jr. , Elizabeth P. Dempsey , Roy Rogge, Input/output linkage in a user interface management system, ACM SIGGRAPH Computer Graphics, v.19 n.3, p.191-197, Jul. 1985
• Pavlidis 85: Theo Pavlidis , Christopher J. Van Wyk, An automatic beautifier for drawings and illustrations, ACM SIGGRAPH Computer Graphics, v.19 n.3, p.225-234, Jul. 1985
• Pfaff 85: G. E. Pfaff, User Interface Management Systems, Springer-Verlag New York, Inc., Secaucus, NJ, 1985
• Shaw 75: David E. Shaw, William R. Swartout, and C. CordeI1 Green. "Inferring Lisp Programs from Examples," Fourth International Joint Conference on Artificial Intelligence. Tbilisi, USSR. Sept. 3-8, 1975. Vol. 1. pp. 260-267.
• Shneiderman 83: Ben Shneiderman. "Direct Manipulation: A Step Beyond Programming Languages," IEEE Computer. Vol. 16, no. 8. Aug. 1983. pp. 57-69.
• Shneiderman 86: B. Shneiderman, Seven plus or minus two central issues in human-computer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.343-349, April 13-17, 1986, Boston, Massachusetts, United States
• Smith 77: David Canfield Smith. Pygmalion: A Computer Program to Model and Stimulate Creative Thought. Basel, Stuttgart: Birkhat~ser, 1977. 187 pages.
• Smith 82: David Canfield Smith, Charles Irby, Ralph Kimball, Bill Verplank, and Erik Harslem. "Designing the Star User Interface," Byte Magazine, April 1982, pp. 242-282.
• Tanner 85: Peter P. Tanner and William A.S. Buxton. "Some Issues in Future User Interface Management System (UIMS) Development," in User Interface Management Systems, Gunther R. Pfaff, ed. Berlin: Springer-Verlag, I985. pp. 67-79.
• Tesler 81: Larry Tesler. 'Toe Smalltalk Environment," Byte Magazine. August 198I, pp. 90-147.
• Thomas 83: James J. Thomas , Griffith Hamlin, Graphical input interaction technique (GIIT), ACM SIGGRAPH Computer Graphics, v.17 n.1, p.5-30, January 1983 [doi>10.1145/988584.988585]
• Williams 83: Gregg Williams. "The Lisa Computer System," Byte Magazine, February 1983, pp. 33-50.
• Williams 84: Gregg Williams. "The Apple Macintosh Computer," Byte Magazine. February 1984. pp. 30-54.
• Xerox 83: Xerox Corporation. Interlisp Reference Manual. Pasadena, CA. October, 1983.

INDEX TERMS

Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.2 Automatic Programming
          Subjects: Program synthesis
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques

General Terms:
Human Factors