Generating remote control interfaces for complex appliances

ABSTRACT

The personal universal controller (PUC) is an approach for improving the interfaces to complex appliances by introducing an intermediary graphical or speech interface. A PUC engages in two-way communication with everyday appliances, first downloading a specification of the appliance's functions, and then automatically creating an interface for controlling that appliance. The specification of each appliance includes a high-level description of every function, a hierarchical grouping of those functions, and dependency information, which relates the availability of each function to the appliance's state. Dependency information makes it easier for designers to create specifications and helps the automatic interface generators produce a higher quality result. We describe the architecture that supports the PUC, and the interface generators that use our specification language to build high-quality graphical and speech interfaces.

REFERENCES

• 1: Marc Abrams , Constantinos Phanouriou , Alan L. Batongbacal , Stephen M. Williams , Jonathan E. Shuster, UTML: an appliance-independent XML user interface language, Proceeding of the eighth international conference on World Wide Web, p.1695-1708, May 1999, Toronto, Canada
• 2: Maddy D. Brouwer-Janse , Raymond W. Bennett , Takaya Endo , Floris L. van Nes , Hugo J. Strubbe , Donald R. Gentner, Interfaces for consumer products: “how to camouflage the computer?”, Proceedings of the SIGCHI conference on Human factors in computing systems, p.287-290, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142816]
• 3: CMU, "Carnegie Mellon Pronuncing Dictionary," 1998. http://www.speech.cs.cmu.edu/cgi-bin/cmudict.
• 4: Dennis J. M. J. de Baar , James D. Foley , Kevin E. Mullet, Coupling application design and user interface design, Proceedings of the SIGCHI conference on Human factors in computing systems, p.259-266, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142806]
• 5: K. F. Eustice , T. J. Lehman , A. Morales , M. C. Munson , S. Edlund , M. Guillen, A universal information appliance, IBM Systems Journal, v.38 n.4, p.575-601, Dec. 1999
• 6: Haartsen, J., Naghshineh, M., Inouye, J., Joeressen, O.J., and Allen, W., "Bluetooth: Vision, Goals, and Architecture." ACM Mobile Computing and Communications Review, 1998. 2(4): pp. 38-45. Oct. www.bluetooth.com.
• 7: HAVi, "Home Audio/Video Interoperability," 2002. http://www.havi.org.
• 8: Todd D. Hodes , Randy H. Katz , Edouard Servan-Schreiber , Lawrence Rowe, Composable ad-hoc mobile services for universal interaction, Proceedings of the 3rd annual ACM/IEEE international conference on Mobile computing and networking, p.1-12, September 26-30, 1997, Budapest, Hungary [doi>10.1145/262116.262121]
• 9: homeautonz, "Home Automation Webring," 2002. http://c.webring.com/webring?ring=homeauto;list.
• 10: inVoca, "inVoca Universal Remote," http://www.invoca.com.
• 11: Won Chul Kim , James D. Foley, Providing high-level control and expert assistance in the user interface presentation design, Proceedings of the SIGCHI conference on Human factors in computing systems, p.430-437, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169346]
• 12: Brad A. Myers, Using handhelds and PCs together, Communications of the ACM, v.44 n.11, p.34-41, Nov. 2001 [doi>10.1145/384150.384159]
• 13: Requirements for Automatically Generating Multi-Modal Interfaces for Complex Appliances, Proceedings of the 4th IEEE International Conference on Multimodal Interfaces, p.377, October 14-16, 2002 [doi>10.1109/ICMI.2002.1167024]
• 14: Jeffrey W. Nichols, Using handhelds as controls for everyday appliances: a paper prototype study, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634324]
• 15: D. R. Olsen, Jr., A programming language basis for user interface, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.171-176, March 1989
• 16: Dan R. Olsen, Jr. , Sean Jefferies , Travis Nielsen , William Moyes , Paul Fredrickson, Cross-modal interaction using XWeb, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.191-200, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354764]
• 17: Philips, Pronto Intelligent Remote Control. Philips Consumer Electronics, 2001. http://www.pronto.philips.com/.
• 18: Shankar Ponnekanti , Brian Lee , Armando Fox , Pat Hanrahan , Terry Winograd, ICrafter: A Service Framework for Ubiquitous Computing Environments, Proceedings of the 3rd international conference on Ubiquitous Computing, p.56-75, September 30-October 02, 2001, Atlanta, Georgia, USA
• 19: Rosenfeld, R., "Universal Speech Interfaces Web Site," 2002. http://www.cs.cmu.edu/~usi/.
• 20: Ronald Rosenfeld , Dan Olsen , Alex Rudnicky, Universal speech interfaces, interactions, v.8 n.6, p.34-44, Nov./Dec. 2001 [doi>10.1145/384076.384085]
• 21: Shriver, S., Black, A.W., Rosenfeld, R. "Audio Signals in Speech Interfaces," in ICSLP. 2000.
• 22: Stefanie Shriver , Arthur Toth , Xiaojin Zhu , Alex Rudnicky , Roni Rosenfeld, A unified design for human-machine voice interaction, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634214]
• 23: Sproat, R., Hunt, A., Ostendorf, P., Taylor, P., Black, A., Lenzo, K., Edgington, M. "SABLE: A Standard for TTS Markup," in International Conference on Spoken Language Processing. 1998. Sydney, Australia:
• 24: Sun, Jini Connection Technology. Sun Microsystems, http://www.sun.com/jini/, 2000.
• 25: Pedro Szekely , Ping Luo , Robert Neches, Beyond interface builders: model-based interface tools, Proceedings of the SIGCHI conference on Human factors in computing systems, p.383-390, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169305]
• 26: UPnP, "Universal Plug and Play Forum," 2002. http://www.upnp.org.
• 27: Brad Vander Zanden , Brad A. Myers, Automatic, look-and-feel independent dialog creation for graphical user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.27-34, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97248]
• 28: Wayne Ward, The CMU air travel information service: understanding spontaneous speech, Proceedings of the workshop on Speech and Natural Language, p.127-129, June 24-27, 1990, Hidden Valley, Pennsylvania [doi>10.3115/116580.116621]
• 29: Charles Wiecha , William Bennett , Stephen Boies , John Gould , Sharon Greene, ITS: a tool for rapidly developing interactive applications, ACM Transactions on Information Systems (TOIS), v.8 n.3, p.204-236, July 1990 [doi>10.1145/98188.98194]
• 30: Gottfried Zimmermann , Gregg Vanderheiden , Al Gilman, Prototype implementations for a universal remote console specification, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506454]

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.1 COMPUTATION BY ABSTRACT DEVICES
      F.1.2 Modes of Computation
          Subjects: Interactive and reactive computation

Additional Classification:
  B. Hardware
  B.4 INPUT/OUTPUT AND DATA COMMUNICATIONS
      B.4.2 Input/Output Devices
          Subjects: Voice

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen); Interaction styles (e.g., commands, menus, forms, direct manipulation); Voice I/O

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Speech recognition and synthesis

General Terms:
Algorithms, Experimentation, Human Factors, Verification

Keywords:
appliances, handheld computers, pebbles, personal digital assistants (PDAs), personal universal controller (PUC), remote control, universal speech interface (USI)