A goal-oriented interface to consumer electronics using planning and commonsense reasoning

ABSTRACT

We are reaching a crisis with design of user interfaces for consumer electronics. Flashing 12:00 time indicators, push-and-hold buttons, and interminable modes and menus are all symptoms of trying to maintain a one-to-one correspondence between functions and physical controls, which becomes hopeless as the number of capabilities of devices grows. We propose instead to orient interfaces around the goals that users have for the use of devices.We present Roadie, a user interface agent that provides intelligent context-sensitive help and assistance for a network of consumer devices. Roadie uses a Commonsense knowledge base to map between user goals and functions of the devices, and an AI partial-order planner to provide mixed-initiative assistance with executing multi-step procedures and debugging help when things go wrong.

REFERENCES

• 1: Avrim Blum, Merrick Furst. Fast Planning Through Planning Graph Analysis. Proc. Of the 14th International Joint Conference on Artificial Intelligence (IJCAI 1997), Montreal, Canada, pages 1636--1642, 1997.
• 2: Boris de Ruyter, Richard van de Sluis. Challenges for End-User Development in Intelligent Environments. In Henry Lieberman, Fabio Paterno, Volker Wulf, eds, End-User Development, Kluwer Academic Publisher, (to appear).
• 3: Jose Espinosa, Henry Lieberman. EventNet: Inferring Temporal Relations Between Commonsense Events. Proc. Fourth Mexican International Conference on Artificial Intelligence, Springer Publisher. November 14-18, 2005. Monterrey, Nuevo Leon, MEXICO (to appear).
• 4: Yolanda Gil, Knowledge refinement in a reflective architecture, Proceedings of the twelfth national conference on Artificial intelligence (vol. 1), p.520-526, October 1994, Seattle, Washington, United States
• 5: Stephen S. Intille , Kent Larson , J. S. Beaudin , J. Nawyn , E. Munguia Tapia , P. Kaushik, A living laboratory for the design and evaluation of ubiquitous computing technologies, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1057062]
• 6: Chia-Hsun Jackie Lee , Leonardo Bonanni , Jose H. Espinosa , Henry Lieberman , Ted Selker, Augmenting kitchen appliances with a shared context using knowledge about daily events, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111533]
• 7: Henry Lieberman, Hugo Liu, Push Singh, and Barbara Barry. Beating Common Sense into Interactive Applications. AI Magazine 25(4): Winter 2005, 63--76.
• 8: Henry Lieberman , Earl Wagner, End-user debugging for e-commerce, Proceedings of the 8th international conference on Intelligent user interfaces, January 12-15, 2003, Miami, Florida, USA [doi>10.1145/604045.604092]
• 9: Henry Lieberman (Ed.). Your Wish is My Command: Programming By Example. The Morgan Kaufmann Series in Interactive Technologies, 2001.
• 10: Hugo Liu , Henry Lieberman , Ted Selker, GOOSE: A Goal-Oriented Search Engine with Commonsense, Proceedings of the Second International Conference on Adaptive Hypermedia and Adaptive Web-Based Systems, p.253-263, May 29-31, 2002
• 11: Jef Raskin, The humane interface: new directions for designing interactive systems, ACM Press/Addison-Wesley Publishing Co., New York, NY, 2000
• 12: Charles Rich , Candace L. Sidner, COLLAGEN: A Collaboration Manager for Software Interface Agents, User Modeling and User-Adapted Interaction, v.8 n.3-4, p.315-350, 1998 [doi>10.1023/A:1008204020038]
• 13: Charles Rich , Candy Sidner , Neal Lesh , Andrew Garland , Shane Booth , Markus Chimani, DiamondHelp: A Collaborative Interface Framework for Networked Home Appliances, Proceedings of the Fifth International Workshop on Smart Appliances and Wearable Computing (IWSAWC) (ICDCSW'05), p.514-519, June 06-10, 2005 [doi>10.1109/ICDCSW.2005.58]
• 14: Push Singh. The public acquisition of Commonsense knowledge. Proc. AAAI Spring Symposium on Acquiring (and Using) Linguistic (and World) Knowledge for Information Access, 2002.
• 15: Universal Plug and Play Device Architecture. 2000. Available: http://www.upnp.org/download/UPnPDA10_20000613.htm
• 16: Alexander Yates , Oren Etzioni , Daniel Weld, A reliable natural language interface to household appliances, Proceedings of the 8th international conference on Intelligent user interfaces, January 12-15, 2003, Miami, Florida, USA [doi>10.1145/604045.604075]
• 17: Gottfried Zimmermann, Gregg Vanderheiden, Al Gilman. Prototype Implementations for a Universal Remote Console Specification. ACM Special Interest Group on Computer-Human Interaction, pages 510--511, ACM Press, 2002.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

  J. Computer Applications
  J.7 COMPUTERS IN OTHER SYSTEMS
      Subjects: Consumer products

General Terms:
Design, Human Factors

Keywords:
commonsense reasoning, consumer electronics, goal-oriented interfaces, planning