What were you thinking?: filling in missing dataflow through inference in learning from demonstration

ABSTRACT

Recent years have seen a resurgence of interest in programming by demonstration. As end users have become increasingly sophisticated, computer and artificial intelligence technology has also matured, making it feasible for end users to teach long, complex procedures. This paper addresses the problem of learning from demonstrations involving unobservable (e.g., mental) actions. We explore the use of knowledge base inference to complete missing dataflow and investigate the approach in the context of the CALO cognitive personal desktop assistant. We experiment with the Pathfinder utility, which efficiently finds all the relationships between any two objects in the CALO knowledge base. Pathfinder often returns too many paths to present to the user and its default shortest path heuristic sometimes fails to identify the correct path. We develop a set of filtering techniques for narrowing down the results returned by Pathfinder and present experimental results showing that these techniques effectively reduce the alternative paths to a small, meaningful set suitable for presentation to a user.

REFERENCES

• 1: Allen, J., Chambers, N., Ferguson, G., Galescu, L., Jung, H., Swift, M., and Taysom, W. (2007). PLOW: a collaborative task learning agent. Proc. AAAI 2007.
• 2: Ken Barker , Bruce Porter , Peter Clark, A library of generic concepts for composing knowledge bases, Proceedings of the 1st international conference on Knowledge capture, October 22-23, 2001, Victoria, British Columbia, Canada [doi>10.1145/500737.500744]
• 3: Hugh Beyer , Karen Holtzblatt, Contextual design: defining customer-centered systems, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 4: Jim Blythe, Task learning by instruction in tailor, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040874]
• 5: Blythe, J., Kapoor, D., Knoblock, C. A., and Lerman, K. Information integration for the masses. J. UCS Special Issue on Wrapping Web Data Islands, 2008.
• 6: Burstein, M., Laddaga, R., McDonald, D., Benyo, B., Roberston, P., Hussain, T., Brinn, M., and McDermott, D. POIROT--Integrated learning of Web service procedures. Proc. AAAI 2008.
• 7: CALO: Cognitive Agent that Learns and Organizes. http://caloproject.sri.com/.
• 8: Chaudhri, V. K., Cheyer, A., Giuli, R., Jarrold, B., Myers, K. L., and Niekrasz, J. A case study in engineering a knowledge base for a personal assistant. Proc. Semantic Desktop and Social Semantic Collaboration Workshop, 2006.
• 9: Allen Cypher , Daniel C. Halbert , David Kurlander , Henry Lieberman , David Maulsby , Brad A. Myers , Alan Turransky, Watch what I do: programming by demonstration, MIT Press, Cambridge, MA, 1993
• 10: Eker, S., Lee, T. J., and Gervasio, M. Iteration learning by demonstration. Proc. AAAI Spring Symposium on Agents that Learn from Human Teachers, 2008.
• 11: Garland, A. and Lesh, N. Learning hierarchical task models by demonstration. MERL Technical Report TR2003-1, Mitsubishi Electric Research Laboratories, 2003.
• 12: Gervasio, M., Lee, T. J., and Eker, S. Learning email procedures for the desktop. Proc. AAAI 2008 Workshop on Enhanced Messaging, AAAI Press (2008).
• 13: Huffman, S. and Laird, J. Flexibly instructable agents, Journal of AI Research, 3, 1995.
• 14: Exploiting Local Similarity for Indexing Paths in Graph-Structured Data, Proceedings of the 18th International Conference on Data Engineering, p.129, February 26-March 01, 2002
• 15: Lieberman, H. (Ed.). Your Wish is My Command: Programming by Example. Morgan Kaufmann, San Francisco, CA, 2001.
• 16: Greg Little , Tessa A. Lau , Allen Cypher , James Lin , Eben M. Haber , Eser Kandogan, Koala: capture, share, automate, personalize business processes on the web, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240767]
• 17: David Morley , Karen Myers, The SPARK Agent Framework, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, p.714-721, July 19-23, 2004, New York, New York [doi>10.1109/AAMAS.2004.267]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Induction

Additional Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning
          Subjects: Knowledge acquisition

General Terms:
Algorithms, Experimentation, Human Factors, Performance

Keywords:
end-user programming, knowledge-base inference, programming by demonstration, programming by example, task learning