Learning information intent via observation

ABSTRACT

Users in an organization frequently request help by sending request messages to assistants that express information intent: an intention to update data in an information system. Human assistants spend a significant amount of time and effort processing these requests. For example, human resource assistants process requests to update personnel records, and executive assistants process requests to schedule conference rooms or to make travel reservations. To process the intent of a request, assistants read the request and then locate, complete, and submit a form that corresponds to the expressed intent. Automatically or semi-automatically processing the intent expressed in a request on behalf of an assistant would ease the mundane and repetitive nature of this kind of work.For a well-understood domain, a straightforward application of natural language processing techniques can be used to build an intelligent form interface to semi-automatically process information intent request messages. However, high performance parsers are based on machine learning algorithms that require a large corpus of examples that have been labeled by an expert. The generation of a labeled corpus of requests is a major barrier to the construction of a parser. In this paper, we investigate the construction of a natural language processing system and an intelligent form system that observes an assistant processing requests. The intelligent form system then generates a labeled training corpus by interpreting the observations. This paper reports on the measurement of the performance of the machine learning algorithms based on real data. The combination of observations, machine learning and interaction design produces an effective intelligent form interface based on natural language processing.

REFERENCES

• 1: William W. Cohen, Einat Minkov, Anthony Tomasic, Learning to Understand Web Site Update Requests, in Proceedings of IJCAI, 2005, pp 1028--1033.
• 2: Anthony Tomasic , John Zimmerman , Isaac Simmons, Linking messages and form requests, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111472]
• 3: Ray Mooney, Learning semantic parsers: An important but under-studied problem, in Working notes of the AAAI spring symposium on language learning, 2004.
• 4: Anthony Tomasic, William Cohen, Susan Fussell, John Zimmerman, Marina Kobayashi, Einat Minkov, Nathan Halstead, Ravi Mosur, and Jason Hum, Learning to Navigate Web Forms, in Workshop on Information Integration on the Web (IIWEB), 2004.
• 5: Andrea Lockerd , Huy Pham , Taly Sharon , Ted Selker, Mr.Web: an automated interactive webmaster, CHI '03 extended abstracts on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/765891.766007]
• 6: William W. Cohen, Minorthird: Methods for Identifying Names and Ontological Relations in Text using Heuristics for Inducing Regularities from Data, http://minorthird.sourceforge.net.
• 7: John Zimmerman , Anthony Tomasic , Isaac Simmons , Ian Hargraves , Ken Mohnkern , Jason Cornwell , Robert Martin McGuire, Vio: a mixed-initiative approach to learning and automating procedural update tasks, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240843]
• 8: Eric Horvitz, Principles of mixed-initiative user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.159-166, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303030]
• 9: Victoria Bellotti , Nicolas Ducheneaut , Mark Howard , Ian Smith, Taking email to task: the design and evaluation of a task management centered email tool, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642672]
• 10: Oren Etzioni, Alon Halevy, Henry Levy, and Luke McDowell, Semantic Email: Adding Lightweight Data Manipulation Capabilities to the Email Habitat. International Workshop on the Web and Databases (WebDB), June 12-13, 2003, San Diego, California.
• 11: Alon Halevy, Oren Etzioni, AnHai Doan, Zachary Ives, Jayant Madhavan, Luke McDowell, Igor Tatarinov, Crossing the Structure Chasm in Conference on Innovated Data Systems Research (CIDR), 2003.
• 12: Bozena H. Thompson , Frederick B. Thompson, Introducing ask, a simple knowledgeable system, Proceedings of the first conference on Applied natural language processing, February 01-03, 1983, Santa Monica, California [doi>10.3115/974194.974198]
• 13: Frank Meng, A natural language interface for information retrieval from forms on the World Wide Web, Proceeding of the 20th international conference on Information Systems, p.540-545, December 12-15, 1999, Charlotte, North Carolina, United States
• 14: P. R. Cohen , M. Dalrymple , D. B. Moran , F. C. Pereira , J. W. Sullivan, Synergistic use of direct manipulation and natural language, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.227-233, March 1989
• 15: William W. Cohen, Pradeep Ravikumar, Stephen Fienberg, A Comparison of String Distance Metrics for Name-Matching Tasks in Workshop on Information Integration on the Web (IIWEB), 2003.
• 16: Terry Winograd, Understanding Natural Language, Ph.D. thesis, Academic Press, 1972.
• 17: Rohit J. Kate, Yuk Wah Wong, Raymond J. Mooney, Learning to Transform Natural to Formal Languages. Proceedings of AAAI, 2005.
• 18: Einat Minkov, Richard C. Wang, Anthony Tomasic, William W. Cohen, NER Systems that Suit User's Preferences: Adjusting the Recall-Precision Trade-off for Entity Extraction in HLT/NAACL, 2006.
• 19: SecondString. http://secondstring.sourceforge.net/
• 20: John D. Lafferty , Andrew McCallum , Fernando C. N. Pereira, Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data, Proceedings of the Eighteenth International Conference on Machine Learning, p.282-289, June 28-July 01, 2001
• 21: Robert E. Schapire , Yoram Singer, Improved boosting algorithms using confidence-rated predictions, Proceedings of the eleventh annual conference on Computational learning theory, p.80-91, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279960]
• 22: W. Cohen, V. Carvalho, and T. Mitchell, Learning to classify email into "speech acts", in Conference Empirical Methods in Natural Language Processing, 2004.
• 23: Mark Dredze , Tessa Lau , Nicholas Kushmerick, Automatically classifying emails into activities, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111471]
• 24: Jianqiang Shen , Lida Li , Thomas G. Dietterich , Jonathan L. Herlocker, A hybrid learning system for recognizing user tasks from desktop activities and email messages, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111473]
• 25: Ivan Felligi, Alan Sunter, "A theory for record linkage", Journal of the American Statistical Society, 64:1183--1210, 1969.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Natural language

General Terms:
Algorithms, Experimentation, Human Factors

Keywords:
domestication, information intent, weak labeling