Learning to generalize for complex selection tasks

ABSTRACT

Selection tasks are common in modern computer interfaces: we are often required to select a set of files, emails, data entries, and the like. File and data browsers have sorting and block selection facilities to make these tasks easier, but for complex selections there is little to aid the user without writing complex search queries. We propose an interactive machine learning solution to this problem called "smart selection," in which the user selects and deselects items as inputs to a selection classifier which attempts at each step to correctly generalize to the user's target state. Furthermore, we take advantage of our data on how users perform selection tasks over many sessions, and use it to train a label regressor that models their generalization behavior: we call this process learning to generalize. We then combine the user's explicit labels as well the label regressor outputs in the selection classifier to predict the user's desired selections. We show that the selection classifier alone takes dramatically fewer mouse clicks than the standard file browser, and when used in conjunction with the label regressor, the predictions of the classifier are significantly more accurate with respect to the target selection state.

REFERENCES

• 1: Xinlong Bao , Jonathan L. Herlocker , Thomas G. Dietterich, Fewer clicks and less frustration: reducing the cost of reaching the right folder, Proceedings of the 11th international conference on Intelligent user interfaces, January 29-February 01, 2006, Sydney, Australia [doi>10.1145/1111449.1111490]
• 2: Bishop, C. "Linear Models for Classification." In Pattern Recognition and Machine Learning. Springer, 2006.
• 3: Avrim Blum , Tom Mitchell, Combining labeled and unlabeled data with co-training, Proceedings of the eleventh annual conference on Computational learning theory, p.92-100, July 24-26, 1998, Madison, Wisconsin, United States [doi>10.1145/279943.279962]
• 4: Allen Cypher, Eager: programming repetitive tasks by demonstration, Watch what I do: programming by demonstration, MIT Press, Cambridge, MA, 1993
• 5: Jerry Alan Fails , Dan R. Olsen, Jr., Interactive machine learning, Proceedings of the 8th international conference on Intelligent user interfaces, January 12-15, 2003, Miami, Florida, USA [doi>10.1145/604045.604056]
• 6: James Fogarty , Desney Tan , Ashish Kapoor , Simon Winder, CueFlik: interactive concept learning in image search, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357061]
• 7: Yoav Freund , Robert E. Schapire, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences, v.55 n.1, p.119-139, Aug. 1997 [doi>10.1006/jcss.1997.1504]
• 8: Kristjansson, T. Culotta, A. Viola, P. McCallum, A. "Interactive Information Extraction with Constrained Conditional Random Fields." In Proc. of AAAI 2004.
• 9: Dan Maynes-Aminzade , Terry Winograd , Takeo Igarashi, Eyepatch: prototyping camera-based interaction through examples, Proceedings of the 20th annual ACM symposium on User interface software and technology, October 07-10, 2007, Newport, Rhode Island, USA [doi>10.1145/1294211.1294219]
• 10: Robert C. Miller , Brad A. Myers, Multiple selections in smart text editing, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502734]
• 11: Mitchell, T.M. "Version Spaces: A Candidate Elimination Approach to Rule Learning." In Proc. of IJCAI 1977.
• 12: Platt, J.C. Burges, C.J.C. Swenson, S. Weare, C. Zheng, A. "Learning a Gaussian Process Prior for Automatically Generating Music Playlists." In Proc. of NIPS 2002.
• 13: Thrun, S. "Is Learning the n-th Thing Any Easier Than Learning the First?" In Proc. of NIPS 1996.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Algorithms, Experimentation, Human Factors

Keywords:
file selection, interactive selection, learning by example, learning to generalize, learning user models, meta-learning, programming by demonstration, transfer learning, user modeling