Fixing the program my computer learned: barriers for end users, challenges for the machine

ABSTRACT

The results of a machine learning from user behavior can be thought of as a program, and like all programs, it may need to be debugged. Providing ways for the user to debug it matters, because without the ability to fix errors users may find that the learned program's errors are too damaging for them to be able to trust such programs. We present a new approach to enable end users to debug a learned program. We then use an early prototype of our new approach to conduct a formative study to determine where and when debugging issues arise, both in general and also separately for males and females. The results suggest opportunities to make machine-learned programs more effective tools.

REFERENCES

• 1: Bandura, A. Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review 8, 2 (1977), 191--215.
• 2: Barry Becker , Ron Kohavi , Dan Sommerfield, Visualizing the simple Baysian classifier, Information visualization in data mining and knowledge discovery, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2001
• 3: Laura Beckwith , Margaret Burnett , Susan Wiedenbeck , Curtis Cook , Shraddha Sorte , Michelle Hastings, Effectiveness of end-user debugging software features: are there gender issues?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055094]
• 4: Margaret Burnett , Curtis Cook , Omkar Pendse , Gregg Rothermel , Jay Summet , Chris Wallace, End-user software engineering with assertions in the spreadsheet paradigm, Proceedings of the 25th International Conference on Software Engineering, May 03-10, 2003, Portland, Oregon
• 5: Chan, H. and Darwiche, A. When do numbers really matter? Journal of Artificial Intelligence Research, 17 (2002), 265--287.
• 6: Jiun-Hung Chen , Daniel S. Weld, Recovering from errors during programming by demonstration, Proceedings of the 13th international conference on Intelligent user interfaces, January 13-16, 2008, Gran Canaria, Spain [doi>10.1145/1378773.1378794]
• 7: Compeau, D. and Higgins, C. Application of social cognitive theory to training for computer skills. Information Systems Research, 6,2 (1995), 118--143.
• 8: Davies, S. P. Display-based problem solving strategies in computer programming, Proc. Wkshp. Empirical Studies of Programmers, Ablex, (1996), 59--76.
• 9: Alyssa Glass , Deborah L. McGuinness , Michael Wolverton, Toward establishing trust in adaptive agents, Proceedings of the 13th international conference on Intelligent user interfaces, January 13-16, 2008, Gran Canaria, Spain [doi>10.1145/1378773.1378804]
• 10: Grigoreanu, V., Cao, J., Kulesza, T., Bogart, C., Rector, K., Burnett, M., Wiedenbeck, S. Can feature design reduce the gender gap in end-user software development environments? Proc. VL/HCC 2008, IEEE, (2008).
• 11: Ko, A. J. Asking and answering questions about the causes of software behaviors, Ph.D. thesis available as Human-Computer Interaction Institute Technical Report CMU-CS-08-122 (2008).
• 12: Andrew J. Ko , Brad A. Myers , Htet Htet Aung, Six Learning Barriers in End-User Programming Systems, Proceedings of the 2004 IEEE Symposium on Visual Languages - Human Centric Computing, p.199-206, September 26-29, 2004 [doi>10.1109/VLHCC.2004.47]
• 13: Kononenko, I. Inductive and Bayesian learning in medical diagnosis. Applied Artificial Intelligence, 7, (1993), 317--337.
• 14: Carmen Lacave , Francisco J. Díez, A review of explanation methods for Bayesian networks, The Knowledge Engineering Review, v.17 n.2, p.107-127, June 2002 [doi>10.1017/S026988890200019X]
• 15: Lieberman, H. (ed.) Your Wish is My Command: Programming By Example, Morgan Kaufmann Publishers, Inc (2001).
• 16: Greg Little , Tessa A. Lau , Allen Cypher , James Lin , Eben M. Haber , Eser Kandogan, Koala: capture, share, automate, personalize business processes on the web, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240767]
• 17: Richard G. McDaniel , Brad A. Myers, Getting more out of programming-by-demonstration, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.442-449, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303127]
• 18: Meyers--Levy, J. Gender differences in information processing: A selectivity interpretation. P. Cafferata & A. Tybout, (Eds) Cognitive and Affective Responses to Advertising, Lexington Books (1989).
• 19: Brad A. Myers , David A. Weitzman , Andrew J. Ko , Duen H. Chau, Answering why and why not questions in user interfaces, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124832]
• 20: Kayur Patel , James Fogarty , James A. Landay , Beverly Harrison, Investigating statistical machine learning as a tool for software development, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357160]
• 21: Poulin, B., Eisner, R., Szafron, D., Lu, P., Greiner, R., Wishart, D. S., Fyshe, A., Pearcy, B., MacDonnell, C., and Anvik, J. Visual explanation of evidence in additive classifiers. Proc. IAAI, (2006).
• 22: Stuart J. Russell , Peter Norvig, Artificial intelligence: a modern approach, Prentice-Hall, Inc., Upper Saddle River, NJ, 1995
• 23: Simone Stumpf , Vidya Rajaram , Lida Li , Margaret Burnett , Thomas Dietterich , Erin Sullivan , Russell Drummond , Jonathan Herlocker, Toward harnessing user feedback for machine learning, Proceedings of the 12th international conference on Intelligent user interfaces, January 28-31, 2007, Honolulu, Hawaii, USA [doi>10.1145/1216295.1216316]
• 24: Simone Stumpf , Erin Sullivan , Erin Fitzhenry , Ian Oberst , Weng-Keen Wong , Margaret Burnett, Integrating rich user feedback into intelligent user interfaces, Proceedings of the 13th international conference on Intelligent user interfaces, January 13-16, 2008, Gran Canaria, Spain [doi>10.1145/1378773.1378781]
• 25: Neeraja Subrahmaniyan , Laura Beckwith , Valentina Grigoreanu , Margaret Burnett , Susan Wiedenbeck , Vaishnavi Narayanan , Karin Bucht , Russell Drummond , Xiaoli Fern, Testing vs. code inspection vs. what else?: male and female end users' debugging strategies, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357153]
• 26: Brad Vander Zanden , Brad A. Myers, Demonstrational and constraint-based techniques for pictorially specifying application objects and behaviors, ACM Transactions on Computer-Human Interaction (TOCHI), v.2 n.4, p.308-356, Dec. 1995 [doi>10.1145/212430.212443]
• 27: Earl J. Wagner , Henry Lieberman, Supporting user hypotheses in problem diagnosis, Proceedings of the 9th international conference on Intelligent user interfaces, January 13-16, 2004, Funchal, Madeira, Portugal [doi>10.1145/964442.964450]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

General Terms:
Design, Human Factors

Keywords:
debugging, end-user programming, machine learning