Why and why not explanations improve the intelligibility of context-aware intelligent systems

ABSTRACT

Context-aware intelligent systems employ implicit inputs, and make decisions based on complex rules and machine learning models that are rarely clear to users. Such lack of system intelligibility can lead to loss of user trust, satisfaction and acceptance of these systems. However, automatically providing explanations about a system's decision process can help mitigate this problem. In this paper we present results from a controlled study with over 200 participants in which the effectiveness of different types of explanations was examined. Participants were shown examples of a system's operation along with various automatically generated explanations, and then tested on their understanding of the system. We show, for example, that explanations describing why the system behaved a certain way resulted in better understanding and stronger feelings of trust. Explanations describing why the system did not behave a certain way, resulted in lower understanding yet adequate performance. We discuss implications for the use of our findings in real-world context-aware applications.

REFERENCES

• 1: Amazon Mechanical Turk. http://www.mturk.com. Retrieved 16 Sep 2008.
• 2: Anderson, J. R., Corbett, A. T., Koedinger, K. R.,&Pelletier, R. (1995). Cognitive Tutors: Lessons Learned. Journal of the Learning Sciences, 4(2), 167--207.
• 3: Andrews, R., Diederich, J.&Tickle, A. (1995). A Survey and Critique of Techniques for Extracting Rules from Trained Artificial Neural Networks. Knowledge Based Systems. 8: 373--389.
• 4: Assad, M., Carmichael, D.J., Kay, J.&Kummerfeld, B. (2007). PersonisAD: Distributed, Active, Scrutable Model Framework for Context-Aware Services. Pervasive 2007, 55--72.
• 5: Daniel Avrahami , Scott E. Hudson, Responsiveness in instant messaging: predictive models supporting inter-personal communication, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124881]
• 6: Victoria Bellotti , Keith Edwards, Intelligibility and accountability: human considerations in context-aware systems, Human-Computer Interaction, v.16 n.2, p.193-212, December 2001 [doi>10.1207/S15327051HCI16234_05]
• 7: Victoria Bellotti , Maribeth Back , W. Keith Edwards , Rebecca E. Grinter , Austin Henderson , Cristina Lopes, Making sense of sensing systems: five questions for designers and researchers, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503450]
• 8: Keith Cheverst , Hee Eon Byun , Dan Fitton , Corina Sas , Chris Kray , Nicolas Villar, Exploring Issues of User Model Transparency and Proactive Behaviour in an Office Environment Control System, User Modeling and User-Adapted Interaction, v.15 n.3-4, p.235-273, August 2005 [doi>10.1007/s11257-005-1269-8]
• 9: Davis, R., Buchanan, B., and Shortliffe, E. (1977). Production rules as a representation for a knowledge-based consultation program. Artificial Intelligence, 8(1): 15--45.
• 10: Anind K. Dey , Alan Newberger, Support for context-aware intelligibility and control, Proceedings of the 27th international conference on Human factors in computing systems, April 04-09, 2009, Boston, MA, USA [doi>10.1145/1518701.1518832]
• 11: Dourish, P., Adler, A.&Smith, B.C. (1996). Organising User Interfaces Around Reflective Accounts. Reflection'96, 235--244.
• 12: Mary T. Dzindolet , Scott A. Peterson , Regina A. Pomranky , Linda G. Pierce , Hall P. Beck, The role of trust in automation reliance, International Journal of Human-Computer Studies, v.58 n.6, p.697-718, June 2003 [doi>10.1016/S1071-5819(03)00038-7]
• 13: Google Web Toolkit. http://code.google.com/webtoolkit/. Retrieved 16 Sep 2008.
• 14: Shirley Gregor , Izak Benbasat, Explanations from intelligent systems: theoretical foundations and implications for practice, MIS Quarterly, v.23 n.4, p.497-530, Dec. 1999 [doi>10.2307/249487]
• 15: Jonathan L. Herlocker , Joseph A. Konstan , John Riedl, Explaining collaborative filtering recommendations, Proceedings of the 2000 ACM conference on Computer supported cooperative work, p.241-250, December 2000, Philadelphia, Pennsylvania, United States [doi>10.1145/358916.358995]
• 16: Andrew J. Ko , Brad A. Myers, Designing the whyline: a debugging interface for asking questions about program behavior, Proceedings of the SIGCHI conference on Human factors in computing systems, p.151-158, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985712]
• 17: Martin Možina , Janez Demšar , Michael Kattan , Blaž Zupan, Nomograms for visualization of naive Bayesian classifier, Proceedings of the 8th European Conference on Principles and Practice of Knowledge Discovery in Databases, p.337-348, September 20-24, 2004, Pisa, Italy
• 18: McGuinness, D.L., Glass, A., Wolverton, M.&Pinheiro da Silva, P. (2007). A Categorization of Explanation Questions for Task Processing Systems. AAAI Workshop on Explanation-Aware Computing.
• 19: Muir, B. (1994). Trust in automation: Part i. theoretical issues in the study of trust and human intervention in automated systems. Ergonomics, 37(11): 1905--1922.
• 20: Brad A. Myers , David A. Weitzman , Andrew J. Ko , Duen H. Chau, Answering why and why not questions in user interfaces, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124832]
• 21: Norman, Donald A. (1988). The Design of Everyday Things. New York, Doubleday.
• 22: Conor Nugent , Pádraig Cunningham, A Case-Based Explanation System for Black-Box Systems, Artificial Intelligence Review, v.24 n.2, p.163-178, October 2005 [doi>10.1007/s10462-005-4609-5]
• 23: B. Schilit , N. Adams , R. Want, Context-Aware Computing Applications, Proceedings of the 1994 First Workshop on Mobile Computing Systems and Applications, p.85-90, December 08-09, 1994 [doi>10.1109/WMCSA.1994.16]
• 24: Joe Tullio , Anind K. Dey , Jason Chalecki , James Fogarty, How it works: a field study of non-technical users interacting with an intelligent system, Proceedings of the SIGCHI conference on Human factors in computing systems, April 28-May 03, 2007, San Jose, California, USA [doi>10.1145/1240624.1240630]
• 25: J. Ross Quinlan, C4.5: programs for machine learning, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1993
• 26: Weiser, M. (1991). The computer for the 21st century. Scientific American, 265(3): 94--104.
• 27: Weiser, M.&Brown, J.S., (1995). Designing Calm Technology. PowerGrid Journal v1.01.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.m Miscellaneous

General Terms:
Human Factors

Keywords:
context-aware, explanations, intelligibility