Justified advice: a semi-naturalistic study of advisory strategies

ABSTRACT

“Wizard of Oz” techniques were used to observe the interaction between users of a statistical package and a human playing the role of an simulated intelligent advisory system. The results emphasized the complexities of the advisory process. More than half of the clients' requests sought help on planning actions toward achieving task goals. Further, protocols collected from the advisor while advice was given revealed the importance of constructing models of the user's current and past interaction with the application, and of addressing the high-level goals that underlie clients' explicit questions. The relevance of these findings to the development of intelligent advisory systems is discussed.

REFERENCES

• 1: Aaronson, A., & Carroll, J. M. The answer is in the question: A protocol study of intelligent help. Tech. Rept. 54235, IBM, Yorktown Heights, New York, July, 1986.
• 2: Amy Aaronson , John M. Carroll, Intelligent help in a one-shot dialog: a protocol study, Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, p.163-168, April 05-09, 1987, Toronto, Ontario, Canada
• 3: Alty, J., & Coombs, M.J. "Face-to-face guidance of university computer users- I. A study of advisory services.". International Journal of Man-Machine Studies 12 (1980), 389-405.
• 4: Bobrow, D. G., Kaplan, R.M., Kay, M., Norman, D. A., Thompson, H., & Winograd, T. "GUS, a frame-driven dialog system". Artificial Intelligence 8, 1 (1977), 155-173.
• 5: Chapanis, A. Man-computer research at Johns Hopkins. In R. A. Kasschau, R. Lachman, & K. Laughery, Ed., Information technology and psychology: Prospects for the future, University of Minnesota Press, Minneapolis, 1982, pp. 238-249.
• 6: Coombs, M. J., & Alty, J. "Face-to-face guidance of university computer users - II. Characterizing advisory interactions.". International Journal of Man-Machine Studies 12 (1980), 407-429.
• 7: Dominick, W. D. User interaction monitoring as a multi-level process. Proceedings of ASIS Annual Meeting, 1977.
• 8: Ericsson, K. A. & Simon, H. A. "Verbal reports as data.". Psychological Review 87 (1980), 215-251.
• 9: John D. Gould , John Conti , Todd Hovanyecz, Composing letters with a simulated listening typewriter, Communications of the ACM, v.26 n.4, p.295-308, April 1983 [doi>10.1145/2163.358100]
• 10: Hill, W. C. Two hidden-operator studies of advicegiving for a direct representation interface: Preliminary report. Tech. Rep. HI-169-87-P, MCC, Austin, 1987.
• 11: Jones, W. P., Cook, S. M., & Molloy, N. A study of a person-to-person advisory situation. Tech. Rept. HI-265-86, Microelectronics and Computer Technology Corporation, Austin, 1986.
• 12: Lang, K., Auld, R., & Lang, T. "The goals and methods of computer users.". International Journal of Man-Machine Studies 17(4) (1982), 375-399.
• 13: Lang, K., Lang, T., & Auld, R. "Support for users of operating systems and applications software.". International Journal of Man.Machine Studies 14(3) (1982), 269-282.
• 14: Malhotra, A., & Sheridan, P. B. Experimental determination of design requirements for a program explanation system. Tech. Rept. 25242, IBM, 1976.
• 15: J. McKendree , J. M. Carroll, Advising roles of a computer consultant, Proceedings of the SIGCHI conference on Human factors in computing systems, p.35-40, April 13-17, 1986, Boston, Massachusetts, United States
• 16: McKendree, J., & Zaback, J. Planning for advising. Tech. Rep. HI-172-87-P, Microeleetronics and Computer Technology Corporation, Au~,;tin, 1987.
• 17: Miller, j. R., Hill, W. C., McKendree, J., Masson, M. E. J., Blumenthal, B., Terveen, L., & Zaback, J. The role of the system image in intelligent user assistance. Proceedings of INTERACT'87, Amsterdam, 1987, pp. 885-890.
• 18: Norman, D. A. Cognitive Engineering. In Norman, D. A. & Draper, S. W., Ed., User Centered System Design, (pp. 31-61). Hillsdale, New Jersey: Lawrence Erlbaum Associates, 1986, pp. 31-61.
• 19: Martha E. Pollack, Information sought and information provided: an empirical study of user/expert dialogues, Proceedings of the SIGCHI conference on Human factors in computing systems, p.155-159, April 1985, San Francisco, California, United States
• 20: Martha Elizabeth Pollack, Inferring domain plans in question-answering, University of Pennsylvania, Philadelphia, PA, 1986
• 21: Stevens, C., & Miller, J. R. Advice objects: Direct manipulation techniques for advisory communication. Microelectronics and Computer Technology Corporation, Austin, 1987.
• 22: Woods, W. A., Kaplan, R., & Nash-Webber, 13. The lunar sciences natural language information system: Final Report. Tech. Rept. No. 2378, Bolt Beranek and Newman, Cambridge, Mas,;, 1972.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.2 Types of Systems
          Subjects: Decision support (e.g., MIS)

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Statistical computing

  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.1 Project and People Management
          Subjects: Management techniques (e.g., PERT/CPM)

General Terms:
Design, Human Factors, Management