Stencils-based tutorials: design and evaluation

ABSTRACT

Users of traditional tutorials and help systems often have difficulty finding the components described or pictured in the procedural instructions. Users also unintentionally miss steps, and perform actions that the documentation's authors did not intend, moving the application into an unknown state. We introduce Stencils, an interaction technique for presenting tutorials that uses translucent colored stencils containing holes that direct the user's attention to the correct interface component and prevent the user from interacting with other components. Sticky notes on the stencil's surface provide necessary tutorial material in the context of the application. In a user study comparing a Stencils-based and paper-based version of the same tutorial in Alice, a complex software application designed to teach introductory computer programming, we found that users of a Stencils-based tutorial were able complete the tutorial 26% faster, with fewer errors, and less reliance on human assistance. Users of the Stencils-based and paper-based tutorials attained statistically similar levels of learning.

REFERENCES

• 1: Alice. http://www.alice.org
• 2: Ron Baecker, Showing instead of telling, Proceedings of the 20th annual international conference on Computer documentation, p.10-16, October 20-23, 2002, Toronto, Ontario, Canada [doi>10.1145/584955.584957]
• 3: Bartlett, J. Transparent Controls for Interactive Graphics. WRL Technical Note TN-30, Digital Equipment Corporation, Palo Alto, CA, July 1992.
• 4: Eric A. Bier , Maureen C. Stone , Ken Pier , William Buxton , Tony D. DeRose, Toolglass and magic lenses: the see-through interface, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.73-80, September 1993 [doi>10.1145/166117.166126]
• 5: Eric A. Bier , Maureen C. Stone , Ken Fishkin , William Buxton , Thomas Baudel, A taxonomy of see-through tools, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.358-364, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191786]
• 6: Booher, H.R., Relative comprehensibility of pictorial information and printed words in proceduralized instructions. Human Factors 17, 3 (1975), 266--277.
• 7: John M. Carroll , Caroline Carrithers, Training wheels in a user interface, Communications of the ACM, v.27 n.8, p.800-806, Aug 1984 [doi>10.1145/358198.358218]
• 8: John M. Carroll , Mary Beth Rosson, Paradox of the active user, Interfacing thought: cognitive aspects of human-computer interaction, MIT Press, Cambridge, MA, 1987
• 9: Carroll, J., Smith-Kerker, P., Ford, J., and Mazur, S. The Minimal Manual. IBM RC 11637, 1986.
• 10: Richard Catrambone , John M. Carroll, Learning a word processing system with training wheels and guided exploration, Proceedings of the SIGCHI/GI conference on Human factors in computing systems and graphics interface, p.169-174, April 05-09, 1987, Toronto, Ontario, Canada
• 11: Matthew Conway , Steve Audia , Tommy Burnette , Dennis Cosgrove , Kevin Christiansen, Alice: lessons learned from building a 3D system for novices, Proceedings of the SIGCHI conference on Human factors in computing systems, p.486-493, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332481]
• 12: Dann, W., Cooper, S. and Pausch, R. Learning to Program with Alice: Beta Version. Pearson Prentice Hall, Upper Saddle River, NJ, USA, 2005.
• 13: Designing Coachmarks. http://www.developer.apple.com/techpubs/mac/AppleGuide/AppleGuide-24.html
• 14: Ken Fishkin , Maureen C. Stone, Enhanced dynamic queries via movable filters, Proceedings of the SIGCHI conference on Human factors in computing systems, p.415-420, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223960]
• 15: Susan D. Goodall, Online help: a part of documentation, Proceedings of the 10th annual international conference on Systems documentation, p.169-174, October 13-16, 1992, Ottawa, Ontario, Canada [doi>10.1145/147001.147024]
• 16: Susan D. Goodall, Online help in the real world, Proceedings of the 9th annual international conference on Systems documentation, p.21-29, October 1991, Chicago, Illinois, United States [doi>10.1145/122778.122783]
• 17: Guzdial, M. Software-Realized Scaffolding to Facilitate Programming for Science Learning. Interactive Learning Environments 4, 1 (1995), 1--44.
• 18: Susan M. Harrison, A comparison of still, animated, or nonillustrated on-line help with written or spoken instructions in a graphical user interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.82-89, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223915]
• 19: Scott E. Hudson , Roy Rodenstein , Ian Smith, Debugging lenses: a new class of transparent tools for user interface debugging, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.179-187, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263542]
• 20: Shari L. Jackson , Joseph Krajcik , Elliot Soloway, The design of guided learner-adaptable scaffolding in interactive learning environments, Proceedings of the SIGCHI conference on Human factors in computing systems, p.187-194, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274672]
• 21: Kevin Knabe, Apple guide: a case study in user-aided design of online help, Conference companion on Human factors in computing systems, p.286-287, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223355.223677]
• 22: Axel Kramer, Translucent patches—dissolving windows, Proceedings of the 7th annual ACM symposium on User interface software and technology, p.121-130, November 02-04, 1994, Marina del Rey, California, United States [doi>10.1145/192426.192474]
• 23: Julian Looser , Mark Billinghurst , Andy Cockburn, Through the looking glass: the use of lenses as an interface tool for Augmented Reality interfaces, Proceedings of the 2nd international conference on Computer graphics and interactive techniques in Australasia and South East Asia, June 15-18, 2004, Singapore [doi>10.1145/988834.988870]
• 24: Susan Palmiter , Jay Elkerton, An evaluation of animated demonstrations of learning computer-based tasks, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.257-263, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108906]
• 25: Susan Palmiter , Jay Elkerton , P. Baggett, Animated demonstrations vs. written instructions for learning procedural tasks: a preliminary investigation, International Journal of Man-Machine Studies, v.34 n.5, p.687-701, May 1991 [doi>10.1016/0020-7373(91)90019-4]
• 26: Chris Quintana , Jim Eng , Andrew Carra , Hsin-Kai Wu , Elliot Soloway, Symphony: a case study in extending learner-centered design through process space analysis, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.473-480, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303136]
• 27: John Rieman, A field study of exploratory learning strategies, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.3, p.189-218, Sept. 1996 [doi>10.1145/234526.234527]
• 28: Schneiderman, B. Direct manipulation: A step beyond programming languages. IEEE Computer 16, 8 (1983), 57--69.
• 29: Selker, T., Barber, R., and Kelley, R. Effective, Selective Presentation of Help Material in a Graphical Environment: Experience with COACH/2, a graphical adaptive help system. IBM Tech Report, 1996.
• 30: Shashaani, L. Gender-Differences in Computer Experience and its Influence on Computer Attitudes. Journal of Educational Computing Research 11, 4 (1994), 347--367.
• 31: Elliot Soloway , Mark Guzdial , Kenneth E. Hay, Learner-centered design: the challenge for HCI in the 21st century, interactions, v.1 n.2, p.36-48, April 1994 [doi>10.1145/174809.174813]
• 32: Piyawadee Sukaviriya , Ellen Isaacs , Krishna Bharat, Multimedia help: a prototype and an experiment, Proceedings of the SIGCHI conference on Human factors in computing systems, p.433-434, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142889]
• 33: Raven Wallace , Elliot Soloway , Joseph Krajcik , Nathan Bos , Joseph Hoffman , Heather Eccleston Hunter , Dan Kiskis , Elisabeth Klann , Greg Peters , David Richardson , Ofer Ronen, ARTEMIS: learner-centered design of an information seeking environment for K-12 education, Proceedings of the SIGCHI conference on Human factors in computing systems, p.195-202, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274673]
• 34: Windows Family. http://www.windows.com.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Training, help, and documentation

Keywords:
interaction technique, transparent overlay, tutorials, user interface design