Pen-top feedback for paper-based interfaces

ABSTRACT

Current paper-based interfaces such as PapierCraft, provide very little feedback and this limits the scope of possible interactions. So far, there has been little systematic exploration of the structure, constraints, and contingencies of feedback-mechanisms in paper-based interaction systems for paper-only environments. We identify three levels of feedback: discovery feedback (e.g., to aid with menu learning), status-indication feedback (e.g., for error detection), and task feedback (e.g., to aid in a search task). Using three modalities (visual, tactile, and auditory) which can be easily implemented on a pen-sized computer, we introduce a conceptual matrix to guide systematic research on pen-top feedback for paper-based interfaces. Using this matrix, we implemented a multimodal pen prototype demonstrating the potential of our approach. We conducted an experiment that confirmed the efficacy of our design in helping users discover a new interface and identify and correct their errors.

REFERENCES

• 1: Anoto, Development Guide for Service Enabled by Anoto Functionality. 2002, Anoto.
• 2: Toshifumi Arai , Dietmar Aust , Scott E. Hudson, PaperLink: a technique for hyperlinking from real paper to electronic content, Proceedings of the SIGCHI conference on Human factors in computing systems, p.327-334, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258782]
• 3: Ravin Balakrishnan , Pranay Patel, The PadMouse: facilitating selection and spatial positioning for the non-dominant hand, Proceedings of the SIGCHI conference on Human factors in computing systems, p.9-16, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274646]
• 4: Stephen A. Brewster, Using nonspeech sounds to provide navigation cues, ACM Transactions on Computer-Human Interaction (TOCHI), v.5 n.3, p.224-259, Sept. 1998 [doi>10.1145/292834.292839]
• 5: Campbell, C. S., S. Zhai, K. W. May, and P. P. Maglio. What You Feel Must Be What You See: Adding Tactile Feedback to the Trackpoint. Proceedings of INTERACT'99, pp. 383--390.
• 6: William W. Gaver , Randall B. Smith , Tim O'Shea, Effective sounds in complex systems: the ARKOLA simulation, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.85-90, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108857]
• 7: William W. Gaver, Synthesizing auditory icons, Proceedings of the INTERCHI '93 conference on Human factors in computing systems, p.228-235, May 1993, Amsterdam, The Netherlands
• 8: François Guimbretière, Paper augmented digital documents, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.51-60, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964702]
• 9: Jeremy M. Heiner , Scott E. Hudson , Kenichiro Tanaka, Linking and messaging from real paper in the Paper PDA, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.179-186, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322600]
• 10: Ken Hinckley , Gonzalo Ramos , Francois Guimbretiere , Patrick Baudisch , Marc Smith, Stitching: pen gestures that span multiple displays, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989866]
• 11: Ken Hinckley , Patrick Baudisch , Gonzalo Ramos , Francois Guimbretiere, Design and analysis of delimiters for selection-action pen gesture phrases in scriboli, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055035]
• 12: Walter Johnson , Herbert Jellinek , Leigh Klotz, Jr. , Ramana Rao , Stuart Card, Bridging the paper and electronic worlds: the paper user interface, Proceedings of the INTERCHI '93 conference on Human factors in computing systems, p.507-512, May 1993, Amsterdam, The Netherlands
• 13: Steinar Kristoffersen , Fredrik Ljungberg, “Making place” to make IT work: empirical explorations of HCI for mobile CSCW, Proceedings of the international ACM SIGGROUP conference on Supporting group work, p.276-285, November 14-17, 1999, Phoenix, Arizona, United States [doi>10.1145/320297.320330]
• 14: Gordon Paul Kurtenbach, The design and evaluation of marking menus, University of Toronto, Toronto, Ont., Canada, 1993
• 15: LeapFrog, Fly Pen, http://www.leapfrog.com. 2005.
• 16: Johnny C. Lee , Paul H. Dietz , Darren Leigh , William S. Yerazunis , Scott E. Hudson, Haptic pen: a tactile feedback stylus for touch screens, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029682]
• 17: Chunyuan Liao , François Guimbretière , Ken Hinckley, PapierCraft: a command system for interactive paper, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095074]
• 18: LightBlueOptics, Portable laser projector, http://www.lightblueoptics.com/. 2005.
• 19: Wendy E. Mackay , Guillaume Pothier , Catherine Letondal , Kaare Bøegh , Hans Erik Sørensen, The missing link: augmenting biology laboratory notebooks, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.571992]
• 20: Microsoft, http://www.microsoft.com. 2003.
• 21: Timothy Miller , Robert Zeleznik, An insidious Haptic invasion: adding force feedback to the X desktop, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.59-64, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288573]
• 22: Elizabeth D. Mynatt , Gerhard Weber, Nonvisual presentation of graphical user interfaces: contrasting two approaches, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.166-172, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191732]
• 23: Moira C. Norrie , Beat Signer, Switching Over to Paper: A New Web Channel, Proceedings of the Fourth International Conference on Web Information Systems Engineering, p.209, December 10-12, 2003
• 24: Ivan Poupyrev , Shigeaki Maruyama , Jun Rekimoto, Ambient touch: designing tactile interfaces for handheld devices, Proceedings of the 15th annual ACM symposium on User interface software and technology, October 27-30, 2002, Paris, France [doi>10.1145/571985.571993]
• 25: Ivan Poupyrev , Makoto Okabe , Shigeaki Maruyama, Haptic feedback for pen computing: directions and strategies, CHI '04 extended abstracts on Human factors in computing systems, April 24-29, 2004, Vienna, Austria [doi>10.1145/985921.986051]
• 26: Christophe Ramstein , Odile Martial , Aude Dufresne , Michel Carignan , Patrick Chassé , Philippe Mabilleau, Touching and hearing GUI's: design issues for the PC-Access system, Proceedings of the second annual ACM conference on Assistive technologies, p.2-9, April 11-12, 1996, Vancouver, British Columbia, Canada [doi>10.1145/228347.228349]
• 27: Nitin Sawhney , Chris Schmandt, Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.353-383, Sept. 2000 [doi>10.1145/355324.355327]
• 28: Bill N. Schilit , Gene Golovchinsky , Morgan N. Price, Beyond paper: supporting active reading with free form digital ink annotations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.249-256, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274680]
• 29: Lisa J. Stifelman, A tool to support speech and non-speech audio feedback generation in audio interfaces, Proceedings of the 8th annual ACM symposium on User interface and software technology, p.171-179, November 15-17, 1995, Pittsburgh, Pennsylvania, United States [doi>10.1145/215585.215970]
• 30: Pierre Wellner, Interacting with paper on the DigitalDesk, Communications of the ACM, v.36 n.7, p.87-96, July 1993 [doi>10.1145/159544.159630]
• 31: WIZCOM, http://www.wizcomtech.com/. 1995.
• 32: Ron Yeh , Chunyuan Liao , Scott Klemmer , François Guimbretière , Brian Lee , Boyko Kakaradov , Jeannie Stamberger , Andreas Paepcke, ButterflyNet: a mobile capture and access system for field biology research, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124859]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Design, Experimentation, Human Factors

Keywords:
feedback, multimodal, paper-based interfaces, pen interfaces