Turning a page on the digital annotation of physical books

ABSTRACT

The Graphical User Interface (GUI) has created an efficient work environment for many applications. However, when users are confined by keyboards and mice, they lose the ability to interact with the virtual world using habits from the real world. Our research examine how emerging modes of authorship, such as wikis, can be used to generate new possibilities for bringing atoms and bits together for digital annotation. Our goal is to combine the everyday habits in reading books with emerging digital possibilities.

In this paper, we present a prototype system called WikiTUI, which brings digital media to physical paper books. This system allows readers to access the digital world through fingertip interactions on books, and enables them to share information with other readers using wiki technology. WikiTUI not only bridges the gap between the digital and the physical worlds, but also facilitates multiple contributions to a reference base spanning across these worlds. We present user evaluations of the WikiTUI prototype and discuss research implications.

REFERENCES

• 1: Ahmad, S., A Usable Real-Time 3D Hand Tracker. IEEE Asilomar Conf. 1994, (1994), 1257--1261.
• 2: Kenneth M. Anderson , Richard N. Taylor , E. James Whitehead, Jr., Chimera: hypermedia for heterogeneous software development enviroments, ACM Transactions on Information Systems (TOIS), v.18 n.3, p.211-245, July 2000 [doi>10.1145/352595.352596]
• 3: Mark Ashdown , Peter Robinson, Escritoire: A Personal Projected Display, IEEE MultiMedia, v.12 n.1, p.34-42, January 2005 [doi>10.1109/MMUL.2005.18]
• 4: Maribeth J. Back , Jonathan Cohen, Page detection using embedded tags, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.159-160, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354441]
• 5: Aaron F. Bobick , Andrew D. Wilson, A State-Based Approach to the Representation and Recognition of Gesture, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.12, p.1325-1337, December 1997 [doi>10.1109/34.643892]
• 6: Jay David Bolter, Writing space: the computer, hypertext, and the history of writing, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1991
• 7: Vannevar Bush, As we may think, interactions, v.3 n.2, p.35-46, March 1996 [doi>10.1145/227181.227186]
• 8: Davis, J. and Shah, M. Visual gesture recognition. Vision, Image and Signal Processing 141, 2, (1994), 101--106.
• 9: Doug Engelbart 1968 Demo. http://sloan.stanford.edu/MouseSite/1968Demo.html
• 10: Wilfred J. Hansen , Christina Haas, Reading and writing with computers: a framework for explaining differences in performance, Communications of the ACM, v.31 n.9, p.1080-1089, Sept. 1988 [doi>10.1145/48529.48532]
• 11: Christian von Hardenberg , François Bérard, Bare-hand human-computer interaction, Proceedings of the 2001 workshop on Perceptive user interfaces, November 15-16, 2001, Orlando, Florida [doi>10.1145/971478.971513]
• 12: Pengyu Hong , Thomas S. Huang , Matthew Turk, Gesture Modeling and Recognition Using Finite State Machines, Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, p.410, March 26-30, 2000
• 13: Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258715]
• 14: Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K. and Tachibana, K. Virtual object manipulation on a table-top AR environment. Proc. ISAR'00 (2000), 111--119.
• 15: R. Kjeldsen , J. Kender, Finding skin in color images, Proceedings of the 2nd International Conference on Automatic Face and Gesture Recognition (FG '96), p.312, October 14-16, 1996
• 16: H. Koike , M. Kobayashi, EnhancedDesk: Integrating Paper Documents and Digital Documents, Proceedings of the Third Asian Pacific Computer and Human Interaction, p.57, July 15-17, 1998
• 17: Integrating paper and digital information on EnhancedDesk: a method for realtime finger tracking on an augmented desk system, ACM Transactions on Computer-Human Interaction (TOCHI), v.8 n.4, p.307-322, December 2001 [doi>10.1145/504704.504706]
• 18: Jérôme Martin , Jean-Baptiste Durand, Automatic Handwriting Gestures Recognition Using Hidden Markov Models, Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, p.403, March 26-30, 2000
• 19: Kim W. May, Conductive ink based page detection for linking digital and physical pages, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634230]
• 20: T. H. Nelson, Complex information processing: a file structure for the complex, the changing and the indeterminate, Proceedings of the 1965 20th national conference, p.84-100, August 24-26, 1965, Cleveland, Ohio, United States [doi>10.1145/800197.806036]
• 21: Nelson, T. H. Literary Machines. Swarthmore, PA, USA, 1981.
• 22: Kenton O'Hara , Abigail Sellen, A comparison of reading paper and on-line documents, Proceedings of the SIGCHI conference on Human factors in computing systems, p.335-342, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258787]
• 23: Kenji Oka , Yoichi Sato , Hideki Koike, Real-Time Tracking of Multiple Fingertips and Gesture Recognition for Augmented Desk Interface Systems, Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, p.429, May 20-21, 2002
• 24: Vladimir I. Pavlovic , Rajeev Sharma , Thomas S. Huang, Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.677-695, July 1997 [doi>10.1109/34.598226]
• 25: Quek, F. K. H., Mysliwiec, T. and Zhao, M. Finger Mouse: A Freehand Pointing Interface. Proc. Int'l Workshop on Automatic Face and Gesture Recognition 1995, (1995), 372--377.
• 26: Bharat Rao , Louis Minakakis, Evolution of mobile location-based services, Communications of the ACM, v.46 n.12, December 2003 [doi>10.1145/953460.953490]
• 27: Patrick Sinclair , Kirk Martinez , David E. Millard , Mark J. Weal, Links in the palm of your hand: tangible hypermedia using augmented reality, Proceedings of the thirteenth ACM conference on Hypertext and hypermedia, June 11-15, 2002, College Park, Maryland, USA [doi>10.1145/513338.513371]
• 28: Starner, T. and Pentland, A. Visual Recognition of American Sign Language Using Hidden Markov Models. M. I. T. Media Laboratory, Perceptual Computing Section, Cambridge MA, USA, 1994.
• 29: Sukthankar, R., Stockton, R., and Mullin, Smarter Presentations: Exploiting Homography in Camera-Projector Systems. Proc. Int'l Conf. Computer Vision '01, (2001), 82--87.
• 30: John Underkoffler , Hiroshi Ishii, Illuminating light: an optical design tool with a luminous-tangible interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.542-549, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274717]
• 31: John Underkoffler , Hiroshi Ishii, Urp: a luminous-tangible workbench for urban planning and design, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.386-393, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303114]
• 32: Pierre Wellner, Interacting with paper on the DigitalDesk, Communications of the ACM, v.36 n.7, p.87-96, July 1993 [doi>10.1145/159544.159630]
• 33: Yamato, J., Ohya, J., and Ishii, K. Recognizing human action in time-sequential images using hidden Markov model. Proc. of the IEEE Conference on Computer Vision and Pattern Recognition 1992, (1992), 379--385.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
      H.5.3 Group and Organization Interfaces

  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.2 Document Preparation

Keywords:
CSCW, annotation, augmented books, augmented reality, computer vision, electronic books, fingertip detection, gestural input, human-computer interaction, hypermedia, paper-based user interface, tangible user interface, wiki