SHARK2: a large vocabulary shorthand writing system for pen-based computers

ABSTRACT

Zhai and Kristensson (2003) presented a method of speed-writing for pen-based computing which utilizes gesturing on a stylus keyboard for familiar words and tapping for others. In SHARK²:, we eliminated the necessity to alternate between the two modes of writing, allowing any word in a large vocabulary (e.g. 10,000-20,000 words) to be entered as a shorthand gesture. This new paradigm supports a gradual and seamless transition from visually guided tracing to recall-based gesturing. Based on the use characteristics and human performance observations, we designed and implemented the architecture, algorithms and interfaces of a high-capacity multi-channel pen-gesture recognition system. The system's key components and performance are also reported.

REFERENCES

• 1: Johnny Accot , Shumin Zhai, Beyond Fitts' law: models for trajectory-based HCI tasks, Proceedings of the SIGCHI conference on Human factors in computing systems, p.295-302, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258760]
• 2: Johnny Accot , Shumin Zhai, More than dotting the i's --- foundations for crossing-based interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503390]
• 3: Eveline J. Bellegarda , Jerome R. Bellegarda , David Nahamoo , Krishna S. Nathan, A Fast Statistical Mixture Algorithm for On-Line Handwriting Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.16 n.12, p.1227-1233, December 1994 [doi>10.1109/34.387484]
• 4: Djamel Bouchaffra , Venu Govindaraju , Sargur Srihari, A Methodology for Mapping Scores to Probabilities, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.21 n.9, p.923-927, September 1999 [doi>10.1109/34.790432]
• 5: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 6: Richard O. Duda , Peter E. Hart , David G. Stork, Pattern Classification (2nd Edition), Wiley-Interscience, 2000
• 7: Clive Frankish , Richard Hull , Pam Morgan, Recognition accuracy and user acceptance of pen interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.503-510, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223972]
• 8: Getschow, C.O., Rosen, M.J. and Goodenough-Trepagnier, C., A systematic approach to design a minimum distance alphabetical keyboard. Proc. RESNA (Rehabilitation Engineering Society of North America) 9th Annual Conference, 1986, 396--398.
• 9: David Goldberg , Cate Richardson, Touch-typing with a stylus, Proceedings of the SIGCHI conference on Human factors in computing systems, p.80-87, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169093]
• 10: Daniel Jurafsky , James H. Martin, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, Prentice Hall PTR, Upper Saddle River, NJ, 2000
• 11: Kurtenbach, G., Sellen, A. and Buxton, W. An empirical evaluation of some articulatory and cognitive aspects of "marking menus". Human Computer Interaction, 8 (1). 1993,1--23.
• 12: C. G. Leedham , A. C. Downton, Automatic recognition and transcription of Pitman's handwritten shorthand-an approach to shortforms, Pattern Recognition, v.20 n.3, p.341-348, 1987 [doi>10.1016/0031-3203(87)90008-2]
• 13: Allan Christian Long, Jr. , James A. Landay , Lawrence A. Rowe, Implications for a gesture design tool, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.40-47, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.302985]
• 14: MacKenzie, I.S. and Soukoreff, R.W. Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction, 17 (1). 2002.
• 15: I. Scott MacKenzie , Shawn X. Zhang, The design and evaluation of a high-performance soft keyboard, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.25-31, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.302983]
• 16: Jennifer Mankoff , Gregory D. Abowd, Cirrin: a word-level unistroke keyboard for pen input, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.213-214, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288611]
• 17: Newman, M.W., Jason, J.L., Hong, J.I. and Landay, J.A. DENIM: An informal web site design tool inspired by observations of practice. Human-Computer Interaction, 18 (3). 2003,259--324.
• 18: Ken Perlin, Quikwriting: continuous stylus-based text entry, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.215-216, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288613]
• 19: Pitrelli, J.F. and Roy, A. Creating Word-Level Language Models for Handwriting Recognition. International Journal on Document Analysis and Recognition, 5 (2&3). 2003,126--137.
• 20: Réjean Plamondon , Sargur N. Srihari, On-Line and Off-Line Handwriting Recognition: A Comprehensive Survey, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.1, p.63-84, January 2000 [doi>10.1109/34.824821]
• 21: Dean Rubine, Specifying gestures by example, Proceedings of the 18th annual conference on Computer graphics and interactive techniques, p.329-337, July 1991
• 22: Schmidt, R.A. Motor control and learning - A Behavioral Emphasis. Human Kinetics Publishers, Inc., 1988.
• 23: Tappert, C.C. Cursive Script Recognition by Elastic Matching. IBM Journal of Research & Development, 26 (6). 1982,756--771.
• 24: C. C. Tappert , C. Y. Suen , T. Wakahara, The State of the Art in Online Handwriting Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.12 n.8, p.787-808, August 1990 [doi>10.1109/34.57669]
• 25: Jacob O. Wobbrock , Brad A. Myers , John A. Kembel, EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.61-70, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964703]
• 26: Zhai, S., Hunter, M. and Smith, B.A. Performance optimization of virtual keyboards. Human-Computer Interaction, 17 (2,3). 2002,89--129.
• 27: Shumin Zhai , Per-Ola Kristensson, Shorthand writing on stylus keyboard, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642630]
• 28: Zhai, S., Kristensson, P.-O. and Smith, B.A. In Search of Effective Text Input Interfaces for Off the Desktop Computing. Interacting with Computers, 16 (3). 2004,to appear.
• 29: Shumin Zhai , Alison Sue , Johnny Accot, Movement model, hits distribution and learning in virtual keyboarding, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503381]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Feature evaluation and selection; Classifier design and evaluation

Keywords:
gesture recognition, shorthand, shorthand recognition, stenography, text input