Free-sketch recognition: putting the chi in sketching

ABSTRACT

Sketch recognition techniques have generally fallen into two camps. Gesture-based techniques, such as those used by the Palm Pilot's Graffiti, can provide high-accuracy, but require the user to learn a particular drawing style in order for shapes to be recognized. Free-sketch recognition allows users to draw shapes as they would naturally, but most current techniques have low accuracies or require significant domain-level tweaking to make them usable. Our goal is to recognize free-hand sketches with high accuracy by developing generalized techniques that work for a variety of domains, including design and education. This is a work-in-progress, but we have made significant advancements toward our over-arching goal.

REFERENCES

• 1: Christine Alvarado , Randall Davis, SketchREAD: a multi-domain sketch recognition engine, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029637]
• 2: Tracy Anne Hammond , Randall Davis, Ladder: a perceptually-based language to simplify sketch recognition user interface development, Massachusetts Institute of Technology, Cambridge, MA, 2007
• 3: Christopher F. Herot, Graphical input through machine recognition of sketches, Proceedings of the 3rd annual conference on Computer graphics and interactive techniques, p.97-102, July 14-16, 1976, Philadelphia, Pennsylvania
• 4: Kara, L. B, Stahovich, T. F. An image-based trainable symbol recognizer for sketch-based interfaces. AAAI Fall Symposium Series 2004: Making Pen-Based Interaction Natural (2004).
• 5: Kim, D. H., Kim M. J. A curvature estimation for pen input segmentation in sketch-based modeling. Computer-Aided Design (2006), 38(3), 238--248.
• 6: Allan Christian Long, Jr. , James A. Landay, Quill: a gesture design tool for pen-based user interfaces, University of California, Berkeley, 2001
• 7: Michael Oltmans , Randall Davis, Envisioning sketch recognition: a local feature based approach to recognizing informal sketches, Massachusetts Institute of Technology, Cambridge, MA, 2007
• 8: Paulson, B., Hammond, T. PaleoSketch: Accurate Primitive Sketch Recognition and Beautification. In Proceedings of Intelligent User Interfaces (IUI'08) (2008).
• 9: Plimmer B. Ink Features. Eurographics Sketchbased Interfaces and Modeling (2007).
• 10: Dean Rubine, Specifying gestures by example, ACM SIGGRAPH Computer Graphics, v.25 n.4, p.329-337, July 1991
• 11: Tevfik Metin Sezgin , Thomas Stahovich , Randall Davis, Sketch based interfaces: early processing for sketch understanding, Proceedings of the 2001 workshop on Perceptive user interfaces, November 15-16, 2001, Orlando, Florida [doi>10.1145/971478.971487]
• 12: Tevfik Metin Sezgin , Randall Davis, HMM-based efficient sketch recognition, Proceedings of the 10th international conference on Intelligent user interfaces, January 10-13, 2005, San Diego, California, USA [doi>10.1145/1040830.1040899]
• 13: Bo Yu , Shijie Cai, A domain-independent system for sketch recognition, Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia, February 11-14, 2003, Melbourne, Australia [doi>10.1145/604471.604499]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.7 DOCUMENT AND TEXT PROCESSING
      I.7.5 Document Capture
          Subjects: Graphics recognition and interpretation

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.6 Segmentation
          Subjects: Edge and feature detection

General Terms:
Human Factors

Keywords:
free-sketch, ladder, multimodal interaction, paleosketch, pen input, shortstraw, sketch recognition, tablet pc