MathPad2: a system for the creation and exploration of mathematical sketches

ABSTRACT

We present mathematical sketching, a novel, pen-based, modeless gestural interaction paradigm for mathematics problem solving. Mathematical sketching derives from the familiar pencil-and-paper process of drawing supporting diagrams to facilitate the formulation of mathematical expressions; however, with a mathematical sketch, users can also leverage their physical intuition by watching their hand-drawn diagrams animate in response to continuous or discrete parameter changes in their written formulas. Diagram animation is driven by implicit associations that are inferred, either automatically or with gestural guidance, from mathematical expressions, diagram labels, and drawing elements. The modeless nature of mathematical sketching enables users to switch freely between modifying diagrams or expressions and viewing animations. Mathematical sketching can also support computational tools for graphing, manipulating and solving equations; initial feedback from a small user group of our mathematical sketching prototype application, MathPad², suggests that it has the potential to be a powerful tool for mathematical problem solving and visualization.

REFERENCES

• 1: Alvarado, C. J. 2000. A Natural Sketching Environment: Bringing the Computer into Early Stages of Mechanical Design. Master's thesis, Massachusetts Institute of Technology.
• 2: Blostein, D., and Grbavec, A. 2001. Recognition of mathematical notation. In Handbook of Character Recognition and Document Image Analysis, World Scientific, H. Bunke and P. Wang, Eds., 557--582.
• 3: Alan Hamilton Borning, Thinglab--a constraint-oriented simulation laboratory., 1979
• 4: Chan, K.-F., and Yeung, D.-Y. 2000. An efficient syntactic approach to structural analysis of on-line handwritten mathematical expressions. Pattern Recognition 33, 3, 375--384.
• 5: Chan, K.-F., and Yeung, D.-Y. 2000. Mathematical expression recognition: A survey. International Journal on Document Analysis and Recognition 3, 1, 3--15.
• 6: PenCalc: A Novel Application of On-Line Mathematical Expression Recognition Technology, Proceedings of the Sixth International Conference on Document Analysis and Recognition, p.774, September 10-13, 2001
• 7: Connell, S. D., and Jain, A. K. 2000. Template-based on-line character recognition. Pattern Recognition 34, 1, 1--14.
• 8: Scott D. Connell , Anil K. Jain, Writer Adaptation for Online Handwriting Recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.3, p.329-346, March 2002 [doi>10.1109/34.990135]
• 9: Christian Heide Damm , Klaus Marius Hansen , Michael Thomsen, Tool support for cooperative object-oriented design: gesture based modelling on an electronic whiteboard, Proceedings of the SIGCHI conference on Human factors in computing systems, p.518-525, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332488]
• 10: Andrew Forsberg , Mark Dieterich , Robert Zeleznik, The music notepad, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.203-210, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288608]
• 11: Mark D. Gross , Ellen Yi-Luen Do, Ambiguous intentions: a paper-like interface for creative design, Proceedings of the 9th annual ACM symposium on User interface software and technology, p.183-192, November 06-08, 1996, Seattle, Washington, United States [doi>10.1145/237091.237119]
• 12: Hecht, E. 2000. Physics: Calculus. Brooks/Cole.
• 13: Takeo Igarashi , Satoshi Matsuoka , Hidehiko Tanaka, Teddy: a sketching interface for 3D freeform design, Proceedings of the 26th annual conference on Computer graphics and interactive techniques, p.409-416, July 1999 [doi>10.1145/311535.311602]
• 14: Kara, L. B., Gennari, L., and Strahovich, T. F. 2004. A sketch-based interface for the design and analysis of simple vibratory mechanical systems. In Proceedings of ASME International Design Engineering Technical Conferences.
• 15: Jean R. Laleuf , Anne Morgan Spalter, A component repository for learning objects: a progress report, Proceedings of the 1st ACM/IEEE-CS joint conference on Digital libraries, p.33-40, January 2001, Roanoke, Virginia, United States [doi>10.1145/379437.379444]
• 16: James A. Landay , Brad A. Myers, Interactive sketching for the early stages of user interface design, Proceedings of the SIGCHI conference on Human factors in computing systems, p.43-50, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223910]
• 17: Hsi-Jian Lee , Jiumn-Shine Wan , g, Design of a mathematical expression recognition system, Proceedings of the Third International Conference on Document Analysis and Recognition (Volume 2), p.1084, August 14-15, 1995
• 18: Li, X., and Yeung, D.-Y. 1997. On-line handwritten alphanumeric character recognition using dominant points in strokes. Pattern Recognition 30, 1, 31--44.
• 19: Matsakis, N. E. 1999. Recognition of Handwritten Mathematical Expressions. Master's thesis, Massachusetts Institute of Technology.
• 20: Erik G. Miller , Paul A. Viola, Ambiguity and constraint in mathematical expression recognition, Proceedings of the fifteenth national/tenth conference on Artificial intelligence/Innovative applications of artificial intelligence, p.784-791, July 1998, Madison, Wisconsin, United States
• 21: Thomas P. Moran , Patrick Chiu , William van Melle, Pen-based interaction techniques for organizing material on an electronic whiteboard, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.45-54, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263508]
• 22: Elizabeth D. Mynatt , Takeo Igarashi , W. Keith Edwards , Anthony LaMarca, Flatland: new dimensions in office whiteboards, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.346-353, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303108]
• 23: Steve Smithies , Kevin Novins , James Arvo, A handwriting-based equation editor, Proceedings of the 1999 conference on Graphics interface '99, p.84-91, September 1999, Kingston, Ontario, Canada
• 24: Anne Morgan Spalter , Rosemary Michelle Simpson, Integrating interactive computer-based learning experiences into established curricula: a case study, Proceedings of the 5th annual SIGCSE/SIGCUE ITiCSEconference on Innovation and technology in computer science education, p.116-119, July 11-13, 2000, Helsinki, Finland
• 25: Varberg, D., and Purcell, E. J. 1992. Calculus with Analytic Geometry. Prentice Hall.
• 26: Young, H. D. 1992. University Physics. Addison-Wesley Publishing Company.
• 27: Richard Zanibbi , Kevin Novins , James Arvo , Katherine Zanibbi, Aiding manipulation of handwritten mathematical expressions through style-preserving morphs, No description on Graphics interface 2001, p.127-134, June 07-09, 2001, Ottawa, Ontario, Canada
• 28: Richard Zanibbi , Dorothea Blostein , James R. Cordy, Recognizing Mathematical Expressions Using Tree Transformation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.11, p.1455-1467, November 2002 [doi>10.1109/TPAMI.2002.1046157]
• 29: Robert C. Zeleznik , Kenneth P. Herndon , John F. Hughes, SKETCH: an interface for sketching 3D scenes, Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.163-170, August 1996 [doi>10.1145/237170.237238]

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.4 MATHEMATICAL SOFTWARE
      Subjects: User interfaces

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

Additional Classification:
  G. Mathematics of Computing
  G.4 MATHEMATICAL SOFTWARE
      Subjects: User interfaces

  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

General Terms:
Documentation, Human Factors

Keywords:
gestures, mathematical sketching, pen-based interfaces