Visual similarity of pen gestures

ABSTRACT

Pen-based user interfaces are becoming ever more popular. Gestures (i.e., marks made with a pen to invoke a command) are a valuable aspect of pen-based UIs, but they also have drawbacks. The challenge in designing good gestures is to make them easy for people to learn and remember. With the goal of better gesture design, we performed a pair of experiments to determine why users find gestures similar. From these experiments, we have derived a computational model for predicting perceived gesture similarity that correlates 0.56 with observation. We will incorporate the results of these experiments into a gesture design tool, which will aid the pen-based UI designer in creating gesture sets that are easier to learn and more memorable.

REFERENCES

• 1: Ashby, E G., Maddox, W. T., and Lee, W. W. On the dangers of averaging across subjects when using multidimensional scaling or the similarity-choice model. Psychological Science, May 1994. 5, 3, 144-151.
• 2: Attneave, F. Dimensions of similarity. American Journal of Psychology, 1950. 63, 516-556.
• 3: Beck, J., Prazdny, K., and Rosenfeld, A. A theory of textural segmentation, in Human and Machine Vision (Beck, J., Hope, B., and Rosenfeld, A., eds.), vol. 8 of Notes and Reports in Computer Science and Applied Mathematics. Academic Press, New York, NY, 1983 1-38. Proceedings of the Conference on Human and Machine Vision, Aug. 1981.
• 4: Robert O. Briggs , Alan R. Dennis , Brenda S. Beck , Jay F. Nunamaker, Jr., Whither the pen-based interface?, Journal of Management Information Systems, v.9 n.3, p.71-90, Winter 1992–93
• 5: William Buxton, There's more to interaction than meets the eye: some issues in manual input, Human-computer interaction, Prentice Hall Press, Upper Saddle River, NJ, 1990
• 6: Stéphane Chatty , Patrick Lecoanet, Pen computing for air traffic control, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.87-94, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238436]
• 7: Andrew Forsberg , Mark Dieterich , Robert Zeleznik, The music notepad, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.203-210, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288608]
• 8: Clive Frankish , Richard Hull , Pam Morgan, Recognition accuracy and user acceptance of pen interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.503-510, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223972]
• 9: International Data Corporation. Smart handheld device market will enjoy robust growth in 1999 and 2000, May 1999. Available at http://www.idc.com/Data/Personal/content/ PS052699PR.htm.
• 10: James A. Landay , Brad A. Myers, Interactive sketching for the early stages of user interface design, Proceedings of the SIGCHI conference on Human factors in computing systems, p.43-50, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223910]
• 11: Lazarte, A.A. and Schonemann, P.H. Saliency metric for subadditive dissimilarity judgments on rectangles. Perception and Psychophysics, Feb. 1991.49, 2, 142-158.
• 12: Lee, Y. L. PDA users can express themselves with Graffiti. InfoWorld, Oct 3 1994. 16, 40, 30.
• 13: James S. Lipscomb, A trainable gesture recognizer, Pattern Recognition, v.24 n.9, p.895-907, 1991 [doi>10.1016/0031-3203(91)90009-T]
• 14: Allan C Long , James A. Landay , Lawrence A. Rowe, PDA and Gesture Uses in Practice: Insights for Designers of Pen-Based, University of California at Berkeley, Berkeley, CA, 1998
• 15: Allan Christian Long, Jr. , James A. Landay , Lawrence A. Rowe, Implications for a gesture design tool, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.40-47, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.302985]
• 16: Lopresti, D. and Tomkins, A. Computing in the ink domain. In International Conference on Human-Computer Interaction (Anzai, Y., Ogawa, K., and Mori, H., eds.), vol. 1 of Advances in Human Factors~Ergonomics. Information Processing Society of Japan and others, Elsevier Science, Jul. 1995 543- 548.
• 17: André Meyer, Pen computing: a technology overview and a vision, ACM SIGCHI Bulletin, v.27 n.3, p.46-90, July 1995 [doi>10.1145/221296.221308]
• 18: Thomas P. Moran , Patrick Chiu , William van Melle , Gordon Kurtenbach, Implicit structure for pen-based systems within a freeform interaction paradigm, Proceedings of the SIGCHI conference on Human factors in computing systems, p.487-494, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223970]
• 19: P. Morrel-Samuels, Clarifying the distinction between lexical and gestural commands, International Journal of Man-Machine Studies, v.32 n.5, p.581-590, May 1990 [doi>10.1016/S0020-7373(05)80034-3]
• 20: Pier, K. and Landay, J.A. Issues for location-independent interfaces. Tech. Rep. ISTL92-4, Xerox Palo Alto Research Center, Dec 1992.
• 21: Byron Reeves , Clifford Nass, The media equation: how people treat computers, television, and new media like real people and places, Cambridge University Press, New York, NY, 1996
• 22: Dean Rubine, Specifying gestures by example, Proceedings of the 18th annual conference on Computer graphics and interactive techniques, p.329-337, July 1991
• 23: Mark A. Tapia , Gordon Kurtenbach, Some design refinements and principles on the appearance and behavior of marking menus, Proceedings of the 8th annual ACM symposium on User interface and software technology, p.189-195, November 15-17, 1995, Pittsburgh, Pennsylvania, United States [doi>10.1145/215585.215973]
• 24: Thomas, H. Spatial models and multidimensional scaling of random shapes. American Journal of Psychology, 1968.81, 4, 551-558.
• 25: Catherine G. Wolf , James R. Rhyne , Hamed A. Ellozy, The paper-like interface, Proceedings of the third international conference on human-computer interaction on Designing and using human-computer interfaces and knowledge based systems (2nd ed.), p.494-501, September 1989, Boston, Massachusetts, United States
• 26: Young, E W. Multidimensional Scaling: History, Theory, and Applications. Lawrence Erlbaum Associates, Hillsdale, NJ, 1987.
• 27: Zhao, R., Kaufmann, H.-J., Kern, T., and Miiller, W. Penbased interfaces in engineering environments. In International Conference on Human-Computer Interaction (Anzai, Y., Ogawa, K., and Mori, H., eds.), vol. 20B of Advances in Human Factors/Ergonomics. Information Processing Society of Japan and others, Elsevier Science, Jul. 1995 531-536.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.3 Clustering
          Subjects: Similarity measures

General Terms:
Design, Experimentation, Human Factors, Management, Measurement, Performance, Theory

Keywords:
multi-dimensional scaling, pen gestures, pen-based user interfaces, perception, similarity