A haptic multimedia handwriting learning system

ABSTRACT

In this paper, we describe a multimedia system for learning handwriting and pronunciation of alphabet letters or characters in different languages. This system provides haptic, audio and visual information according to the desired letter or character chosen by a user. Letters or characters from the Arabic, English, French, Japanese, and Spanish languages have been considered, although the system utilizes an XML-based schema to easily introduce new characters from another language.

Three different modes of learning can be chosen in terms of haptic information: full guidance, partial guidance and a no guidance mode (no haptic feedback). The full guidance guides the user to follow a pre-recorded letter trajectory; whereas in partial guidance, a user can freely follow a letter-drawing path, but if the user deviates significantly, the system automatically brings him/her back to the optimal displayed path. The no guidance mode allows users to perform letter handwriting with only visual information. This system guides users to write a character, in a similar way as a teacher holds a student.s hand. Moreover, the character trajectory is displayed as the user is performing it. The results of this system evaluation show its potential as a virtual tool for learning handwriting.

REFERENCES

• 1: Mic Bowman , Saumya K. Debray , Larry L. Peterson, Reasoning about naming systems, ACM Transactions on Programming Languages and Systems (TOPLAS), v.15 n.5, p.795-825, Nov. 1993 [doi>10.1145/161468.161471]
• 2: R. Palluel-Germain , F. Bara , A. Hillairet de Boisferon , B. Hennion , P. Gouagout , E. Gentaz, A Visuo-Haptic Device - Telemaque - Increases Kindergarten Children's Handwriting Acquisition, Proceedings of the Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.72-77, March 22-24, 2007 [doi>10.1109/WHC.2007.13]
• 3: Henmi, K., and Yoshikawa, T. Virtual lesson and its application to virtual calligraphy system. Proceedings of the IEEE International Conference on Robotics and Automation, 1998
• 4: Cl Teo , E. Burdet , Hp Lim, A Robotic Teacher of Chinese Handwriting, Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.335, March 24-25, 2002
• 5: Rosenblum, S., Weiss, P. L., and Parush, Shula. Product and Process Evaluation of Handwriting Difficulties. Journal Educational Psychology Review 15, 1 (March, 2003), 41--81.
• 6: Bara, F., Gentaz, E. & Coléé, P. The visuo-haptic and haptic exploration of letters increases the kindergarten-children's reading acquisition. Cognitive Development, 19, 2004, 433--449.
• 7: Bayart, B., Pocheville, A., and Kheddar, A. An adaptive haptic guidance software module for I-Touch: example through a handwriting teaching simulation and a 3D maze. In proceedings of the Haptic Audio Visual Environments and their Applications, Ottawa, Canada, 2005.
• 8: M. Eid, A. Alamri, and A. El Saddik, "MPEG-7 Description of Haptic Applications Using HAML", Proceedings of the HAVE2006, Ottawa, Canada, 2006.
• 9: SensAble Technologies Inc.: http:// www.sensable.com, accessed July 8, 2007.
• 10: MPB Technologies: http://www.mpb-technologies.ca, accessed July 8, 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Haptic I/O; Evaluation/methodology

General Terms:
Design, Measurement, Performance

Keywords:
education, haptic playback, haptics, multimedia, virtual teaching