A collaborative multimodal handwriting training environment for visually impaired students

ABSTRACT

The spatial motor skills used for handwriting are particularly difficult for visually impaired people to develop. These skills are required in order to sign an aesthetically pleasing and repeatable signature, which is often required for documents such as legal agreements and job applications. Our multimodal system with haptic guidance, sonification and tactile feedback is designed to assist when teaching visually impaired students to form letters, and eventually, a signature. As tactile technologies become commonplace, appearing even in mobile phones, our system may also provide useful insight into the use of nonvisual feedback for a variety of applications.

REFERENCES

• 1: Amirabdollahian, F., Loureiro, R. and Harwin, W. Minimum Jerk Trajectory Control for Rehabilitation and Haptic Applications. In Proceedings of the IEEE International Conference on Robotics & Automation, (Washington DC, 2002), 3380--3385.
• 2: Arter, C., McCall, S., Bowyer, T. Handwriting and children with visual impairments British Journal of Special Education 23, 1 (1996), 25--28.
• 3: Astrom, K. J. and Hagglund, T. PID Controllers. International Society for Measurement and Control, Research Triangle Park, N.C., 1995.
• 4: Andrew Crossan , Stephen Brewster, Multimodal Trajectory Playback for Teaching Shape Information and Trajectories to Visually Impaired Computer Users, ACM Transactions on Accessible Computing (TACCESS), v.1 n.2, p.1-34, October 2008 [doi>10.1145/1408760.1408766]
• 5: Crossan, A., Williamson, J. and Brewster, S. A General Purpose Control-Based Playback for Force Feedback Systems. In Proceedings of Eurohaptics 2006 (Paris, France, 2006).
• 6: Dang, T., Annaswamy, T. M. and Srinivasan, M. A. Development and Evaluation of an Epidural Injection Simulator with Force Feedback for Medical Training. In Proceedings of Medicine Meets Virtual Reality, (USA, 2001), IOS Press, 97--102.
• 7: David Feygin , Madeleine Keehner , Frank Tendick, Haptic Guidance: Experimental Evaluation of a Haptic Training Method for a Perceptual Motor Skill, Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.40, March 24-25, 2002
• 8: Green, T. R. G. and Blackwell, A. F. Cognitive dimensions of information artefacts: a tutorial. http://www.cl.cam.ac.uk/~afb21/CognitiveDimensions/CDtutorial.pdf (1998)
• 9: Green, T. R. G. & Petre, M. Usability analysis of visual programming environments: a 'cognitive dimensions' framework. Journal of Visual Languages and Computing 2, 7 (1996), 131--174.
• 10: Henmi, K. and Yoshikawa, T. Virtual lesson and its application to virtual calligraphy system. In Proceedings of IEEE International Conference on Robotics and Automation, (1998), 1275--1280.
• 11: Eve Hoggan , Stephen Brewster, Designing audio and tactile crossmodal icons for mobile devices, Proceedings of the 9th international conference on Multimodal interfaces, November 12-15, 2007, Nagoya, Aichi, Japan [doi>10.1145/1322192.1322222]
• 12: Horstmann, M., Hagen, C., King, A., Dijkstra, S., Crombie, D., Evans, G., Ioannidis, G. T., Blenkhorn, P., Herzog, O. and Schlieder, C. TeDUB: Automatic interpretation and presentation of technical diagrams for blind people. In Proceedings of Conference and Workshop on Assistive Technologies for Vision and Hearing Impairment - CVHI'2004, EURO-ASSIST-VHI-2: Accessibility, Mobility and Social Integration, (Granada, Spain, 2004).
• 13: Martin Kurze, TDraw: a computer-based tactile drawing tool for blind people, Proceedings of the second annual ACM conference on Assistive technologies, p.131-138, April 11-12, 1996, Vancouver, British Columbia, Canada [doi>10.1145/228347.228368]
• 14: Landua, S. and Wells, L. Merging Tactile Sensory Input and Audio Data by Means of the Talking Tactile Tablet. In Proceedings of Eurohaptics 2003, (Dublin, Ireland, 2003), 414--418.
• 15: Mullins, J., Mawson, C., Nahavandi, S. Haptic handwriting aid for training and rehabilitation. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, (2005), 2690--2694
• 16: Beryl Plimmer , Andrew Crossan , Stephen A. Brewster , Rachel Blagojevic, Multimodal collaborative handwriting training for visually-impaired people, Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, April 05-10, 2008, Florence, Italy [doi>10.1145/1357054.1357119]
• 17: Rassmus-Gröhn, K., Magnusson, C. and Eftring, H. User evaluations of a virtual haptic-audio line drawing prototype. In Proceedings of Haptic and Audio Interaction Design (HAID2006), (University of Glasgow, UK, 2006), LNCS vol 4129, 81--91.
• 18: Sassoon, R. Handwriting: The Way to Teach It. Second Edition. Paul Chapman, London, 2003.
• 19: SensAble Technologies www.sensable.com
• 20: Taylor, T. Handwriting: Multisensory Approaches to Assessing and Improving Handwriting Skills, David Fulton, London, 2001.
• 21: Cl Teo , E. Burdet , Hp Lim, A Robotic Teacher of Chinese Handwriting, Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, p.335, March 24-25, 2002
• 22: Steven Wall , Stephen Brewster, Feeling what you hear: tactile feedback for navigation of audio graphs, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124941]
• 23: Yokokohji, Y., Hollis, R. L., Kanade, T., Henmi, K. and Yoshikawa, T. Toward Machine Mediated Training of Motor Skills -Skill Transfer from Human to Human via Virtual Environment. In Proceedings of RO-MAN, (Japan, 1996), 32--37.
• 24: Yu, W. and Brewster, S. A. Evaluation of Multimodal Graphs for Blind People. Journal of Universal Access in the Information Society 2, 2 (2003), 105--124.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.m Miscellaneous

General Terms:
Design, Human Factors

Keywords:
handwriting, haptic guidance, signature, sonification, tactile, visually-impaired