Evaluating prosodic cues as a means to disambiguate algebraic expressions: an empirical study

ABSTRACT

The automatic translation of written mathematical expressions to their spoken equivalent is a difficult task. Written mathematics makes use of specialized symbols and a 2-dimensional layout that is hard to translate into clear and unambiguous spoken words. Our approach is to use prosody to help listeners follow along to mathematical expressions spoken aloud with text-to-speech synthesized voices. To achieve this, we developed and empirically tested XSL transformation rules that automatically translate mathematical expressions marked-up with Presentation MathML into corresponding markup using the Speech Synthesis Markup Language (SSML). In this paper, we report on the results from an empirical study we conducted that showed that the simple insertion of pauses inside spoken mathematical expressions dramatically improved subjects' ability to disambiguate between two similar algebraic expressions. Result from our study should benefit designers of screen readers and related audio-based tools that produce spoken renderings of mathematical expressions.

REFERENCES

• 1: D. Archambault, B. St¨oger, D. Fitzpatrick, and K. Miesenberger. Access to scientific content by visually impaired people. UPGRADE, Digital Journal of CEPIS, 8(2):29--42, 2007.
• 2: R. Ausbrooks, S. Buswell, D. Carlisle, S. Dalmas, S. Devitt, A. Diaz, M. Froumentin, R. Hunter, P. Ion, M. Kohlhase, R. Miner, N. Poppelier, B. Smith, N. Soiffer, R. Sutor, and S. Watt. Mathematical Markup Language(MathML) Version 2.0(second edition): W3C Recommendation. www.w3.org/TR/2003/REC-MathML2-20031021/, October 2003.
• 3: C. Bernareggi and D. Archambault. Mathematics on the web: Emerging opportunities for visually impaired people. In W4A '07: Proceedings of the 2007 international cross-disciplinary conference on Web accessibility(W4A), pages 108--111. ACM, 2007.
• 4: C. Burnett, R. Walker, and A. Hunt. Speech Synthesis Markup Language(SSML) Version 1.0. W3C Recommendation. www.w3.org/TR/speech-synthesis/, 2004.
• 5: L. A. Chang. Handbook for Spoken Mathematics (Larry's Speakeasy). Lawrence Livermore National Laboratory, The Regents of the University of California, 1983.
• 6: Daisy Consortium. DAISY 3 modular extension for MathML. www.daisy.org/projects/mathml/ mathml-in-daisy-spec.html, 2007.
• 7: Daisy Consortium. Structure guidelines for DAISY 3. www.daisy.org/z3986/structure/SG-DAISY3/, 2008.
• 8: EasyReader from Dolphin Computer Access, Inc. www.yourdolphin.com/.
• 9: J. Engelen. Modern digital libraries, the case of the audio-book boom. In ICCHP '08: Proceedings of the 11th international conference on Computers Helping People with Special Needs, pages 284--290, Berlin, Heidelberg, 2008. Springer-Verlag.
• 10: H. Ferreira and D. Freitas. Audio rendering of mathematical formulae using MathML and AudioMath. In User Interfaces for All, pages 391--399, 2004.
• 11: D. Fitzpatrick and A. I. Karshmer. Multi-modal mathematics: Conveying math using synthetic speech and speech recognition. In ICCHP '04: Proceedings of the 11th international conference on Computers Helping People with Special Needs, pages 644--647, Berlin, Heidelberg, 2004. Springer-Verlag.
• 12: gh PLAYERTMfrom gh, LLC. www.gh-accessibility.com/.
• 13: J. J. Glutting, E. A. Youngstrom, and M. W. Watkins. ADHD and college students: Exploratory and confirmatory factor structures with student and parent data. Psychological Assessment, 17(1):44--55, 2005.
• 14: L. Hecker, L. Burns, L. Katz, J. Elkind, and K. Elkind. Benefits of assistive reading software for students with attention disorders. Annals of Dyslexia, 52:243--272, 2002.
• 15: C. Henderson. College Freshmen with Disabilities, 2001: A Biennial Statistical Profile. American Council on Education, HEATH Resource Center, 2001.
• 16: A. Karshmer, G. Gupta, and E. Pontelli. Mathematics and accessibility: A survey. Technical report, University of Texas at Dallas, 2007.
• 17: D. Leas, E. Persoon, N. Soiffer, and M. Zacherle. Daisy 3: A standard for accessible multimedia books. IEEE Multimedia, 15(4):28--37, 2008.
• 18: National Council on Disability (U.S.). People with disabilities and postsecondary education: Position paper. National Council on Disability, 2003.
• 19: T. V. Raman. Audio system for technical readings. PhD thesis, Ithaca, NY, USA, 1994.
• 20: T. V. Raman. Cascaded speech style sheets. In Selected papers from the sixth international conference on World Wide Web, pages 1377--1383, Essex, UK, 1997. Elsevier Science Publishers Ltd.
• 21: S. Shaywitz and B. Shaywitz. The neurobiology of reading and dyslexia. Focus on Basics, 5, 2001.
• 22: N. Soiffer. Mathplayer: Web-based math accessibility. In Assets '05: Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, pages 204--205. ACM, 2005.
• 23: N. Soiffer. Mathplayer v2.1: Web-based math accessibility. In Assets '07: Proceedings of the 9th international ACM SIGACCESS conference on Computers and accessibility, pages 257--258. ACM, 2007.
• 24: P. E. Steele and T. R. Wolanin. Higher Education Opportunities for Students with Disabilities: A Primer for Policymakers. Institute for Higher Education Policy, 2004.
• 25: R. Stevens. Principles for the Design of Auditory Interfaces to Present Complex Information to Blind People. PhD thesis, The University of York, 1996.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

General Terms:
Design, Experimentation, Human Factors

Keywords:
DAISY, MathML, SSML, accessibility, synthetic speech