Optimal audio-visual representations for illiterate users of computers

ABSTRACT

We present research leading toward an understanding of the optimal audio-visual representation for illustrating concepts for illiterate and semi-literate users of computers. In our user study, which to our knowledge is the first of its kind, we presented to 200 illiterate subjects each of 13 different health symptoms in one representation randomly selected among the following ten: text, static drawings, staticphotographs, hand-drawn animations, and video, each with and without voice annotation. The goal was to see how comprehensible these representation types were for an illiterate audience. We used a methodology for generating each of the representations tested in a way that fairly stacks one representational type against the others.

Our main results are that (1) voice annotation generally helps in speed of comprehension, but bimodal audio-visual information can be confusing for the target population; (2) richer information is not necessarily better understood overall; (3) the relative value of dynamic imagery versus static imagery depends on various factors. Analysis of these statistically significant results and additional detailed results are also provided.

REFERENCES

• 1: Bery, S. National Council of Applied Economic Research, 2006. http://www.rediff.com/money/2004/mar/09shining.htm
• 2: Blattner, M. M., Sumikawa, D. A. and Greenberg, R. M. Earcons and icons: Their structure and common design principles. Human-Computer Interaction, 1989. 4, 11--44.
• 3: Jim Bolek , Jamie Cowgill, Development of a symbol system for use in the health care industry, Proceedings of the 2005 conference on Designing for User eXperience, November 03-05, 2005, San Francisco, California
• 4: Michael D. Byrne , Richard Catrambone , John T. Stasko, Evaluating animations as student aids in learning computer algorithms, Computers & Education, v.33 n.4, p.253-278, Dec. 1999 [doi>10.1016/S0360-1315(99)00023-8]
• 5: Chand, A. Designing for the Indian rural population: Interaction design challenges. Proc. Development by Design Conference, (2002).
• 6: ChanLin, L. -J. Animation to teach students of different knowledge levels. Journal of Instructional Psychology, 25, 166--175. (1998).
• 7: Alan Cooper , Robert Reimann , Robert M. Reimann , Hugh Dubberly, About Face 2.0: The Essentials of Interaction Design, John Wiley & Sons, Inc., New York, NY, 2003
• 8: DelGaldo, M. and Nielsen, J. International users interface, John Wiley & Sons, Inc., New York, NY, 1996
• 9: Danny Dowhal, A seven-dimensional approach to graphics, Proceedings of the 14th annual international conference on Systems documentation: Marshaling new technological forces: building a corporate, academic, and user-oriented triangle, p.149-160, October 19-22, 1996, Research Triangle Park, North Carolina, United States [doi>10.1145/238215.238266]
• 10: Dreyfuss, H. Symbol Sourcebook: An Authoritative Guide to International Graphic Symbols. New York: Van Nostrand Reinhold, 1984.
• 11: Jeana Frost , Brian K. Smith, Visualizing health: imagery in diabetes education, Proceedings of the 2003 conference on Designing for user experiences, June 06-07, 2003, San Francisco, California [doi>10.1145/997078.997094]
• 12: Gaver, W. W. Auditory icons: Using sound in computer interfaces. Human-Computer Interaction, 2, 167--177. (1989).
• 13: Gaver, W. W. The sonicfinder: An interface that uses auditory icons. Human-Computer Interaction, 4:67--94, (1989).
• 14: Goetze, M., and Strothotte, T. An approach to help functionally illiterate people with graphical reading aids. In Smart Graphics Symposium UK 2001.
• 15: Sally Grisedale , Mike Graves , Alexander Grünsteidl, Designing a graphical user interface for healthcare workers in rural India, Proceedings of the SIGCHI conference on Human factors in computing systems, p.471-478, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258869]
• 16: William K. Horton, The ICON Book: Visual Symbols for Computer Systems and Documentation, John Wiley & Sons, Inc., New York, NY, 1994
• 17: William Horton, Designing icons and visual symbols, Conference companion on Human factors in computing systems: common ground, p.371-372, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/257089.257378]
• 18: Huenerfauth, M. Developing design recommendations for computer interfaces accessible to illiterate users. Master's thesis, University College Dublin, (2002).
• 19: Khai N. Truong , Gillian R. Hayes , Gregory D. Abowd, Storyboarding: an empirical determination of best practices and effective guidelines, Proceedings of the 6th conference on Designing Interactive systems, June 26-28, 2006, University Park, PA, USA [doi>10.1145/1142405.1142410]
• 20: David Kieras, Diagrammatic displays for engineered systems: effects on human performance in interacting with malfunctioning systems, International Journal of Man-Machine Studies, v.36 n.6, p.861-895, June 1992 [doi>10.1016/0020-7373(92)90077-X]
• 21: Kress, G. and Van Leeuwen, T. Reading Images: The Grammar of Visual Design. London: Routledge. 1996
• 22: Andrew Large , Jamshid Beheshti , Alain Breuleux , Andre Renaud, Effect of animation in enhancing descriptive and procedural texts in a multimedia learning environment, Journal of the American Society for Information Science, v.47 n.6, p.437-448, June 1996 [doi>10.1002/(SICI)1097-4571(199606)47:6<437::AID-ASI4>3.0.CO;2-W]
• 23: Lourie S. World literacy: where we stand today - One Billion Illiterates - editorial, UNESCO Courier. (July 1990)
• 24: Aaron Marcus, Icon and symbol design issues for graphical user interfaces, International users interface, John Wiley & Sons, Inc., New York, NY, 1996
• 25: Aaron Marcus, Icons, symbols, and signs: visible languages to facilitate communication, interactions, v.10 n.3, May + June 2003 [doi>10.1145/769759.769774]
• 26: Aaron Marcus , Nick Smilonich , Lynne Thompson, The cross-GUI handbook for multiplatform user interface design, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1994
• 27: Mitra. S. Self organizing systems for mass computer literacy: Findings from the hole in the wall experiments. International Journal of Development Issues, Vol. 4, No. 1 (2005) 71-81.
• 28: Medhi, I. Pitti B. and Toyama K. Text-Free UI for Employment Search. Proc. Asian Applied Computing Conference. Nepal, December 2005.
• 29: Medhi, I., Sagar A., and Toyama K. Text-Free User Interfaces for Illiterate and Semi-Literate Users. International Conference on Information and Communication Technologies and Development (Berkeley, USA), May 2006.
• 30: Modley, R., Myers, W. R., and Comer, D. G. Handbook of Pictorial Symbols (Dover Pictorial Archive Series) (Paperback- Jun 1, 1976).
• 31: Julie B. Morrison , Barbara Tversky, The (in)effectiveness of animation in instruction, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634290]
• 32: Nathan, M. J., Kintsch, W. & Young, E. A theory of algebra-word-problem comprehension and its implications for the design of learning environments. Cognition and Instruction, 9, 329--389. (1992).
• 33: Susan Palmiter , Jay Elkerton , P. Baggett, Animated demonstrations vs. written instructions for learning procedural tasks: a preliminary investigation, International Journal of Man-Machine Studies, v.34 n.5, p.687-701, May 1991 [doi>10.1016/0020-7373(91)90019-4]
• 34: Palmiter, S. & Elkerton, J. Animated demonstrations for learning procedural computer-based tasks. Human-Computer Interaction, 8, 193--216. (1993).
• 35: Kaushik Ghosh , Tapan S. Parikh , Apala Lahiri Chavan, Design considerations for a financial management system for rural, semi-literate users, CHI '03 extended abstracts on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/765891.766014]
• 36: Tapan Parikh , Kaushik Ghosh , Apala Chavan, Design studies for a financial management system for micro-credit groups in rural india, Proceedings of the 2003 conference on Universal usability, November 10-11, 2003, Vancouver, British Columbia, Canada
• 37: Parikh, T. HISAAB: An Experiment in Numerical Interfaces, Media Lab Asia Panel Discussion, Baramati Initiative on ICT and Development, (2002).
• 38: Park, O. -C. & Gittelman, S. S. Selective use of animation and feedback in computerbased instruction. Educational Technology, Research, and Development, 40, 27--38. (1992).
• 39: John F. Pane , Albert T. Corbett , Bonnie E. John, Assessing dynamics in computer-based instruction, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.197-204, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238482]
• 40: Stephen J. Payne , Louise Chesworth , Elaine Hill, Animated demonstrations for exploratory learners, Interacting with Computers, v.4 n.1, p.3-22, April 1992 [doi>10.1016/0953-5438(92)90010-D]
• 41: Ratnam, B. V., Reddy P. K., and Reddy, G. S. eSagu: An IT based personalized agricultural extension system prototype - analysis of 51 Farmers' case studies International Journal of Education and Development using Information and Communication Technology (IJEDICT), 2005, Vol. 2, Issue 1, pp. 79--94.
• 42: Lloyd P. Rieber , Michael J. Hannafin, Effects of textual and animated orienting activities and practice on learning from computer-based instruction, Computers in the Schools, v.5 n.1-2, p.77-89, 1988 [doi>10.1300/J025v05n01_07]
• 43: Rieber, L. P. The effects of computer animated elaboration strategies and practice on factual and application learning in an elementary science lesson. Journal of Educational Computing Research, 5, 431--444. (1989).
• 44: Lloyd P. Rieber , Mary J. Boyce, The effects of computer animation on adult learning and retrieval tasks, Journal of Computer Based Instruction, v.17 n.2, p.46-52, Spring 1990
• 45: Schnotz, W. & Grzondziel, H. Individual and co-operative learning with interactive animated pictures. European Journal of Psychology of Education, 14, 245--265. (1999).
• 46: Barbara Tversky , Julie Bauer Morrison , Mireille Betrancourt, Animation: can it facilitate?, International Journal of Human-Computer Studies, v.57 n.4, p.247-262, October 2002 [doi>10.1006/ijhc.2002.1017]
• 47: The DIL team, Innovative ICT Tools for Information Provision in Agricultural Extension International Conference on Information and Communication Technologies and Development (Berkeley, USA), May 2006.
• 48: Thompson, S. V. & Riding, R. J. The effect of animated diagrams on the understanding of a mathematical demonstration in 11- to 14-year-old pupils. British Journal of Educational Psychology, 60, 93--98. (1990).
• 49: Werner, D, Thuman, C and Maxwell, J. Where There Is No Doctor: A Village Health Care Handbook (Paper back May 1992)

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: User-centered design

Keywords:
audio-visual icons, illiterate users, text-free user interfaces