Earpod: eyes-free menu selection using touch input and reactive audio feedback

ABSTRACT

We present the design and evaluation of earPod: an eyes-free menu technique using touch input and reactive auditory feedback. Studies comparing earPod with an iPod-like visual menu technique on reasonably-sized static menus indicate that they are comparable in accuracy. In terms of efficiency (speed), earPod is initially slower, but outperforms the visual technique within 30 minutes of practice. Our results indicate that earPod is potentially a reasonable eyes-free menu technique for general use, and is a particularly exciting technique for use in mobile device interfaces.

REFERENCES

• 1: KidsClick! web search for kids by librarians: http://sunsite.berkeley.edu/KidsClick!/.
• 2: Wikipedia: http://www.wikipedia.org/.
• 3: Barry Arons, SpeechSkimmer: a system for interactively skimming recorded speech, ACM Transactions on Computer-Human Interaction (TOCHI), v.4 n.1, p.3-38, March 1997 [doi>10.1145/244754.244758]
• 4: Bernstein, L. (1997). Detection and discrimination of interaural disparities: Modern earphone-based studies, in Binaural and Spatial Hearing in Real and Virtual Environments, R.H. Gilkey et. al. Editors. Lawrence Erlbaum. p. 117--138.
• 5: Berry, G. (1997). The Esterel Primer. Technical report, Ecole des Mines de Paris and INRIA.
• 6: Blattner, M., Sumikawa, D., and Greenberg, R. (1989). Earcons and icons: Their structure and common design principles. Human Computer Interaction, 4(1). p. 11--44.
• 7: Stephen Brewster , Joanna Lumsden , Marek Bell , Malcolm Hall , Stuart Tasker, Multimodal 'eyes-free' interaction techniques for wearable devices, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642694]
• 8: Stephen A. Brewster, Using nonspeech sounds to provide navigation cues, ACM Transactions on Computer-Human Interaction (TOCHI), v.5 n.3, p.224-259, Sept. 1998 [doi>10.1145/292834.292839]
• 9: Stephen A Brewster , Peter G Cryer, Maximising screen-space on mobile computing devices, CHI '99 extended abstracts on Human factors in computing systems, May 15-20, 1999, Pittsburgh, Pennsylvania [doi>10.1145/632716.632855]
• 10: Stephen A. Brewster , Peter C. Wright , Alistair D. N. Edwards, An evaluation of earcons for use in auditory human-computer interfaces, Proceedings of the INTERACT '93 and CHI '93 conference on Human factors in computing systems, p.222-227, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169179]
• 11: J. Callahan , D. Hopkins , M. Weiser , B. Shneiderman, An empirical comparison of pie vs. linear menus, Proceedings of the SIGCHI conference on Human factors in computing systems, p.95-100, May 15-19, 1988, Washington, D.C., United States [doi>10.1145/57167.57182]
• 12: Doel, K., and Pai, D. (2001). A Java audio synthesis system for programmers. International Conference on Auditory Display.
• 13: Gaver, W. (1989). The Sonic Finder: An interface that uses auditory icons. Human Computer Interaction, 4(1). p. 67--94.
• 14: William W. Gaver , Randall B. Smith, AUDITORY ICONS IN LARGE-SCALE COLLABORATIVE ENVIRONMENTS, ACM SIGCHI Bulletin, v.23 n.1, p.96, Jan. 1991 [doi>10.1145/122672.1048010]
• 15: Guo, J., and Guo, A. (2005). Crossmodal interaction between olfactory and visual learning in Drosophila. Science, 309. p. 307--310.
• 16: Beverly L. Harrison , Kenneth P. Fishkin , Anuj Gujar , Carlos Mochon , Roy Want, Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.17-24, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274647]
• 17: Gordon Paul Kurtenbach, The design and evaluation of marking menus, University of Toronto, Toronto, Ont., Canada, 1993
• 18: Chunyuan Liao , François Guimbretière , Corinna E. Loeckenhoff, Pen-top feedback for paper-based interfaces, Proceedings of the 19th annual ACM symposium on User interface software and technology, October 15-18, 2006, Montreux, Switzerland [doi>10.1145/1166253.1166285]
• 19: Joseph Luk , Jerome Pasquero , Shannon Little , Karon MacLean , Vincent Levesque , Vincent Hayward, A role for haptics in mobile interaction: initial design using a handheld tactile display prototype, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124800]
• 20: Marics, M., and Engelbeck, G. (1997). Designing voice menu applications for telephones, in Handbook of Human-Computer Interaction, Helander, M., Landauer, T., and Prabhu, P. Editors. Elsevier. p. 1085--1102.
• 21: Miller, D. (1981). The depth/breadth tradeoff in hierarchical computer menus. Human Factors Society Conference. p. 296--300.
• 22: Minoru Kobayashi , Chris Schmandt, Dynamic Soundscape: mapping time to space for audio browsing, Proceedings of the SIGCHI conference on Human factors in computing systems, p.194-201, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258702]
• 23: Elizabeth D. Mynatt, Transforming graphical interfaces into auditory interfaces, Conference companion on Human factors in computing systems, p.67-68, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223355.223432]
• 24: Kent L. Norman , Ben Shneiderman, The Psychology of Menu Selection: Designing Cognitive Control at the Human/Computer Interface, Greenwood Publishing Group Inc., Westport, CT, 1991
• 25: Antti Oulasvirta , Sakari Tamminen , Virpi Roto , Jaana Kuorelahti, Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055101]
• 26: Paap, K., and Cooke, N. (1997). Designing menus, in Handbook of Human-Computer Interaction, Helander, M., Landauer, T., and Prabhu, P. Editors. Elsevier. p. 533--572.
• 27: Jason Pascoe , Nick Ryan , David Morse, Using while moving: HCI issues in fieldwork environments, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.417-437, Sept. 2000 [doi>10.1145/355324.355329]
• 28: Steffen Pauws , Don Bouwhuis , Berry Eggen, Programming and enjoying music with your eyes closed, Proceedings of the SIGCHI conference on Human factors in computing systems, p.376-383, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332460]
• 29: Antti Pirhonen , Stephen Brewster , Christopher Holguin, Gestural and audio metaphors as a means of control for mobile devices, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503428]
• 30: Abhishek Ranjan , Ravin Balakrishnan , Mark Chignell, Searching in audio: the utility of transcripts, dichotic presentation, and time-compression, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124879]
• 31: Paul Resnick , Robert A. Virzi, Skip and scan: cleaning up telephone interface, Proceedings of the SIGCHI conference on Human factors in computing systems, p.419-426, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142881]
• 32: T. L. Roberts , G. Engelbeck, The effects of device technology on the usability of advanced telephone functions, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.331-337, March 1989
• 33: Rosenberger, J., and Gasco, M. (1983). Comparing location estimators: Trimmed means, medians, and trimean. in Understanding robust and exploratory data analysis, Hoagan, D., Mosteller, F., and Tukey, J., Editors. John Wiley: New York.
• 34: Deb K. Roy , Chris Schmandt, NewsComm: a hand-held interface for interactive access to structured audio, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.173-180, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238471]
• 35: Nitin Sawhney , Chris Schmandt, Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.353-383, Sept. 2000 [doi>10.1145/355324.355327]
• 36: Chris Schmandt, Audio hallway: a virtual acoustic environment for browsing, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.163-170, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288597]
• 37: Chris Schmandt , Kwan Hong Lee , Jang Kim , Mark Ackerman, Impromptu: managing networked audio applications for mobile users, Proceedings of the 2nd international conference on Mobile systems, applications, and services, June 06-09, 2004, Boston, MA, USA [doi>10.1145/990064.990074]
• 38: Ben Shneiderman , Catherine Plaisant, Designing the User Interface: Strategies for Effective Human-Computer Interaction (4th Edition), Pearson Addison Wesley, 2004
• 39: Lisa Stifelman , Barry Arons , Chris Schmandt, The audio notebook: paper and pen interaction with structured speech, Proceedings of the SIGCHI conference on Human factors in computing systems, p.182-189, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365096]
• 40: Lisa J. Stifelman , Barry Arons , Chris Schmandt , Eric A. Hulteen, VoiceNotes: a speech interface for a hand-held voice notetaker, Proceedings of the INTERACT '93 and CHI '93 conference on Human factors in computing systems, p.179-186, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169150]
• 41: Bernhard Suhm , Barbara Freeman , David Getty, Curing the menu blues in touch-tone voice interfaces, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634147]
• 42: Witten, I., Cleary, J., and Greenberg, S. (1984). On frequency-based menu-splitting algorithms. International Journal of Man-Machine Studies, 21(2). p. 135--148.
• 43: Min Yin , Shumin Zhai, The benefits of augmenting telephone voice menu navigation with visual browsing and search, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124821]
• 44: Shengdong Zhao , Maneesh Agrawala , Ken Hinckley, Zone and polygon menus: using relative position to increase the breadth of multi-stroke marking menus, Proceedings of the SIGCHI conference on Human Factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1124772.1124933]
• 45: Shengdong Zhao , Ravin Balakrishnan, Simple vs. compound mark hierarchical marking menus, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029639]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

General Terms:
Human Factors

Keywords:
auditory menu, gestural interaction