Gestural and audio metaphors as a means of control for mobile devices

ABSTRACT

This paper discusses the use of gesture and non-speech audio as ways to improve the user interface of a mobile music player. Their key advantages mean that users could use a player without having to look at its controls when on the move. Two very different evaluations of the player took place: one based on a standard usability experiment (comparing the new player to a standard design) and the other a video analysis of the player in use. Both of these showed significant usability improvements for the gesture/audio-based interface over a standard visual/pen-based display. The similarities and differences in the results produced by the two studies are discussed

REFERENCES

• 1: Blattner, M., Sumikawa, D. and Greenberg, R. Earcons and icons: Their structure and common design principles. Human Computer Interaction 4, 1 (1989), 11--44.
• 2: Stephen A Brewster , Peter G Cryer, Maximising screen-space on mobile computing devices, CHI '99 extended abstracts on Human factors in computing systems, May 15-20, 1999, Pittsburgh, Pennsylvania [doi>10.1145/632716.632855]
• 3: Brewster, S.A. and Walker, V.A. Non-Visual Interfaces for Wearable Computers. In Proceedings of IEE workshop on Wearable Computing (IEE, London) IEE, 00/145, 2000.
• 4: Stephen A. Brewster , Peter C. Wright , Alistair D. N. Edwards, An evaluation of earcons for use in auditory human-computer interfaces, Proceedings of the INTERCHI '93 conference on Human factors in computing systems, p.222-227, May 1993, Amsterdam, The Netherlands
• 5: Clark-Carter, D.D., Heyes, A.D. and Howarth, C. The efficiency and walking speed of visually impaired people. Ergonomics 29, 6 (1986), 779--789.
• 6: L. Garcia , F. Quek, Qualitative research in information systems: time to be subjective?, Proceedings of the IFIP TC8 WG 8.2 international conference on Information systems and qualitative research, p.444-465, January 1997, Philadelphia, Pennsylvania, United States
• 7: Don Gentner , Jakob Nielsen, The Anti-Mac interface, Communications of the ACM, v.39 n.8, p.70-82, Aug. 1996 [doi>10.1145/232014.232032]
• 8: Beverly L. Harrison , Kenneth P. Fishkin , Anuj Gujar , Carlos Mochon , Roy Want, Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.17-24, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274647]
• 9: Hart, S.G. and Wickens, C. Workload assessment and prediction. In MANPRINT, an approach to systems integration, Booher, H.R. (Ed.), Van Nostrand Reinhold, New York, 1990, 257--296.
• 10: Johnson, P. Usability and Mobility; Interactions on the move. In First Workshop on HCI with Mobile Devices (Glasgow, UK) Glasgow University, 1998.
• 11: Kvale, S. Issues of validity in qualitative research. Chartwell Bratt, London, 1989.
• 12: Lakoff, G. and Johnson, M. Metaphors we live by. The University of Chicago Press, Chicago, 1980.
• 13: Lee, S.H. Usability testing for developing effective interactive multimedia software: concepts, dimensions, and procedures. Educational Technology & Society 2, 2 (1999).
• 14: Allan Christian Long, Jr. , James A. Landay , Lawrence A. Rowe, Implications for a gesture design tool, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.40-47, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.302985]
• 15: Nardi, B.A. and Zarmer, C.L. Beyond models and metaphors: Visual formalism in user interface design. Journal of Visual Languages and Computing 4 (1993), 5--33.
• 16: Steffen Pauws , Don Bouwhuis , Berry Eggen, Programming and enjoying music with your eyes closed, Proceedings of the SIGCHI conference on Human factors in computing systems, p.376-383, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332460]
• 17: Petrie, H., Furner, S. and Strothotte, T. Design Lifecycles and Wearable Computers for Users with Disabilities. In First workshop on HCI mobile devices (Glasgow, UK) Glasgow University, 1998.
• 18: A. Pirhonen, Analysis of the Concept of Redundancy Concerning the Design of Multimodal Combinations of Output-Elements, Proceedings of the Third Asian Pacific Computer and Human Interaction, p.273, July 15-17, 1998
• 19: Pirhonen, A. Creating virtual objects. In Proceedings of IRIS 22 Technical Report TR-21, Vol. 3. University of Jyväskylä, Department of Computer Science and Information Systems, 1999, pp. 57--69.
• 20: Nitin Sawhney , Chris Schmandt, Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.353-383, Sept. 2000 [doi>10.1145/355324.355327]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Auditory (non-speech) feedback

Additional Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.3 Other Architecture Styles
          Subjects: Cellular architecture (e.g., mobile)

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing
          Subjects: Speech recognition and synthesis

General Terms:
Design, Human Factors, Performance, Theory

Keywords:
evaluation, gestures, metaphor, mobile computing, non-speech audio