Evaluating the appropriateness of speech input in marine applications: a field evaluation

ABSTRACT

This paper discusses the first of three studies which collectively represent a convergence of two ongoing research agendas: (1) the empirically-based comparison of the effects of evaluation environment on mobile usability evaluation results; and (2) the effect of environment -- in this case lobster fishing boats -- on achievable speech-recognition accuracy. We describe, in detail, our study and outline our results to date based on preliminary analysis. Broadly speaking, the potential for effective use of speech for data collection and vessel control looks very promising -- surprisingly so! We outline our ongoing analysis and further work.

REFERENCES

• 1: Alexander, T., Schlick, C., Sievert, A. and Leyk, D., 2008. Assessing Human Mobile Computing Performance by Fitt's Law. in Lumsden, J. ed. Handbook of Research on User Interface Design and Evaluation for Mobile Technology, Information Science Reference, Hershey, 830--846.
• 2: Crease, M., Lumsden, J. and Longworth, B., 2007. A Technique for Incorporating Dynamic Paths in Lab-Based Mobile Evaluations. In Proceedings of the British HCI'2007 Conference, (Lancaster, UK, 2007), BCS, 99--108.
• 3: Crossan, A., Murray-Smith, R., Brewster, S. and Musizza, B., 2008. Instrumented Usability Analysis for Mobile Devices. in Lumsden, J. ed. Handbook of Research on User Interface Design and Evaluation for Mobile Technology, Information Science Reference, Hershey, 927--944.
• 4: Hart, S. G. and Wickens, C., 1990. Workload assessment and prediction. in Booher, H. R. ed. MANPRINT: an approach to systems integration, Van Nostrand Reinhold, New York, 257--296.
• 5: Kaikkonen, A., Kallio, T., Kekalainen, A., Kankainen, A. and Cankar, M., 2005. Usability Testing of Mobile Applications: A Comparison Between Laboratory and Field Testing. Journal of Usability Studies, 1(1). 4--16.
• 6: Kaikkonen, A., Kekalainen, A., Cankar, M., Kallio, T. and Kankainen, A., 2008. Will Laboratory Tests be Valid in Mobile Contexts? in Lumsden, J. ed. Handbook of Research on User Interface Design and Evaluation for Mobile Technology, Information Science Reference, Hershey, 897--909.
• 7: Kjeldskov, J., Skov, M. B., Als, B. S. and Høegh, R. T., 2004. Is It Worth the Hassle? Exploring the Added Value of Evaluating the Usability of Context-Aware Mobile Systems in the Field. In Proceedings of the 6th International Symposium on Mobile Human-Computer Interaction (MobileHCI'04), (Glasgow, Scotland, 2004), 61--73.
• 8: Kjeldskov, J. and Stage, J., 2004. New Techniques for Usability Evaluation of Mobile Systems. International Journal of Human Computer Studies (IJHCS), 60(5--6). 599--620.
• 9: Lumsden, J., Kondratova, I. and Durling, S., 2007. Investigating Microphone Efficacy for Facilitation of Mobile Speech-Based Data Entry. In Proceedings of the British HCI'2007 Conference, (Lancaster, UK, 2007), BCS, 89--98.
• 10: Lumsden, J., Kondratova, I. and Langton, N., 2006. Bringing A Construction Site Into The Lab: A Context-Relevant Lab-Based Evaluation Of A Multimodal Mobile Application. In Proceedings of the 1st International Workshop on Multimodal and Pervasive Services (MAPS'2006), (Lyon, France, 2006), IEEE.
• 11: Christian Monrad Nielsen , Michael Overgaard , Michael Bach Pedersen , Jan Stage , Sigge Stenild, It's worth the hassle!: the added value of evaluating the usability of mobile systems in the field, Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles, p.272-280, October 14-18, 2006, Oslo, Norway [doi>10.1145/1182475.1182504]
• 12: Sharon Oviatt, Taming recognition errors with a multimodal interface, Communications of the ACM, v.43 n.9, p.45-51, Sept. 2000 [doi>10.1145/348941.348979]
• 13: Pick, H., Siegel, G., Fox, P., Garber, S. and Kearney, J., 1989. Inhibiting the Lombard Effect. Journal of the Acoustical Society of America, 85(2). 894--900.
• 14: Price, K., Lin, M., Feng, J., Goldman, R., Sears, A. and Jacko, J., 2004. Data Entry on the Move: An Examination of Nomadic Speech-Based Text Entry. In Proceedings of the 8th ERCIM Workshop "User Interfaces For All" (UI4All'04), (Vienna, Austria, 2004), Springer-Verlag LNCS, 460--471.
• 15: Nitin Sawhney , Chris Schmandt, Nomadic radio: speech and audio interaction for contextual messaging in nomadic environments, ACM Transactions on Computer-Human Interaction (TOCHI), v.7 n.3, p.353-383, Sept. 2000 [doi>10.1145/355324.355327]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Voice I/O

General Terms:
Experimentation, Human Factors, Measurement, Performance

Keywords:
evaluation, field study, speech input