Towards the perfect infrastructure for usability testing on mobile devices

ABSTRACT

In this paper, we describe various setups that allow usability professionals to conduct effective user studies on mobile devices. We describe the factors relevant when building a solution for mobile device observation and the various designs we worked with in the Google user experience research environment as we iterated to meet changing study needs. We highlight several systems that can successfully be used in an industry environment, including a novel setup that is fully portable, can be used in a usability lab as well as in the field, accommodates a large variety of different mobile devices, and allows for live observation by product teams around the world.

REFERENCES

• 1: Worldwide cellular connections pass 2.5 billion http://www.ovum.com/go/content/c,377,66726
• 2: International Telecommunication Union. ICT Statistics Database. http://www.itu.int/ITU--D/icteye/Indicators/Indicators.aspx
• 3: mobileways.de Remote S60. http://mobileways.de/M/1/3/0/
• 4: Rudy Schusteritsch , Shailendra Rao , Kerry Rodden, Mobile search with text messages: designing the user experience for google SMS, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1057020]
• 5: Scott Weiss, Handheld Usability, John Wiley & Sons, Inc., New York, NY, 2002
• 6: Noldus Mobile Device Camera http://www.noldus.com/site/doc200402054
• 7: Roto, V., Oulasvirta, A., Haikarainen, T., Kuorelahti, J., Lehmuskallio, H., and Nyyssönen, T. Examining Mobile Phone Use in the Wild With QuasiExperimentation. HIIT, Helsinki, Finland, 2004. http://www.hiit.fi/publications/pub_files/hiit2004--1.pdf
• 8: Antti Oulasvirta, The fragmentation of attention in mobile interaction, and what to do with it, interactions, v.12 n.6, p.16-18, November + December 2005 [doi>10.1145/1096554.1096555]
• 9: AVerMedia Surveillance Solutions. http://www.aver.com/security.html
• 10: Shuttle Computer Inc. http://www.shuttle.com
• 11: PY Software Active WebCam. http://www.pysoft.com

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Prototyping

General Terms:
Design, Human Factors

Keywords:
iterative usability lab design, mobile device usability observation infrastructure, mobile device usability testing, user experience research