A comparison of synchronous remote and local usability studies for an expert interface

ABSTRACT

Synchronous remote usability studies can be a convenient and cost-effective alternative to conventional local usability studies. Although they are common in the field, there has been little research comparing synchronous remote usability studies with local studies. In our comparison of remote and local studies of an expert interface, the primary differences were in the participant's and facilitator's qualitative experience. The remote and local studies agreed closely (with no significant differences) in terms of the number of usability issues found, their type, and their severity. While our comparison focuses on an expert interface and more work is needed to understand remote studies in general, our experience suggests that evaluators of expert interfaces will have comparable success identifying usability issues with either remote or local studies.

REFERENCES

• 1: Boren, T., and Ramey, J. Thinking Aloud: Reconciling Theory and Practice. IEEE Trans. on Professional Communication, pg. 261--278. Sept. 2000.
• 2: Maria R. Ebling , Bonnie E. John, On the contributions of different empirical data in usability testing, Proceedings of the conference on Designing interactive systems: processes, practices, methods, and techniques, p.289-296, August 17-19, 2000, New York City, New York, United States [doi>10.1145/347642.347766]
• 3: Eclipse Project. www.eclipse.org
• 4: Glance Networks. www.glance.net
• 5: Gough, D. and Phillips, H. Remote Online Usability Testing: Why, How, and When to use it. www.boxesandarrows.com/archives/remote_online_usability_testing_why_how_and_when_to_use_it.php
• 6: H. Rex Hartson , José C. Castillo , John Kelso , Wayne C. Neale, Remote evaluation: the network as an extension of the usability laboratory, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.228-235, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238511]
• 7: David M. Hilbert , David F. Redmiles, Separating the wheat from the chaff in Internet-mediated user feedback expectation-driven event monitoring, ACM SIGGROUP Bulletin, v.20 n.1, p.35-40, Apr. 1999 [doi>10.1145/327556.327611]
• 8: Jacques, R. and Savastano, H. Remote vs. Local Usability Evaluation of Web Sites. IHM HCI 2001.
• 9: Nielsen, Severity Ratings for Usability Problems. www.useit.com/papers/heuristic/severityrating.html
• 10: Julie Ratner, Human Factors and Web Development, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2002
• 11: Urbansim Project. www.urbansim.org

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Theory and methods; User-centered design

Keywords:
remote evaluation, usability research, usability testing and evaluation, user studies