Towards a new method for the evaluation of reality based interaction

ABSTRACT

In this paper we present work toward a new method of evaluation for Reality-Based Interaction Styles that we call Cognitive Description and Evaluation of Interaction (CoDeIn). This framework is similar to GOMS, but also shares similarities with cognitive architectures, such as ACT-R and Soar. We apply this new approach to a Tangible User Interface, built as a Wizard of Oz interface, to illustrate the new method, and compare CoDeIn.s results to a GOMS model. We show that CoDeIn provides some promise of representing the parallel processing and new modes of physical interaction inherent in reality-based interfaces, and CoDeIn does a better job in predicting the completion time of an example task than GOMS.

REFERENCES

• 1: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 2: G. Christou and R. J. K. Jacob, Evaluating and Comparing Interaction Styles, in J. Jorge, N. Jardim Nunes and J. Falcao e Cunha, eds., DSV-IS 2003: 10th Workshop on the Design, Specification and Verification of Interactive Systems, Springer Verlag, Funchal, Portugal, 2003, pp. 406--409.
• 3: W. D. Gray, B. E. John and M. E. Atwood, Project Ernestine: Validating a GOMS Analysis for Predicting and Explaining Real-World Task Performance, Human Computer Interaction, 8 (1993), pp. 237--309.
• 4: David Harel, Statecharts: A visual formalism for complex systems, Science of Computer Programming, v.8 n.3, p.231-274, June 1, 1987 [doi>10.1016/0167-6423(87)90035-9]
• 5: E. Hutchins, J. Hollan and D. Norman, Direct Manipulation Interfaces, in D. A. Norman and S. W. Draper, eds., User Centered System Design: New Perspectives in Human--Computer Interaction, Lawrence Erlbaum Associates, Hillsdale, NJ, 1986, pp. 87--124.
• 6: Robert J. K. Jacob, What is the next generation of human-computer interaction?, CHI '06 extended abstracts on Human factors in computing systems, April 22-27, 2006, Montréal, Québec, Canada [doi>10.1145/1125451.1125768]
• 7: R. J. K. Jacob, Reality-Based Interaction: Understanding the Next Generation of Human-Computer Interfaces, 2004.
• 8: Robert J. K. Jacob , Leonidas Deligiannidis , Stephen Morrison, A software model and specification language for non-WIMP user interfaces, ACM Transactions on Computer-Human Interaction (TOCHI), v.6 n.1, p.1-46, March 1999 [doi>10.1145/310641.310642]
• 9: Bonnie E. John , David E. Kieras, The GOMS family of user interface analysis techniques: comparison and contrast, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.4, p.320-351, Dec. 1996 [doi>10.1145/235833.236054]
• 10: Bonnie E. John , David E. Kieras, Using GOMS for user interface design and evaluation: which technique?, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.4, p.287-319, Dec. 1996 [doi>10.1145/235833.236050]
• 11: B. Shneiderman, Direct Manipulation: A Step Beyond Programming Languages, IEEE Computer, 16 (1983).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors; Human information processing
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Theory and methods

General Terms:
Human Factors, Measurement, Theory

Keywords:
GOMS, RBI, cognitive architecture, evaluation method, interface evaluation, reality based interaction