Comparing interfaces based on what users watch and do

ABSTRACT

With the development of novel interfaces controlled through multiple modalities, new approaches are needed to analyze the process of interaction with such interfaces and evaluate them at a fine grain of detail. In order to evaluate the usability and usefulness of such interfaces, one needs tools to collect and analyze richly detailed data pertaining to both the process and outcomes of user interaction. Eye tracking is a technology that can provide detailed data on the allocation and shifts of users' visual attention across interface entities. Eye movement data, when combined with data from other input modalities (such as spoken commands, haptic actions with the keyboard and the mouse, etc.), results in just such a rich data on set. However, integrating, analyzing and visualizing multimodal data on user interactions remains a difficult task. In this paper we report on a first step toward developing a suite of tools to facilitate this task. We designed and implemented an Eye Tracking Analysis System that generates combined gaze and action visualizations from eye movement data and interaction logs. This new visualization allows an experimenter to see the visual attention shifts of users interleaved with their actions on each screen of a multi-screen interface. A pilot experiment on comparing two interfaces — a traditional interface and a speech-controlled one — to an educational multimedia application was carried out to test the utility of our tool.

REFERENCES

• 1: Antti Aaltonen , Aulikki Hyrskykari , Kari-Jouko Räihä, 101 spots, or how do users read menus?, Proceedings of the SIGCHI conference on Human factors in computing systems, p.132-139, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274664]
• 2: D. C. R. Benei, D. Ottens, and R. Horst. Use of an Eyetracking System in the Usability Laboratory. In Proceedings of the 35th Annual Meeting, pages 461-465. Human Factors and Ergonomics Society, 1991.
• 3: Ron Van Buskirk , Mary LaLomia, A comparison of speech and mouse/keyboard GUI navigation, Conference companion on Human factors in computing systems, p.96, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223355.223447]
• 4: Michael D. Byrne , John R. Anderson , Scott Douglass , Michael Matessa, Eye tracking the visual search of click-down menus, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.402-409, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303118]
• 5: Peter Faraday , Alistair Sutcliffe, Designing effective multimedia presentations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.272-278, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258753]
• 6: J. H. Goldberg and X. P. Kotval. Eye Movement-Based Evaluation of the Computer Interface. In S. K. Kumar, editor, Advances in Occupational Ergonomics and Safety, pages 529-532. IOS Press, Amsterdam, 1998.
• 7: S.R. Hansen, S. R., N. H. Narayanan, and D. Schrimpsher. Helping Learners Visualize and Comprehend Algorithms. Interactive Multimedia Electronic Journal of Computer- Enhanced Learning, 2(1):available at http://imej.wfu.edu, May 2000.
• 8: M.A. Just and P. A. Carpenter. Eye Fixations and Cognitive Processes, Cognitive Psychology, 8:441-480, 1976.
• 9: Clare-Marie Karat , Christine Halverson , Daniel Horn , John Karat, Patterns of entry and correction in large vocabulary continuous speech recognition systems, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.568-575, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303160]
• 10: N. H. Narayanan and Schrimpsher, D. Extending Eye Tracking to Analyze Interactions with Multimedia Information Presentations. In People and Computers )(IV (HCI 2000 Conference Proceedings), Annual Conference Series, to appear. British Computer Society, Springer-Verlag, 2000.
• 11: Dario D. Salvucci, Inferring intent in eye-based interfaces: tracing eye movements with process models, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.254-261, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303055]
• 12: Linda E. Sibert , Robert J. K. Jacob, Evaluation of eye gaze interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.281-288, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332445]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Voice I/O; Graphical user interfaces (GUI); Evaluation/methodology

General Terms:
Design, Experimentation, Human Factors, Management, Measurement, Performance, Theory

Keywords:
eye tracking, interaction log analysis, speech-controlled interface, visualization