Modelling "user understanding" in simple communication tasks

ABSTRACT

We present an architectural model for adaptive interfaces based on eye-gaze patterns and facial expression analysis. In our approach, each basic visual sign can adapt its appearance and level of detail during the communication process. Atomic Communication Units (ACUs) -- analogous to graphical output primitives - encapsulate the intended denotation, the encoding of the message and a method for the judgment of the communication goal. We have analyzed feedback cycles in human-human communication tasks, and propose applications scenarios for ACUs.

REFERENCES

• 1: H. von Foerster, Cybernetics of Cybernetics (2nd edition), Future Systems, Minneapolis, (1996).
• 2: P. Patrik, Intelligent User Interfaces -- Introduction and survey -- Research Report DKS03-01 / ICE 01, Delft University of Technology, (2003).
• 3: Edward Ross, Intelligent User Interfaces: Survey and Research Directions, University of Bristol, Bristol, UK, 2000
• 4: Pat Langley, User modeling in adaptive interfaces, Proceedings of the seventh international conference on User modeling, p.357-370, June 1999, Banff, Canada
• 5: Peter Brusilovsky , David W. Cooper, Domain, task, and user models for an adaptive hypermedia performance support system, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502724]
• 6: R. M. Young , T. R. G. Green , T. Simon, Programmable user models for predictive evaluation of interface designs, Proceedings of the SIGCHI conference on Human factors in computing systems: Wings for the mind, p.15-19, March 1989
• 7: Brad A. Myers, Creating user interfaces by demonstration, Academic Press Professional, Inc., San Diego, CA, 1988
• 8: W. Kintsch, Comprehension: A Paradigm for Cognition, MA: Cambridge University Press, (1998).
• 9: Young Woo Sohn , Stephanie M. Doane, Evaluating Comprehension-Based User Models: Predicting Individual User Planning and Action, User Modeling and User-Adapted Interaction, v.12 n.2-3, p.171-205, 2002 [doi>10.1023/A:1015072424236]
• 10: L. A. Suchman, Plans and Situated Actions, Cambride University Press (1987).
• 11: Gerhard Fischer, User Modeling in Human–Computer Interaction, User Modeling and User-Adapted Interaction, v.11 n.1-2, p.65-86, 2001 [doi>10.1023/A:1011145532042]
• 12: D. D. Salvucci, An integrated model of eye movements and visual encoding. Cognitive Systems Research, 1(4) 2001, pp. 201--220.
• 13: Dario D. Salvucci , John R. Anderson, Intelligent gaze-added interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems, p.273-280, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332444]
• 14: Pernilla Qvarfordt , Shumin Zhai, Conversing with the user based on eye-gaze patterns, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055004]
• 15: B. Kiss, B. Benedek, G. Szijarto, B. Takacs: Closed Loop Dialog Model of Face-to-Face Communication with a Photo-Real Virtual Human, Visual Communications and Image Processing 2004 (EI25) SPIE Electronic Imaging, (2004).
• 16: M. Argyle, M. Cook, Gaze and Mutual Gaze, Cambridge University Press, London, (1977).
• 17: P. Ekman, W. Friesen, Unmasking the Face. A guide to recognizing emotions from facial expressions, Palo Alto: Consulting Psychologists Press, (1984).
• 18: S. Morishima, Face Analysis and Synthesis, IEEE Signal Processing Magazine, Vol.18, No.3 (2001), pp.26--34.
• 19: M. Pantic, Facial gesture recognition from static dual-view face images, International Conference on Measuring Behaviour, (2002), pp. 195--197.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

Keywords:
adaptive interfaces, eye-gaze patterns, mental models, visual language