Closing the loop: modelling action, perception and information

ABSTRACT

Visual interfaces to computer systems are interactive. The cycle of visual interaction involves both visual perception and action. This paper examines formal models of interactive systems and cognitive models of users. Neither completely captures the special nature of visual interaction. In order to investigate this, the paper examines two forms of non-visual interaction: mathematics for the blind and interaction by smell (nasal interaction). Finally three forms of more pragmatic design-oriented method are considered: information rich task analysis (<u>what</u> information is required), status-event analysis (<u>when</u> it is perceived) and models of information (<u>how</u> to visually interact with it).

REFERENCES

• 1: Gregory D. Abowd , Alan J. Dix, Integrating status and event phenomena in formal specifications of interactive systems, Proceedings of the 2nd ACM SIGSOFT symposium on Foundations of software engineering, p.44-52, December 06-09, 1994, New Orleans, Louisiana, United States
• 2: Christopher Ahlberg , Ben Shneiderman, Visual information seeking: tight coupling of dynamic query filters with starfield displays, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.313-317, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191775]
• 3: Christopher Ahlberg , Staffan Truvé, Exploring terra incognita in the design space of query devices, Proceedings of the IFIP TC2/WG2.7 Working Conference on Engineering for Human-Computer Interaction, p.49-68, August 01, 1995
• 4: P. Barnard (1985). Interacting Cognitive Subsystems: A psycholinguistic approach to short-term memory. Progress in the Psychology of Language, Ed. A. Ellis. Hove, Lawrence Erlbaum Associates. pp. 197--258.
• 5: P. J. Barnard and J. D. Teasdale (1991). Interacting Cognitive Subsystems: A systematic approach to cognitive-affective interaction and change. Cognition and Emotion, 5:1--39.
• 6: K. Brodie (1993). A classification scheme for scientific visualisation. Animation and Scientific Visualisation, Eds. R. A. Earnshaw and D. Watson. London, Academic Press. pp. 125--140.
• 7: Steve Bryson , Creon Levit, The Virtual Wind Tunnel, IEEE Computer Graphics and Applications, v.12 n.4, p.25-34, July 1992 [doi>10.1109/38.144824]
• 8: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 9: Alan Dix , Gregory Abowd, Delays and temporal incoherence due to the mediated status-status mappings, ACM SIGCHI Bulletin, v.28 n.2, p.47-49, April 1996 [doi>10.1145/226650.226667]
• 10: A. Dix and S. A. Brewster (1994). Causing Trouble with Buttons. Ancillary Proceedings of HCI'94, Glasgow.
• 11: Alan Dix , Janet Finlay , Gregory Abowd , Russell Beale, Human-computer interaction, Prentice-Hall, Inc., Upper Saddle River, NJ, 1997
• 12: A. J. Dix (1991). Formal Methods for Interactive Systems. Academic Press.
• 13: Alan Dix, Pace and interaction, Proceedings of the conference on People and computers VII, p.193-207, January 1993, York, United Kingdom
• 14: Alan Dix , Devina Ramduny , Julie Wilkinson, Long-term interaction: learning the 4 Rs, Conference companion on Human factors in computing systems: common ground, p.169-170, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/257089.257237]
• 15: A. J. Dix and C. Runciman (1985). Abstract models of interactive systems. People and Computers: Designing the Interface, Cambridge University Press. pp. 13--22.
• 16: A. D. N. Edwards (1989). Soundtrack: an auditary interface for blind users. Human-Computer Interaction, 4(1):45--66.
• 17: Alistair D. N. Edwards, Extra-ordinary human-computer interaction: interfaces for users with disabilities, Cambridge University Press, New York, NY, 1995
• 18: E. Hutchins and T. Klausen (1991). Distributed cognition in an airline cockpit. Cognition in communication at work. Eds. Y. Engestrm and D. Middleton. CUP:, Cambridge.
• 19: Bonnie E. John, Extensions of GOMS analyses to expert performance requiring perception of dynamic visual and auditory information, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.107-116, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97262]
• 20: J. Larkin (1989). Display based problem solving. Complex Information Processing: The Impact of Herbert A. Simon, Eds. D. Klahr and K. Kotovsky. New Jersey, Lawrence Erlbaum Associates.
• 21: C. Linchan and L. McCarthy (1995). A Task Analysis of Students doing Mathematics: Contributing to the design of the input and manipulation language. MATHS Project Internal Report, IR-15.3, University College Cork.
• 22: Ray Monk, Fundamentals of Human-Computer Interaction, Academic Press, Inc., Orlando, FL, 1984
• 23: D. A. Norman (1988). The Psychology of Everyday Things. Basic Books.
• 24: Stephen J. Payne, Acquisition of display-based skill, Conference companion on Human factors in computing systems, p.299-300, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/259963.260507]
• 25: S. J. Payne and T. R. G. Green (1986). Task action grammars: a model of mental representation of task language. Human-Computer Interaction, 2(2): 93--133.
• 26: P. Reisner (1981). Formal grammar and human factors design of an interactive graphics system. IEE Transactions on Software Engineering, 7(2):229--240.
• 27: George G. Robertson , Jock D. Mackinlay , Stuart K. Card, Cone Trees: animated 3D visualizations of hierarchical information, Proceedings of the SIGCHI conference on Human factors in computing systems: Reaching through technology, p.189-194, April 27-May 02, 1991, New Orleans, Louisiana, United States [doi>10.1145/108844.108883]
• 28: A. Shepherd (1995). Task analysis as a framework for examining HCI tasks. Perspectives on HCI: Diverse Approaches, Eds. A. Monk and N. Gilbert. London, Academic Press. pp. 145--174.
• 29: R. Stevens and A. Edwards (1994). Analysis of audio approaches. MATHS Project Internal Report, IR-6, University of York.
• 30: L. A. Suchman (1987). Plans and Situated Actions. Cambridge, Cambridge University Press.
• 31: Edward Tufte, Envisioning information, Graphics Press, Cheshire, CT, 1990

INDEX TERMS

Keywords:
aural interfaces, cognitive models, formal methods, status-event analysis