Gaze-orchestrated dynamic windows

ABSTRACT

Consider a large-format display before the user, bearing a multiplicity of “windows,” like little movies, the majority dynamic and in color. There are upwards of 20 windows, say, more than a person can ordinarily absorb at once. Some of the windows come and go, reflecting their nature as direct TV linkages into real-time, real-world events. Others are non-real-time, some dynamic, others static but capable of jumping into motion. Such an ensemble of information inputs reflects the managerial world of the top-level executive of the not too distant electronic future: a world of brevity, fragmentation, variety, above all one of an overwhelming onslaught of events. The multiplicity and simultaneity of such a display situation ordinarily would make coping with it untenable. The intent of the reported research is to introduce order and control, through the creation of a dynamic, gaze-interactive interface. Making the behavior and reactivity of the “windows” contingent upon measured eyemovements - the point-of-regard of the observer - aims both to help the observer to cope with the onslaught of events on the one hand, yet enable on the other hand continuing close contact with that everchanging ensemble. A simulation of such a world is described and demonstrated in the composite medium of computer, videodisc, and video special effects. Eye-tracking technology, integrated with speech and manual inputs, controls the display's visual dynamics, and orchestrates its sound accompaniments. All elements are combined to form a testbed for the conception generally, and to explore the associated human factors and stagecraft.

REFERENCES

• 1: FOULDS, R. A. Studies in the use of visual line of gaze as a selection technique in nonvocal communication. Proposal to National Science Foundation by Tufts-New England Medical Center, Department of Rehabilitation Medicine, Richard A. Foulds, Principal Investigator, July, 1980.
• 2: KAHNEMAN D. Attention and effort. Englewood Cliffs, N.J.: Prentice Hall, 1973.
• 3: KEEN, P. G. W. & SCOTT-MORTON, M.S. Decision support systems: an organizational perspective. Reading, MA: Addison-Wesley, 1978.
• 4: MARKS, L. E. Multimodel perception. In E. C. Carterette & M. P. Friedman, Eds., Handbook of Perception, Vol. VII (Perceptual Coding), New York: Academic Press, 1978.
• 5: MONTY, R. A. & SENDERS J. W. (Eds.) Eye movements and psychological processes. Hillsdale, N.J.: Erlbaum Associates, 1976.
• 6: NEISSER, U. & BECKLEN, R. Selective looking: attending to visually specified events, Cognitive Psychology, 7(4), 480-494, 1975.
• 7: RABB, F. H., BLOOD, E. B., STEINER, T. O. & JONES, H.R. Magnetic position and orientation tracking system. IEEE Transaction on Aerospace and Electronic Systems, Vol. AES-15, No. 5, September 1979, 709-718.
• 8: RINARD, G. A. & RUGG, D. E. Current state of development and testing of an ocular control device. 4th Conference of Systems and Devices for the Disabled, Seattle, WA, June 1977.
• 9: RINARD, G. A. & RUGG, D. E. An ocular control device for use by the severely handicapped. 1976 Conference on Systems and Devices for the Disabled, Boston, MA, 1976.
• 10: SENDERS, J. W. FISHER, D. G. & MONTY, R. A. (Eds.) Eye movements and higher psychological processes. Hillsdale, N.J.: Erlbaum Associates, 1978.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Windowing systems

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.3 Picture/Image Generation

General Terms:
Design, Human Factors, Theory

Keywords:
Eye-tracking, Graphical interface, Man-machine interfaces, Videodisc