Mnemonic rendering: an image-based approach for exposing hidden changes in dynamic displays

ABSTRACT

Managing large amounts of dynamic visual information involves understanding changes happening out of the user's sight. In this paper, we show how current software does not adequately support users in this task, and motivate the need for a more general approach. We propose an image-based storage, visualization, and implicit interaction paradigm called mnemonic rendering that provides better support for handling visual changes. Once implemented on a system, mnemonic rendering techniques can benefit all applications. We explore its rich design space and discuss its expected benefits as well as limitations based on feedback from users of a small-screen and a wall-size prototype.

REFERENCES

• 1: Banbury, B. and Tremblay, S., eds. (2004). A cognitive approach to situation awareness: theory and application. Ashgate.
• 2: Baudisch, P., Cutrell, E., and Robertson, G. (2003). High-density cursor: A visualization technique that helps users keep track of fast-moving mouse cursors. Interact. p. 236--243.
• 3: Patrick Baudisch , Carl Gutwin, Multiblending: displaying overlapping windows simultaneously without the drawbacks of alpha blending, Proceedings of the SIGCHI conference on Human factors in computing systems, p.367-374, April 24-29, 2004, Vienna, Austria [doi>10.1145/985692.985739]
• 4: Eric A. Bier , Maureen C. Stone , Ken Pier , William Buxton , Tony D. DeRose, Toolglass and magic lenses: the see-through interface, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.73-80, September 1993 [doi>10.1145/166117.166126]
• 5: Bay-Wei Chang , Jock D. Mackinlay , Polle T. Zellweger , Takeo Igarashi, A negotiation architecture for fluid documents, Proceedings of the 11th annual ACM symposium on User interface software and technology, p.123-132, November 01-04, 1998, San Francisco, California, United States [doi>10.1145/288392.288585]
• 6: Bay-Wei Chang , David Ungar, Animation: from cartoons to the user interface, Proceedings of the 6th annual ACM symposium on User interface software and technology, p.45-55, December 1993, Atlanta, Georgia, United States [doi>10.1145/168642.168647]
• 7: Olivier Chapuis , Nicolas Roussel, Metisse is not a 3D desktop!, Proceedings of the 18th annual ACM symposium on User interface software and technology, October 23-26, 2005, Seattle, WA, USA [doi>10.1145/1095034.1095038]
• 8: Cockburn, A., Greenberg, S., Jones, S., McKenzie, B., and Moyle, M. (2003). Improving web page revisitation: Analysis, design and evaluation. Special Issue on Web Navigation Skills, IT & Society, 3(1). p. 159--183.
• 9: W. Keith Edwards , Takeo Igarashi , Anthony LaMarca , Elizabeth D. Mynatt, A temporal model for multi-level undo and redo, Proceedings of the 13th annual ACM symposium on User interface software and technology, p.31-40, November 06-08, 2000, San Diego, California, United States [doi>10.1145/354401.354409]
• 10: Endsley, M., Bolté, B., and Jones, D. (2003). Designing for situation awareness: CRC.
• 11: Freeman, E. and Fertig, S. (1995). Lifestreams: Organizing your electronic life. AAAI Fall Symposium: AI Applications in Knowledge Navigation and Retrieval.
• 12: Jonathan Grudin, Partitioning digital worlds: focal and peripheral awareness in multiple monitor use, Proceedings of the SIGCHI conference on Human factors in computing systems, p.458-465, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365312]
• 13: Hunt, A. R. and Andkingstone, A. (2003). Covert and overt voluntary attention: linked or independent? Cognitive Brain Research, 18. p. 102--105.
• 14: Dugald Ralph Hutchings , Greg Smith , Brian Meyers , Mary Czerwinski , George Robertson, Display space usage and window management operation comparisons between single monitor and multiple monitor users, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989867]
• 15: Dugald Ralph Hutchings , John Stasko, Revisiting display space management: understanding current practice to inform next-generation design, Proceedings of the 2004 conference on Graphics interface, p.127-134, May 17-19, 2004, London, Ontario, Canada
• 16: Itti, L., Rees, G., and Tsotsos, J., eds. (2005). Neurobiology of attention. Elsevier.
• 17: Jacob, R. J. K. (1993). Eye-movement-based human-computer interaction techniques: Toward non-command interfaces. In Advances in Human-Computer Interaction, D. Hix, Editor. Ablex. chapter 6. p. 151--190.
• 18: Christopher Jaynes , Stephen Webb , R. Matt Steele , Michael Brown , W. Brent Seales, Dynamic shadow removal from front projection displays, Proceedings of the conference on Visualization '01, October 21-26, 2001, San Diego, California
• 19: Kawasaki, Y. and Igarashi, T. (2004). Regional undo for spreadsheets. ACM UIST Symposium on User Interface Software and Technology. Demo abstract.
• 20: David Kurlander , Steven Feiner, A history-based macro by example system, Proceedings of the 5th annual ACM symposium on User interface software and technology, p.99-106, November 15-18, 1992, Monteray, California, United States [doi>10.1145/142621.142633]
• 21: Mertz, C., Chatty, S., and Vinot, J. (2000). Pushing the limits of ATC user interface design beyond S&M interaction: the DigiStrips experience. 3rd USA/Europe Air Traffic Management R&D Seminar.
• 22: Tomer Moscovich , John F. Hughes, Navigating documents with the virtual scroll ring, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029642]
• 23: Jun Rekimoto, Time-machine computing: a time-centric approach for the information environment, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.45-54, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322582]
• 24: Renaud, K. (2000). Expediting rapid recovery from interruptions by providing a visualization of application activity. OZCHI. p. 348--355.
• 25: Rensink, R. A. (2002). Change detection. Annual Review of Psychology, 53. p. 245--277.
• 26: Meredith Ringel, When one isn't enough: an analysis of virtual desktop usage strategies and their implications for design, CHI '03 extended abstracts on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/765891.765976]
• 27: Salembier, P. and Marques, F. (1999). Region-based representations of image and video: Segmentation tools for multimedia services. IEEE Transactions on Circuits and Systems for Video Technology, 9(8). p. 1147--1169.
• 28: Amy Skopik , Carl Gutwin, Improving revisitation in fisheye views with visit wear, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055079]
• 29: G. M. Smith , m. c. schraefel, The radial scroll tool: scrolling support for stylus- or touch-based document navigation, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029641]
• 30: James Tam , Saul Greenberg, A framework for asynchronous change awareness in collaborative documents and workspaces, International Journal of Human-Computer Studies, v.64 n.7, p.583-598, July 2006 [doi>10.1016/j.ijhcs.2006.02.004]
• 31: Tomitsch, M. (2003). Trends and evolution of window interfaces. Thesis, University of Technology: Vienna.
• 32: Matthew Turk, Computer vision in the interface, Communications of the ACM, v.47 n.1, January 2004 [doi>10.1145/962081.962107]
• 33: Varakin, D. A. and Levin, D. T. (2004). Unseen and unaware: Implications of recent research on failures of visual awareness for human-computer interface design. Human-Computer Interaction, 19. p. 389--422.
• 34: Venolia, G. D., Dabbish, L., Cadiz, J. J., and Gupta, A. (2001). Supporting email Workflow. Tech Report MSR-TR-2001-88. Microsoft Research.
• 35: Roel Vertegaal , Aadil Mamuji , Changuk Sohn , Daniel Cheng, Media eyepliances: using eye tracking for remote control focus selection of appliances, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1057041]
• 36: Xiaoxing Wang , Jesse Jin, A quantitative analysis for decomposing visual signal of the gaze displacement, Proceedings of the Pan-Sydney area workshop on Visual information processing, p.153-159, May 01, 2001, Sydney, Australia
• 37: Alan Wexelblat , Pattie Maes, Footprints: history-rich tools for information foraging, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.270-277, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303060]
• 38: Steve Whittaker , Candace Sidner, Email overload: exploring personal information management of email, Proceedings of the SIGCHI conference on Human factors in computing systems: common ground, p.276-283, April 13-18, 1996, Vancouver, British Columbia, Canada [doi>10.1145/238386.238530]
• 39: Woods, D. D. and Watts, J. C. (1997). How not to have to navigate through too many displays. In Handbook of Human-Computer Interaction. Elsevier Science.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

General Terms:
Design, Human Factors

Keywords:
change visualization, mnemonic rendering