White rooms and morphing don't mix: setting and the evaluation of visualization techniques

ABSTRACT

The results presented in this paper illustrate how a specific map visualization technique is sensitive to setting: a comparative evaluation of the technique gives conflicting results depending on where it takes place. While prior research has explored the impact of factors other than basic visual perception on visualization techniques, relatively little attention has been directed toward the physical setting in which the technique is used. We present results from a study involving 120 participants, comparing the effectiveness of two different geovisualization techniques in promoting recall of map layout. Recall was shown to be sensitive to setting, such that one technique in particular was more effective in a noisy public space than in a controlled, 'white-room' environment. The results have implications for the validation and measurement of information visualization techniques as a whole, and in particular for those employing motion as a communicative attribute.

REFERENCES

• 1: Baecker R.M. et al, Human-Computer Interaction: Toward the Year 2000, Chapter 2: Usability Inspection Methods, Morgan Kaufmann, 1995.
• 2: Bell, A., Fisher, J. D., Baum, A., and Greene, T. E. Environental Psychology, 3rd Edition. Holt, Reinhart and Winston, Inc. 1990.
• 3: Hugh Beyer , Karen Holtzblatt, Contextual design, interactions, v.6 n.1, p.32-42, Jan./Feb. 1999 [doi>10.1145/291224.291229]
• 4: Michael D. Byrne , Richard Catrambone , John T. Stasko, Evaluating animations as student aids in learning computer algorithms, Computers & Education, v.33 n.4, p.253-278, Dec. 1999 [doi>10.1016/S0360-1315(99)00023-8]
• 5: Giuseppe Carenini, An analysis of the influence of need for cognition on dynamic queries usage, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634293]
• 6: Chaomei Chen , Mary P. Czerwinski, Empirical evaluation of information visualizations: an introduction, International Journal of Human-Computer Studies, v.53 n.5, p.631-635, Nov. 2000 [doi>10.1006/ijhc.2000.0421]
• 7: Cleveland, W.S. and McGill, R. Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods. Journal of the American Statistical Association, 79, 387 (1984), 531--554.
• 8: G. O. Domik , B. Gutkauf, User modeling for adaptive visualization systems, Proceedings of the conference on Visualization '94, October 17-21, 1994, Washinton, D.C.
• 9: Brian J. Foley, Designing visualization tools for learning, CHI 98 conference summary on Human factors in computing systems, p.309-310, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/286498.286784]
• 10: Freundschuh, S. Micro- and Macro-scale environments. In Cognitive Mapping, Past, Present and Future. Kitchin, R. and Freundschuh, S. (Eds.), Routledge, London, UK (2000) 125--146.
• 11: Garling, T. and Evans, G. W. (Eds.) Environment, Cognition, and Action -- an Integrated Approach. Oxford University Press, Oxford, UK (1992).
• 12: Golledge, R. G., Cognition of Physical and Built Environments, in Environment, Cognition, and Action -- an Integrated Approach, Garling, T. and Evans, G. W. (Eds.), Oxford University Press, Oxford, UK (1992).
• 13: Christopher G. Healey , Kellogg S. Booth , James T. Enns, High-speed visual estimation using preattentive processing, ACM Transactions on Computer-Human Interaction (TOCHI), v.3 n.2, p.107-135, June 1996 [doi>10.1145/230562.230563]
• 14: Christopher G. Healey , James T. Enns, Building perceptual textures to visualize multidimensional datasets, Proceedings of the conference on Visualization '98, p.111-118, October 18-23, 1998, Research Triangle Park, North Carolina, United States
• 15: Hutchins, E. Cognition in the Wild. MIT Press, Cambridge, USA (1995).
• 16: Kitchin, R. and Freundschuh, S. (Eds.) Cognitive Mapping, Past, Present and Future. Routledge, London, UK (2000).
• 17: Robert Kosara , Christopher G. Healey , Victoria Interrante , David H. Laidlaw , Colin Ware, User Studies: Why, How, and When?, IEEE Computer Graphics and Applications, v.23 n.4, p.20-25, July 2003 [doi>10.1109/MCG.2003.1210860]
• 18: Kosslyn, S. M. Elements of Graph Design. W.H. Freeman & Co., New York, USA (1994).
• 19: Kuller, R., Environmental Assessment from a Neuropsychological Perspective, in Environment, Cognition, and Action -- an Integrated Approach, Garling, T. and Evans, G. W. (Eds.). Oxford University Press, Oxford, UK (1992).
• 20: Lloyd, R. Understanding and Learning Maps. In Cognitive Mapping, Past, Present and Future. Kitchin, R. and Freundschuh, S. (Eds.), Routledge, London, UK (2000) 84--107.
• 21: MacEachren, A.M., How Maps Work: Representation, Visualization and Design. The Guilford Press, New York, USA (1995).
• 22: Mack, A. and Rock, I. Inattentional Blindness. MIT Press, Cambridge, USA (1998).
• 23: Mark, D. M., Freska, C., Hirtle, S. C., Lloyd, R., and Tversky, B. Cognitive models of geographical space. Int. J. Geographical Information Science, 13, 8 (1999), 747--774.
• 24: Cognitive Effects of Animated Visualization in Exploratory Visual Data Analysis, Proceedings of the Fifth International Conference on Information Visualisation, p.77, July 25-27, 2001
• 25: Catherine Plaisant, The challenge of information visualization evaluation, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989880]
• 26: Pomerantz, J. R. Perceptual organization in information processing. In Issues in Cognitive Modeling, A. M. Aitkenhead and J. M. Slack (Eds.), Erlbaum, Hillsdale, USA (1985), 127--158.
• 27: Pylyshyn, Z. Some Primitive Mechanisms of Spatial Attention. Cognition 50 (1994), 363--384.
• 28: Reiber, L.P. Animation as Feedback in a Computer-Based Simulation: Representation Matters. In Journal of Educational Technology Research and Development, 44, 1 (1996), 5--22.
• 29: Derek F. Reilly , Kori M. Inkpen, Map morphing: making sense of incongruent maps, Proceedings of Graphics Interface 2004, p.231-238, May 17-19, 2004, London, Ontario, Canada
• 30: Sedig, K., Rowhani, S., Morey, J., and Liang, H. Application of information visualization techniques to the design of a mathematical mindtool: a usability study. Information Visualization 2 (2003), 142--159.
• 31: Semin, G. R. and Smith, E. R. Interfaces of social psychology with situated and embodied cognition. Journal of Cognitive Systems Research 3 (2002) 385--396.
• 32: Melanie Tory , Torsten Möller, Human Factors in Visualization Research, IEEE Transactions on Visualization and Computer Graphics, v.10 n.1, p.72-84, January 2004 [doi>10.1109/TVCG.2004.1260759]
• 33: J. Gregory Trafton , Susan S. Kirschenbaum , Ted L. Tsu , Robert T. Miyamoto , James A. Ballas , Paula D. Raymond, Turning pictures into numbers: extracting and generating information from complex visualizations, International Journal of Human-Computer Studies, v.53 n.5, p.827-850, Nov. 2000 [doi>10.1006/ijhc.2000.0419]
• 34: Colin Ware , Robert Bobrow, Motion to support rapid interactive queries on node--link diagrams, ACM Transactions on Applied Perception (TAP), v.1 n.1, p.3-18, July 2004 [doi>10.1145/1008722.1008724]
• 35: Zhang, J., Johnson, K. A., Malin, J. T., Smith, J. W., Human-Centered Information Visualization, International Workshop on Dynamic Visualization and Learning, Tubingen, Germany, July 18--19, 2002.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Interaction styles (e.g., commands, menus, forms, direct manipulation)

Keywords:
GIS, contextual evaluation, ecological validity, external validity, geographic visualization, information visualization, methodology, morphing