Predicting graph reading performance: a cognitive approach

ABSTRACT

Performance and preference measures are commonly used in the assessment of visualization techniques. This is important and useful in understanding differences in effectiveness between different treatments. However, these measures do not answer how and why the differences are caused. And sometimes, performance measures alone may not be sensitive enough to detect differences. In this paper, we introduce a cognitive approach for visualization effectiveness and efficiency assessment. A model of user performance, mental effort and cognitive load (memory demand) is proposed and further mental effort and visualization efficiency measures are incorporated into our analysis. It is argued that 1) combining cognitive measures with traditional methods provides us new insights and practical guidance in visualization assessment. 2) analyzing human cognitive process not only helps to understand how viewers interact with visualizations, but also helps to predict user performance in initial stage. 3) keeping cognitive load induced by a visualization low allows more memory resources to be available for high level complex cognitive activities. A case study conducted supports our arguments.

REFERENCES

• 1: Atkinson R., Shiffrin R. (1968) Human memory: A proposed system and its control processes. In K. Spence and J. Spence, editors, The psychology of learning and motivation. Academic Press.
• 2: Baddeley, A. D. (1986) Working memory. Oxford, UK: Oxford University Press.
• 3: Batagelj, V., Mrvar, A. (1998) Pajek: Program for large network analysis. Connections, 21(2):47-57.
• 4: John Bovey , Florence Benoy , Peter Rodgers, Using games to investigate movement for graph comprehension, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989872]
• 5: Stuart K. Card , Jock D. Mackinlay , Ben Shneiderman, Readings in information visualization: using vision to think, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
• 6: Chaomei Chen , Mary P. Czerwinski, Empirical evaluation of information visualizations: an introduction, International Journal of Human-Computer Studies, v.53 n.5, p.631-635, Nov. 2000 [doi>10.1006/ijhc.2000.0421]
• 7: Coy, W. B. (1997) Evaluation anxiety, task-irrelevant self-statements, and working memory: An evaluation of the cognitive interference theory. PHD thesis, Bowling Green State University, Ohio.
• 8: De Waard, D. (1996) The measurement of drivers' mental workload. PHD thesis, University of Groningen.
• 9: Giuseppe Di Battista , Peter Eades , Roberto Tamassia , Ioannis G. Tollis, Graph Drawing: Algorithms for the Visualization of Graphs, Prentice Hall PTR, Upper Saddle River, NJ, 1998
• 10: Earles, J. L., Kersten, A. W., Mas, B. B., Miccio, D. M. (2004) Aging and Memory for Self-Performanced Tasks: Effects of Task Difficulty and Time Pressure. Journal of Gerontology: Psychological Sciences, Vol. 59B, No. 6, 285-293.
• 11: Eysenck, M. W. (1979) Anxiety, learning, and memory: A reconceptualization. Journal of research in Personality, 13, 363-385.
• 12: Mohammad Ghoniem , Jean-Daniel Fekete , Philippe Castagliola, A Comparison of the Readability of Graphs Using Node-Link and Matrix-Based Representations, Proceedings of the IEEE Symposium on Information Visualization (INFOVIS'04), p.17-24, October 10-12, 2004 [doi>10.1109/INFOVIS.2004.1]
• 13: Huang, W., Hong, S., Eades, P. (2005) Layout effects on sociogram perception. Proceedings of 13th International Symposium on Graph Drawing (GD'05), September 2005, Limerick, Ireland.
• 14: Hwang, M. I. (1995) The effectiveness of graphic and tabular presentation under time pressure and task complexity. Information Resources Management Journal. 8(3), 25-31.
• 15: Just, M. A., Carpenter, P. A. (1992) A Capacity Theory of Comprehension: Individual Differences in Working Memory. Psychological Review. Vol. 99, No. 1, 122-149.
• 16: Kalyuga, S., Chandler, P., Sweller, J. (1999) Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology. 13: 351-371.
• 17: Kerick, S. E., Allender, L. E. (2004) Effects of cognitive workload on decision accuracy, shooting performance, and cortical activity of soldiers. http://handle.dtic.mil/100.2/ADA433487. Accessed 15 Nov. 2005
• 18: McGrath, C., Krackhardt, D., Blythe, J. (2003) Visualizing complexity in networks: seeing both the forest and the trees. Connections 25 (1): 37-47.
• 19: Li, W. (2005) Navigating clustered graphs. Masters thesis, University of Sydney.
• 20: Miller, G. (1956) The magical number seven, plus or minus two: some limits on our capacity for processing information. Psychological Review, (63):81-97.
• 21: Paas, F. (1992) Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive load approach. Journal of Educational Psychology 84: 429-434.
• 22: Paas, F., Tuovinen J. E., Tabbers, H., van Gerven, P. W. M. (2003) Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63-71.
• 23: Paas, F., Van Merrienboer, J. (1993) The efficiency of instructional condition: an approach to combine mental effort and performance measures, Human factors 35:734-743.
• 24: Paas, F., Van Merrienboer, J. (1994). Instructional control of cognitive load in the training of complex cognitive tasks. Educational Psychology Review, 6, 51-71.
• 25: Pulo, K. (2005) Structural Focus + Context Navigation of Relational Data. PhD thesis, University of Sydney.
• 26: Purchase, H. C., Carrington, D. A., Allder J-A. (2002) Graph Layout Aesthetics in UML Diagrams: User Preferences. J. Graph Algorithms Appl. 6(3): 255-279.
• 27: Helen C. Purchase, Which Aesthetic has the Greatest Effect on Human Understanding?, Proceedings of the 5th International Symposium on Graph Drawing, p.248-261, September 18-20, 1997
• 28: Purchase, H. C. (1998) Performance of layout algorithms: Comprehension, not computation. Journal of Visual Languages and Computing 9: 647-657.
• 29: Richards, Method and apparatus for processing and displaying multivariate time series data, U. S. patent no. 5,121,469 (June 9, 1992).
• 30: Sjoberg, H. (1977) Interaction of task difficulty, activation, and work load. Journal of Human Stress 3(1), 33-38.
• 31: Tohill, J. M., Holyoak, K. J. (2000) The impact of anxiety on analogical reasoning. Thinking and Reasoning, 6, 27-40.
• 32: Melanie Tory , Torsten Möller, Human Factors in Visualization Research, IEEE Transactions on Visualization and Computer Graphics, v.10 n.1, p.72-84, January 2004 [doi>10.1109/TVCG.2004.1260759]
• 33: Tuovinen, J., Paas, F. (2004) Exploring multidimensional approaches to the efficiency of instructional conditions. Instructional Science 32: 133-152.
• 34: Van Gerven, P., Paas, F., Van Merrienboer, J., Schmidt, H. (2003) Memory load and the cognitive papillary response in aging. Psychophysiology, 41, 167-174.
• 35: Velez, M. C., Silver, D., Tremaine, M. (2005) Understanding Visualization through Spatial Ability Differences. Proceedings of IEEE Visualization, October, 2005, 23-28. Minneapolis, MN, USA.
• 36: Colin Ware , Helen Purchase , Linda Colpoys , Matthew McGill, Cognitive measurements of graph aesthetics, Information Visualization, v.1 n.2, p.103-110, June 2002 [doi>10.1057/palgrave.ivs.9500013]
• 37: Wickens, C, D., Hollands, J. G. (1999) Engineering psychology and human performance. Third edition, Prentice Hall.
• 38: Wood, R. E. (1986) Task complexity: definition of the construct. Organizational behavior and human decision processes, 37, 60-82.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  G. Mathematics of Computing
  G.2 DISCRETE MATHEMATICS
      G.2.2 Graph Theory

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.0 General
          Subjects: Cognitive simulation

General Terms:
Design, Human Factors, Performance

Keywords:
cognitive load, cognitive model, graph reading, mental effort, social network, visualization efficiency