An empirical evaluation of interactive visualizations for preferential choice

ABSTRACT

Many critical decisions for individuals and organizations are often framed as preferential choices: the process of selecting the best option out of a set of alternatives. This paper presents a task-based empirical evaluation of ValueCharts, a set of interactive visualization techniques to support preferential choice. The design of our study is grounded in a comprehensive task model and we measure both task performance and insights. In the experiment, we not only tested the overall usefulness and effectiveness of ValueCharts, but we also assessed the differences between two versions of ValueCharts, a horizontal and a vertical one. The outcome of our study is that ValueCharts seem very effective in supporting preferential choice and the vertical version appears to be more effective than the horizontal one.

REFERENCES

• 1: Robert Amar , John Stasko, BEST PAPER: A Knowledge Task-Based Framework for Design and Evaluation of Information Visualizations, Proceedings of the IEEE Symposium on Information Visualization, p.143-150, October 10-12, 2004 [doi>10.1109/INFOVIS.2004.10]
• 2: Natalia Andrienko , Gennady Andrienko, Informed spatial decisions through coordinated views, Information Visualization, v.2 n.4, p.270-285, December 2003 [doi>10.1057/palgrave.ivs.9500058]
• 3: Toshiyuki Asahi , David Turo , Ben Shneiderman, Visual decision-making: using treemaps for the analytic hierarchy process, Conference companion on Human factors in computing systems, p.405-406, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223355.223747]
• 4: R. Bade, F. Ritter, and B. Preim. Usability comparison of mouse-based interaction techniques for predictable 3d rotation. In Smart Graphics, pages 138--150, 2005.
• 5: Jeanette Bautista , Giuseppe Carenini, An integrated task-based framework for the design and evaluation of visualizations to support preferential choice, Proceedings of the working conference on Advanced visual interfaces, May 23-26, 2006, Venezia, Italy [doi>10.1145/1133265.1133308]
• 6: V. Belton. VISA: Visual Interactive Sensitivity Analysis. SIMUL8 Corporation, Boston, MA, 2008.
• 7: Stuart K. Card , Jock D. Mackinlay , Ben Shneiderman, Readings in information visualization: using vision to think, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1999
• 8: Giuseppe Carenini , John Loyd, ValueCharts: analyzing linear models expressing preferences and evaluations, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989885]
• 9: R. T. Clemen. Making Hard Decisions. Duxbury Press, Belmont, CA, USA, 2nd edition, 1996.
• 10: J. Cohen. Statistical Power Analysis for the Behavioral Sciences (2 ed.). Lawrence Earlbaum associates.
• 11: B. DiEugenio, M. Glass, and M. J. Trolio. The DIAG experiments: Natural language generation for intelligent tutoring systems. In The Second International Natural Language Generation Conference.
• 12: W. Edwards and F. H. Barron. SMARTS and SMARTER: Improved Simple Methods for Multiattribute Utility Measurement. Organizational Behavior and Human Decision Processes, 60(4):306--325, 1996.
• 13: K. A. Ericsson and H. A. Simon. Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge, MA, 1984.
• 14: Ken Hinckley , Randy Pausch , Dennis Proffitt , James Patten , Neal Kassell, Cooperative bimanual action, Proceedings of the SIGCHI conference on Human factors in computing systems, p.27-34, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258571]
• 15: Janni Nielsen , Torkil Clemmensen , Carsten Yssing, Getting access to what goes on in people's heads?: reflections on the think-aloud technique, Proceedings of the second Nordic conference on Human-computer interaction, October 19-23, 2002, Aarhus, Denmark [doi>10.1145/572020.572033]
• 16: Catherine Plaisant, The challenge of information visualization evaluation, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989880]
• 17: Purvi Saraiya , Chris North , Karen Duca, An Evaluation of Microarray Visualization Tools for Biological Insight, Proceedings of the IEEE Symposium on Information Visualization, p.1-8, October 10-12, 2004 [doi>10.1109/INFOVIS.2004.5]
• 18: Ben Shneiderman, The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations, Proceedings of the 1996 IEEE Symposium on Visual Languages, p.336, September 03-06, 1996
• 19: S. Siegel and N. J. J. Castellan. Nonparametric statistics for the behavioral sciences. McGraw Hill, 1988.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.2 Types of Systems
          Subjects: Decision support (e.g., MIS)

  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.6 Methodology and Techniques
          Subjects: Interaction techniques

General Terms:
Design, Experimentation, Human Factors

Keywords:
empirical evaluation, preferential choice, user studies, visualization techniques