A method and advisor tool for multimedia user interface design

ABSTRACT

This paper describes a multimedia user interface design method and a design assistant tool which supports the method. The method covers specification of user requirements and information architecture, selection of appropriate media to represent the information content, design for directing attention to important information and interaction design to enhance user engagement. Guidelines for media selection and design for attractiveness, i.e. usability and user experience, are given. The method was evaluated in a case study design of a crowd control simulation training system, which demonstrated the method was usable and gave good solutions against an expert gold standard design. The tool provides advice on media selection and attention effects that match specification of the information content expressed as information types and communication goals. A usability evaluation was carried out to measure the usefulness and effectiveness of the tool in comparison to the method, and the results showed that the tool has a positive impact on multimedia design.

REFERENCES

• Alty, 1997.: Multimedia. In: Tucker, A.B. (Ed.), Computer Science and Engineering Handbook, CRC Press, New York. pp. 1551-1570.
• ré et al., 1996.: André, E., Müller, J., Rist, T., 1996. WIP/PPP knowledge-based methods for fully automated multimedia editing. In: Proceedings EUROMEDIA-96, London, pp. 95-102.
• Annett, 1996.: Recent developments in hierarchical task analysis. In: Robertson, S.A. (Ed.), Contemporary Ergonomics 1996, Taylor Francis, London.
• Arens et al., 1993.: Arens, Y., Hovy, E., Van Mulken, S., 1993. Structure and rules in automated multimedia presentation planning. In: Proceedings: IJCAI-93: 13th International Joint Conference on Artificial Intelligence.
• Bailey et al., 2001.: Brian P. Bailey , Joseph A. Konstan , John V. Carlis, DEMAIS: designing multimedia applications with interactive storyboards, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500179]
• Bailey, 1982.: Robert W. Bailey, Human Performance Engineering: A Guide for System Designers, Prentice Hall Professional Technical Reference, 1982
• Bayles, 2002.: Designing online banner advertisements: should we animate?. CHI Letters. v4 i1. 363-366.
• Bernsen, 1994.: Foundations of multimodal representations: a taxonomy of representational modalities. Interacting with Computers. v6 i4. 347-371.
• Bernsen, 2002.: Multimodality in language and speech systems: from theory to design support tool. In: Granström, B., House, D., Karlsson, I. (Eds.), Multimodality in Language and Speech Systems, Kluwer Academic Publishers, Dordrecht. pp. 93-148.
• Bertin, 1983.: Jacques Bertin, Semiology of graphics, University of Wisconsin Press, 1983
• Bevan, 1997.: Usability issues in web site design. In: Smith, G.S., Koubek, R.J. (Eds.), Design of Computing Systems: Social and Ergonomic Considerations: Proceedings Seventh International Conference on Human-Computer Interaction, San Francisco 24-29 August, Elsevier, Berlin. pp. 803-806.
• Bieger and Glock, 1984.: The information content of picture-text instructions. Journal of Experimental Education. v53. 68-76.
• Black et al., 2004.: Black, W., Jowett, S., Mavroudakis, T., McNaught, J., Theodoulidis, B., Zarri, G.P., Zervanou, K., 2004. Ontology-enablement of a system for semantic annotation of digital documents. In: Proceedings SemAnnot 2004: Knowledge Markup and Semantic Annotation Fourth International Workshop, Third International Semantic Web Conference ISWC 2004: http://CEUR-WS.org, Hiroshima.
• Booher, 1975.: Relative comprehensibility of pictorial information and printed word in proceduralized instructions. Human Factors. v17 i3. 266-277.
• Brave and Nass, 2002.: Emotion in human-computer interaction. In: Jacko, J., Sears, A. (Eds.), Handbook of human-computer Interaction, Lawrence Erlbaum Associates, Mahwah, NJ. pp. 251-271.
• Carroll, 2000.: John M. Carroll, Making Use: Scenario-Based Design of Human-Computer Interactions, MIT Press, Cambridge, MA, 2000
• Casner, 1991.: Stephen M. Casner, Task-analytic approach to the automated design of graphic presentations, ACM Transactions on Graphics (TOG), v.10 n.2, p.111-151, April 1991 [doi>10.1145/108360.108361]
• Clark, 1996.: Using Language. Cambridge University Press, Cambridge.
• Cockburn, 2001.: Alistair Cockburn, Writing Effective Use Cases, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2000
• Cowie and Wilks, 2000.: Information extraction. In: Dale, R., Moisl, H., Summers, H. (Eds.), Handbook of Natural Language Processing, Marcel Dekker, New York.
• Diaper and Waelend, 2000.: World wide web working whilst ignoring graphics: good news for web page designers. Interacting with Computers. v13 i2. 163-181.
• Faraday and Sutcliffe, 1996.: Pete Faraday , Alistair Sutcliffe, An empirical study of attending and comprehending multimedia presentations, Proceedings of the fourth ACM international conference on Multimedia, p.265-275, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244225]
• Faraday and Sutcliffe, 1997.: Peter Faraday , Alistair Sutcliffe, Designing effective multimedia presentations, Proceedings of the SIGCHI conference on Human factors in computing systems, p.272-278, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258753]
• Faraday and Sutcliffe, 1998a.: Pete Faraday , Alistair Sutcliffe, Making contact points between text and images, Proceedings of the sixth ACM international conference on Multimedia, p.29-37, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290752]
• Faraday and Sutcliffe, 1998b.: Pete Faraday , Alistair Sutcliffe, Providing advice for multimedia designers, Proceedings of the SIGCHI conference on Human factors in computing systems, p.124-131, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274663]
• Faraday and Sutcliffe, 1999.: Pete Faraday , Alistair Sutcliffe, Authoring animated Web pages using “contact points”, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.458-465, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303131]
• Gardiner and Christie, 1987.: Applying Cognitive Psychology to User Interface Design. Wiley, Chichester.
• Gray and Salzman, 1998.: Wayne D. Gray , Marilyn C. Salzman, Damaged merchandise? a review of experiments that compare usability evaluation methods, Human-Computer Interaction, v.13 n.3, p.203-261, September 1998 [doi>10.1207/s15327051hci1303_2]
• Grice, 1975.: Logic and conversation. Syntax and Semantics. v3.
• Guan and Zhang, 2004.: The design and implementation of a web-based personal digital library. Journal of the Institution of Engineers. v44 i3. 59-77.
• Hallnas and Redstrom, 2002.: Lars Hallnäs , Johan Redström, From use to presence: on the expressions and aesthetics of everyday computational things, ACM Transactions on Computer-Human Interaction (TOCHI), v.9 n.2, p.106-124, June 2002 [doi>10.1145/513665.513668]
• Hegarty and Just, 1993.: Constructing mental models of text and diagrams. Journal of Memory and Language. v32. 717-742.
• Heller and Martin, 1995.: Rachelle S. Heller , C. Dianne Martin, A Media Taxonomy, IEEE MultiMedia, v.2 n.4, p.36-45, August 1995 [doi>10.1109/93.482294]
• Heller et al., 2001.: Rachelle S. Heller , C. Dianne Martin , Nuzi Haneef , Sonja Gievska-Krliu, Using a theoretical multimedia taxonomy framework, Journal on Educational Resources in Computing (JERIC), v.1 n.1es, March 2001 [doi>10.1145/376697.376701]
• Hochberg, 1986.: Presentation of motion and space in video and cinematic displays. In: Boff, K.R., Kaufman, L., Thomas, J.P. (Eds.), Handbook of Perception and Human Performance, 1: Sensory Processes and Perception, Wiley, New York.
• Hornof and Halverson, 2003.: Anthony J. Hornof , Tim Halverson, Cognitive strategies and eye movements for searching hierarchical computer displays, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642656]
• ISO, 1998.: ISO, 1998. ISO 14915 Multimedia user interface design software ergonomic requirements, Part 1: introduction and framework. International Standards Organisation.
• ISO, 2000.: ISO, 2000. ISO 14915-3: Software ergonomics for multimedia user interfaces. Part 3: media selection and combination. International Standards Organisation.
• Johnson et al., 1988.: Peter Johnson , Hilary Johnson , Ray Waddington , Alan Shouls, Task-related knowledge structures: analysis, modelling and application, Proceedings of the Fourth Conference of the British Computer Society on People and computers IV, p.35-62, October 1988, Univ. of Manchester, United Kingdom
• Kristof and Satran, 1995.: Ray Kristof , Amy Satran, Interactivity by Design, Adobe Press, 2002
• Landay and Myers, 2001.: James A. Landay , Brad A. Myers, Sketching Interfaces: Toward More Human Interface Design, Computer, v.34 n.3, p.56-64, March 2001
• Lavie and Tractinsky, 2004.: Talia Lavie , Noam Tractinsky, Assessing dimensions of perceived visual aesthetics of web sites, International Journal of Human-Computer Studies, v.60 n.3, p.269-298, March 2004 [doi>10.1016/j.ijhcs.2003.09.002]
• Levie and Lentz, 1982.: Effects of text illustrations: a review of research. Educational Computing and Technology Journal. v30 i4. 159-232.
• Luz and Bernsen, 2001.: Saturnino Luz , Niels Ole Bernsen, A Tool for Interactive Advice on the Use of Speech in Multimodal Systems, Journal of VLSI Signal Processing Systems, v.29 n.1/2, p.129-137, August-September 2001 [doi>10.1023/A:1011183800658]
• Macdonald, 2003.: What is Web Design? (Essential Design Handbooks). Rotovision, Mies, Switzerland.
• Mann and Thompson, 1988.: Rhetorical structure theory: toward a functional theory of text organisation. Text. v8 i3. 243-281.
• May and Barnard, 1995.: Cinematography and interface design. In: Nordbyn, K., Helmersen, P.H., Gilmore, D.J., Arnesen, S.A. (Eds.), Proceedings: Fifth IFIP TC 13 International Conference on Human-Computer Interaction, Lillehammer, Norway 27-29 June 1995, Chapman&Hall, London. pp. 26-31.
• Maybury, 1999.: Putting usable intelligence into multimedia applications. In: Proceedings of ICMCS99, IEEE International Conference on Multimedia Systems'99, Florence 7-11 June 1999, vol. 1. IEEE Computer Society Press, Los Alamitos, CA. pp. 107-110.
• Mori et al., 2002.: Giulio Mori , Fabio Paternò , Carmen Santoro, CTTE: support for developing and analyzing task models for interactive system design, IEEE Transactions on Software Engineering, v.28 n.8, p.797-813, August 2002 [doi>10.1109/TSE.2002.1027801]
• Mullet and Sano, 1995.: Designing Visual Interfaces: Communication Oriented Techniques. SunSoft Press, Englewood Cliffs, NJ.
• Narayanan and Hegarty, 1998.: N. Hari Narayanan , Mary Hegarty, On designing comprehensible interactive hypermedia manuals, International Journal of Human-Computer Studies, v.48 n.2, p.267-301, feb. 1998 [doi>10.1006/ijhc.1997.0169]
• Narayanan and Hegarty, 2002.: N. Hari Narayanan , M. Hegarty, Multimedia design for communication of dynamic information, International Journal of Human-Computer Studies, v.57 n.4, p.279-315, October 2002 [doi>10.1006/ijhc.2002.1019]
• Nielsen, 1993.: Usability Engineering. Academic Press, Boston, MA.
• Nielsen, 1995.: Multimedia and Hypertext: The Internet and Beyond. AP Professional, Boston, MA.
• Nielsen, 2000.: Jakob Nielsen, Designing Web Usability: The Practice of Simplicity, New Riders Publishing, Thousand Oaks, CA, 1999
• Norman, 1988.: The Psychology of Everyday Things. Basic Books, New York.
• Norman, 2004.: Emotional Design: Why we Love (or Hate) Everyday Things. Basic Books, New York.
• Ortony et al., 1988.: The Cognitive Structure of Emotions. Cambridge University Press, Cambridge.
• Oviatt, 2003.: Sharon Oviatt, Multimodal interfaces, The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, L. Erlbaum Associates Inc., Hillsdale, NJ, 2002
• Patterson, 1982.: Guidelines for Auditory Warnings Systems on Civil Aircraft. CAA Paper 82017. Civil Aviation Authority, London.
• Pezdek and Maki, 1988.: Picture memory: recognizing added and deleted details. Journal of Experimental Psychology: Learning, Memory and Cognition. v14 i3. 468-476.
• Reeves and Nass, 1996.: Byron Reeves , Clifford Nass, The media equation: how people treat computers, television, and new media like real people and places, Cambridge University Press, New York, NY, 1996
• Reeves et al., 2004.: Leah M. Reeves , Jennifer Lai , James A. Larson , Sharon Oviatt , T. S. Balaji , Stéphanie Buisine , Penny Collings , Phil Cohen , Ben Kraal , Jean-Claude Martin , Michael McTear , TV Raman , Kay M. Stanney , Hui Su , Qian Ying Wang, Guidelines for multimodal user interface design, Communications of the ACM, v.47 n.1, January 2004 [doi>10.1145/962081.962106]
• Rosson and Carroll, 1995.: Mary Beth Rosson , John M. Carroll, Integrating task and software development for object-oriented applications, Proceedings of the SIGCHI conference on Human factors in computing systems, p.377-384, May 07-11, 1995, Denver, Colorado, United States [doi>10.1145/223904.223953]
• Scaife and Rogers, 1996.: Mike Scaife , Yvonne Rogers, External cognition: how do graphical representations work?, International Journal of Human-Computer Studies, v.45 n.2, p.185-213, Aug. 1996 [doi>10.1006/ijhc.1996.0048]
• Spool et al., 1999.: Jared M. Spool , Tara Scanlon , Carolyn Snyder , Will Schroeder , Terri DeAngelo, Web site usability: a designer's guide, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• Sutcliffe, 1997.: Alistar Sutcliffe, Task-related information analysis, International Journal of Human-Computer Studies, v.47 n.2, p.223-257, Aug. 1997 [doi>10.1006/ijhc.1997.0118]
• Sutcliffe, 1999.: A design method for effective information delivery in multimedia presentations. New Review of Hypermedia&Multimedia, Applications&Research. v5. 29-58.
• Sutcliffe, 2000.: Pay attention! Or How to make sure the user finds the message in multimedia. In: Scapin, D.L., Vergison, E. (Eds.), Proceedings: ERGO-IHM 2000, Biarritz France, CRT, ILS&ESTAI. pp. 2-12.
• Sutcliffe, 2003.: Multimedia and Virtual Reality: Designing Multisensory User Interfaces. Lawrence Erlbaum Associates, Mahwah, NJ.
• Sutcliffe and Faraday, 1994.: Systematic design for task related multimedia interfaces. Information and Software Technology. v36 i4. 225-234.
• Szekely et al., 1992.: Pedro Szekely , Ping Luo , Robert Neches, Facilitating the exploration of interface design alternatives: the HUMANOID model of interface design, Proceedings of the SIGCHI conference on Human factors in computing systems, p.507-515, May 03-07, 1992, Monterey, California, United States [doi>10.1145/142750.142912]
• Teasdale and Barnard, 1993.: Affect, Cognition and Change: Re-modelling Depressive Thought. Lawrence Erlbaum Associates, Hove.
• Tractinsky et al., 2000.: What is beautiful is usable. Interacting with Computers. v13 i2. 127-145.
• Treisman, 1988.: Features and objects: fourteenth Bartlett memorial lecture. Quarterly Journal of Experimental Psychology. v40A i2. 201-237.
• Tufte, 1997.: Edward R. Tufte, Visual explanations: images and quantities, evidence and narrative, Graphics Press, Cheshire, CT, 1997
• Van der Donckt, 1999.: Development milestones towards a tool for working with guidelines. Interacting with Computers. v12 i2. 81-118.
• Van der Donckt et al., 2001.: Van der Donckt, J., Limbourg, Q., Florins, M., Oger, F., Macq, B., 2001. Synchronised, model-based design of multiple user interfaces. In: Proceedings: 2001 Workshop on Multiple User Interfaces over the Internet (HCI-IHM'2001), Lille 10-14 September 2001.
• Wickens, 2002.: Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science. v3 i2. 150-177.
• Wickens, 1992.: Engineering Psychology and Human Performance. second ed. Harper Collins, New York.
• Wilson et al., 1997.: Stephanie Wilson , Mathilde Bekker , Peter Johnson , Hilary Johnson, Helping and hindering user involvement — a tale of everyday design, Proceedings of the SIGCHI conference on Human factors in computing systems, p.178-185, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258699]
• Zhou and Feiner, 1998.: Michelle X. Zhou , Steven K. Feiner, Visual task characterization for automated visual discourse synthesis, Proceedings of the SIGCHI conference on Human factors in computing systems, p.392-399, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274698]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.4 Systems and Software
          Subjects: Performance evaluation (efficiency and effectiveness)
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems

General Terms:
Design

Keywords:
Advisor tool, Aesthetics, Design method, Guidelines, Multimedia