Finding the right shots: assessing usability and performance of a digital video library interface

ABSTRACT

The authors developed a system in which visually dense displays of thumbnail imagery in storyboard views are used for shot-based video retrieval. The views allow for effective retrieval, as evidenced by the success achieved by expert users with the system in interactive query for NIST TRECVID 2002 and 2003. This paper demonstrates that novice users also achieve comparatively high retrieval performance with these views using the TRECVID 2003 benchmarks. Through an analysis of the user interaction logs, heuristic evaluation, and think-aloud protocol, the usability of the video information retrieval system is appraised with respect to shot-based retrieval. Design implications are presented based on these TRECVID usability evaluations regarding efficient, effective information retrieval interfaces to locate visual information from video corpora.

REFERENCES

• 1: Christel, M., Huang, C., Moraveji, N., and Papernick, N. Exploiting Multiple Modalities for Interactive Video Retrieval. In Proc. ICASSP 2004 (Montreal, Canada, May 2004), Vol. III, pp. 1032--1035.
• 2: Alan Dix , Janet Finley , Gregory Abowd , Russell Beale, Human-computer interaction (2nd ed.), Prentice-Hall, Inc., Upper Saddle River, NJ, 1998
• 3: Ann Doubleday , Michele Ryan , Mark Springett , Alistair Sutcliffe, A comparison of usability techniques for evaluating design, Proceedings of the conference on Designing interactive systems: processes, practices, methods, and techniques, p.101-110, August 18-20, 1997, Amsterdam, The Netherlands [doi>10.1145/263552.263583]
• 4: Jean-Luc Gauvain , Lori Lamel , Gilles Adda, The LIMSI Broadcast News transcription system, Speech Communication, v.37 n.1-2, p.89-108, May 2002 [doi>10.1016/S0167-6393(01)00061-9]
• 5: Anita Komlodi , Gary Marchionini, Key frame preview techniques for video browsing, Proceedings of the third ACM conference on Digital libraries, p.118-125, June 23-26, 1998, Pittsburgh, Pennsylvania, United States [doi>10.1145/276675.276688]
• 6: Lee, H. and Smeaton, A.F. Designing the User Interface for the Fischlar Digital Video Library. J. Digital Info, 2, 4 (May 2002), http://jodi.ecs.soton.ac.uk/Articles/v02/i04/Lee/.
• 7: Francis C. Li , Anoop Gupta , Elizabeth Sanocki , Li-wei He , Yong Rui, Browsing digital video, Proceedings of the SIGCHI conference on Human factors in computing systems, p.169-176, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332425]
• 8: Janni Nielsen , Torkil Clemmensen , Carsten Yssing, Getting access to what goes on in people's heads?: reflections on the think-aloud technique, Proceedings of the second Nordic conference on Human-computer interaction, October 19-23, 2002, Aarhus, Denmark [doi>10.1145/572020.572033]
• 9: Jakob Nielsen , Rolf Molich, Heuristic evaluation of user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.249-256, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97281]
• 10: Jakob Nielsen, Evaluating the thinking-aloud technique for use by computer scientists, Advances in human-computer interaction (vol. 3), Ablex Publishing Corp., Norwood, NJ, 1993
• 11: Jakob Nielsen, Heuristic evaluation, Usability inspection methods, John Wiley & Sons, Inc., New York, NY, 1994
• 12: NIST, Digital Video Retrieval at NIST: TREC Video Retrieval Evaluation, 2001-2004, http://www-nlpir.nist.gov/projects/trecvid/.
• 13: Dulce Ponceleon , Savitha Srinivasan , Arnon Amir , Dragutin Petkovic , Dan Diklic, Key to effective video retrieval: effective cataloging and browsing, Proceedings of the sixth ACM international conference on Multimedia, p.99-107, September 13-16, 1998, Bristol, United Kingdom [doi>10.1145/290747.290760]
• 14: Rowe, L.A. and Jain, R. ACM SIGMM Retreat Report on Future Directions in Multimedia Research, March 2004, www.acm.org/sigmm/main/events/sigmm_retreat/sigmm-retreat03-final.htm.
• 15: Smeaton, A., Kraaij, W., and Over P. The TREC Video Retrieval Evaluation (TRECVID): A Case Study and Status Report. In Proc. RIAO 2004 (Avignon, France, April 2004).
• 16: Smeaton, A.F., and Over, P. The TREC-2002 Video Track Report. In TREC 2002 Proc. (NIST, Gaithersburg, MD, Nov. 2002), http://trec.nist.gov/pubs/trec11/t11_proceedings.html.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

General Terms:
Experimentation, Human Factors

Keywords:
TRECVID, storyboard, video retrieval