Addressing the challenge of visual information access from digital image and video libraries

ABSTRACT

While it would seem that digital video libraries should benefit from access mechanisms directed to their visual contents, years of TREC Video Retrieval Evaluation (TRECVID) research have shown that text search against transcript narrative text provides almost all the retrieval capability, even with visually oriented generic topics. A within-subjects study involving 24 novice participants on TRECVID 2004 tasks again confirms this result. The study shows that satisfaction is greater and performance is significantly better on specific and generic information retrieval tasks from news broadcasts when transcripts are available for search. Additional runs with 7 expert users reveal different novice and expert interaction patterns with the video library interface, helping explain the novices' lack of success with image search and visual feature browsing for visual information needs. Analysis of TRECVID visual features well suited for particular tasks provides additional insights into the role of automated feature classification for digital image and video libraries.

REFERENCES

• 1: Michael Christel , Neema Moraveji, Finding the right shots: assessing usability and performance of a digital video library interface, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027691]
• 2: Enser, P.G.B. Pictorial information retrieval. Journal of Documentation, 51, 2 (1995), 126--170.
• 3: Peter Enser , Christine Sandom, Retrieval of Archival Moving Imagery - CBIR Outside the Frame?, Proceedings of the International Conference on Image and Video Retrieval, p.206-214, July 18-19, 2002
• 4: James C. French , A. C. Chapin , Worthy N. Martin, An application of multiple viewpoints to content-based image retrieval, Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, May 27-31, 2003, Houston, Texas
• 5: Graham, J. Search engine Google sets sights on video. USA Today, Jan. 25, 2005, www.usatoday.com/tech/news/2005-01-25-google-usat_x.htm http://www.usatoday.com/tech/news/2005-01-25-google-usat_x.htm.
• 6: Alexander G. Hauptmann , Michael G. Christel, Successful approaches in the TREC video retrieval evaluations, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027681]
• 7: Joemon M. Jose , Jonathan Furner , David J. Harper, Spatial querying for image retrieval: a user-oriented evaluation, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.232-240, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291000]
• 8: Kraaij, W., Smeaton, A.F., Over, P., and Arlandis, J. TRECVID 2004 - An Introduction. In TRECVID 2004 Proceedings, http://www-nlpir.nist.gov/projects/trecvid/.
• 9: Lee, H. and Smeaton, A.F. Designing the User Interface for the Físchlár Digital Video Library, J. Digital Info. 2(4), http://jodi.ecs.soton.ac.uk/Articles/v02/i04/Lee/, May 2002.
• 10: Marjo Markkula , Eero Sormunen, End-User Searching Challenges Indexing Practices inthe Digital Newspaper Photo Archive, Information Retrieval, v.1 n.4, p.259-285, January 2000 [doi>10.1023/A:1009995816485]
• 11: Nicolas Moënne-Loccoz , Bruno Janvier , Stéphane Marchand-Maillet , Eric Bruno, Managing video collections at large, Proceedings of the 1st international workshop on Computer vision meets databases, June 13-13, 2004, Paris, France [doi>10.1145/1039470.1039484]
• 12: Jakob Nielsen, Heuristic evaluation, Usability inspection methods, John Wiley & Sons, Inc., New York, NY, 1994
• 13: Rasmussen, E. Indexing images. Annual Review of Information Science and Technology, 32 (1997), 169-196.
• 14: Kerry Rodden , Kenneth R. Wood, How do people manage their digital photographs?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642682]
• 15: Shatford, S. Analyzing the Subject of a Picture: A Theoretical Approach. Cataloguing & Classification Quarterly, 6, 3 (Spring 1986), 39--62.
• 16: Shneiderman, B., Byrd, D., and Croft, W.B. Clarifying Search: A User-Interface Framework for Text Searches. D-Lib Magazine, 3, 1 (January 1997), <http://www.dlib.org>.
• 17: Arnold W. M. Smeulders , Marcel Worring , Simone Santini , Amarnath Gupta , Ramesh Jain, Content-Based Image Retrieval at the End of the Early Years, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.12, p.1349-1380, December 2000 [doi>10.1109/34.895972]
• 18: Von-Wun Soo , Chen-Yu Lee , Chung-Cheng Li , Shu Lei Chen , Ching-chih Chen, Automated semantic annotation and retrieval based on sharable ontology and case-based learning techniques, Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, May 27-31, 2003, Houston, Texas
• 19: Urban, J., Jose, J.M., and van Rijgsbergen, C.J. An Adaptive Technique for Content-Based Image Retrieval. Multimedia Tools and Applications, 25 (2005).
• 20: Marcel Worring , Arnold W. M. Smeulders , Simone Santini, Interaction in Content-Based Image Retrieval: The Evaluation of the State-of-the-Art Review, Proceedings of the 4th International Conference on Advances in Visual Information Systems, p.26-36, November 02-04, 2000
• 21: Meng Yang , Barbara M. Wildemuth , Gary Marchionini, The relative effectiveness of concept-based versus content-based video retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027613]
• 22: Ka-Ping Yee , Kirsten Swearingen , Kevin Li , Marti Hearst, Faceted metadata for image search and browsing, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642681]
• 23: Zhou, X.S., and Huang, T. Relevance Feedback in Image Retrieval: A Comprehensive Review. ACM Multimedia Systems Journal, 8, 6 (2003), 536--544.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.7 Digital Libraries
          Subjects: User issues
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

General Terms:
Experimentation, Human Factors

Keywords:
TRECVID results analysis, video retrieval