Collages as dynamic summaries for news video

ABSTRACT

This paper introduces the video collage, a novel effective interface for browsing and interpreting video collections. The paper discusses how collages are automatically produced, illustrates their use, and evaluates their effectiveness as summaries across news stories. Collages are presentations of text and images derived from multiple video sources, which provide an interactive visualization for a set of video documents, summarizing their contents and providing a navigation aid for further exploration. The dynamic creation of collages is based on user context, e.g., an originating query, coupled with automatic processing to refine the candidate imagery. Named entity identification and common phrase extraction provides descriptive text. The dynamic manipulation of collages allows user-directed browsing and reveals additional detail. The utility of collages as summaries is examined with respect to other published news summaries.

REFERENCES

• 1: Christopher Ahlberg , Ben Shneiderman, Visual information seeking: tight coupling of dynamic query filters with starfield displays, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.313-317, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191775]
• 2: Bates, M.J. The design of browsing and berrypicking techniques for the on-line search interface. Online Review 13(5) (1989), 407--431.
• 3: Daniel M. Bikel , Scott Miller , Richard Schwartz , Ralph Weischedel, Nymble: a high-performance learning name-finder, Proceedings of the fifth conference on Applied natural language processing, p.194-201, March 31-April 03, 1997, Washington, DC [doi>10.3115/974557.974586]
• 4: John Boreczky , Andreas Girgensohn , Gene Golovchinsky , Shingo Uchihashi, An interactive comic book presentation for exploring video, Proceedings of the SIGCHI conference on Human factors in computing systems, p.185-192, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332428]
• 5: Shih-Fu Chang , Gwendal Auffret , Jonathan Foote , Chung-Shen Li , Behzad Shahraray , Tanveer Syeda-Mahmood , HongJiang Zhang, Multimedia access and retrieval (panel session): the state of the art and future directions, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.443-445, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319684]
• 6: Michael G. Christel, Visual digests for news video libraries, Proceedings of the seventh ACM international conference on Multimedia (Part 1), p.303-311, October 30-November 05, 1999, Orlando, Florida, United States [doi>10.1145/319463.319633]
• 7: Christel, M. and Warmack, A. The Effect of Text in Storyboards for Video Navigation, in Proc. IEEE ICASSP (Salt Lake City UT, May 2001), vol. III, 1409--1412.
• 8: Clarkson, P. and Rosenfeld, R. Statistical language modeling using the CMU-Cambridge toolkit, in Proc. Eurospeech '97 (Rhodes, Greece, Sept. 1997), Assoc., 2707--2710.
• 9: Wei Ding , Gary Marchionini , Dagobert Soergel, Multimodal surrogates for video browsing, Proceedings of the fourth ACM conference on Digital libraries, p.85-93, August 11-14, 1999, Berkeley, California, United States [doi>10.1145/313238.313266]
• 10: Google search engine (as of April, 2002), © 2002 Google, <u>http://www.google.com</u>.
• 11: Hearst, M.A. User Interfaces and Visualization. In Modern Information Retrieval, edited by R. Baeza-Yates and B. Ribeiro-Neto, Addison Wesley/ACM Press, New York, 1999.
• 12: Infoplease.com. 2001 People in the News, © 2002 Learning Network, <u>http://www.infoplease.com/ipa/A0878485.html</u>.
• 13: Andrew Large , Jamshid Beheshti , Alin Breuleux , Andre Renaud, Multimedia and comprehension: the relationship among text, animation, and captions, Journal of the American Society for Information Science, v.46 n.5, p.340-347, June 1995 [doi>10.1002/(SICI)1097-4571(199506)46:5<340::AID-ASI5>3.0.CO;2-S]
• 14: Francis C. Li , Anoop Gupta , Elizabeth Sanocki , Li-wei He , Yong Rui, Browsing digital video, Proceedings of the SIGCHI conference on Human factors in computing systems, p.169-176, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332425]
• 15: Andrew Merlino , Daryl Morey , Mark Maybury, Broadcast news navigation using story segmentation, Proceedings of the fifth ACM international conference on Multimedia, p.381-391, November 09-13, 1997, Seattle, Washington, United States [doi>10.1145/266180.266390]
• 16: Miller, D., Schwartz, R., Weischedel, R., and Stone, R. Named Entity Extraction for Broadcast News, in Proc. DARPA Broadcast News Workshop (Washington DC, March 1999), <u>http://www.nist.gov/speech/publications/darpa99/html/ie20/ie20.htm</u>.
• 17: Shneiderman, B. The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. HCI Lab, Inst. Systems Research, Inst. Advanced Computer Studies, Dept. of Computer Science Tech. Report CS-TR-3665, Univ. of Maryland, (July 1996).
• 18: C. J. Van Rijsbergen, Information Retrieval, Butterworth-Heinemann, Newton, MA, 1979
• 19: Howard D. Wactlar , Michael G. Christel , Yihong Gong , Alexander G. Hauptmann, Lessons Learned from Building a Terabyte Digital Video Library, Computer, v.32 n.2, p.66-73, February 1999 [doi>10.1109/2.745722]
• 20: Who2.com. Find Famous People Fast!, © 2002 by Who2?, <u>http://www.who2.com</u>.
• 21: Zhang, H.J., Gong, Y.H., Smoliar, S.W., and Yan, S.Y. Automatic Parsing of news video, in Proc. IEEE Conf. on Multimedia Computing and Systems (Boston MA, May 1994), 45--54.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Abstracting methods

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing

General Terms:
Algorithms, Design, Human Factors

Keywords:
information visualization, video collage, video surrogate