Exploiting redundancy in cross-channel video retrieval

ABSTRACT

Video producers, in telling a news story, tend to repeat important visual and speech material multiple times in adjacent shots, thus creating a certain level of redundancy. We describe this phenomenon, and use it to develop a framework to incorporate redundancy for cross-channel retrieval of visual items using speech. Testing our models in a series of retrieval experiments, we find that incorporating the fact that information occurs redundantly into cross-channel retrieval leads to significant improvements in retrieval performance.

REFERENCES

• 1: M. Campbell, A. Haubold, S. Ebadollahi, M. R. Naphade, A. P. Natsev, J. R. Smith, J. Tesic, and L. Xie. IBM research TRECVID--2006 video retrieval system. In TREC Video Retrieval Evaluation Proceedings, 2006.
• 2: S.-F. Chang, W. Hsu, W. Jiang, L. Kennedy, X. Dong, A. Yanagawa, and E. Zavesky. Columbia University TRECVID-2006 video search and high-leve feature extraction. In TREC Video Retrieval Evaluation Proceedings, 2006.
• 3: T.-S. Chua, S.-Y. Neo, Y. Zheng, H.-K. Goh, Y. Xiao, S. Tang, and M. Zhao. TRECVID 2006 by NUS-I2R. In TREC Video Retrieval Evaluation Proceedings, 2006.
• 4: A. G. Hauptmann, M.-Y. Chen, M. Christel, W.-H. Lin, R. Yan, and J. Yang. Multi-lingual broadcast news retrieval. In TREC Video Retrieval Evaluation Proceedings, 2006.
• 5: D. Hiemstra. Using Language Models for Information Retrieval. PhD thesis, University of Twente, 2001.
• 6: Bouke Huurnink , Maarten de Rijke, The value of stories for speech-based video search, Proceedings of the 6th ACM international conference on Image and video retrieval, p.266-271, July 09-11, 2007, Amsterdam, The Netherlands [doi>10.1145/1282280.1282322]
• 7: Milind Naphade , John R. Smith , Jelena Tesic , Shih-Fu Chang , Winston Hsu , Lyndon Kennedy , Alexander Hauptmann , Jon Curtis, Large-Scale Concept Ontology for Multimedia, IEEE MultiMedia, v.13 n.3, p.86-91, July 2006 [doi>10.1109/MMUL.2006.63]
• 8: Jay M. Ponte , W. Bruce Croft, A language modeling approach to information retrieval, Proceedings of the 21st annual international ACM SIGIR conference on Research and development in information retrieval, p.275-281, August 24-28, 1998, Melbourne, Australia [doi>10.1145/290941.291008]
• 9: M. F. Porter, An algorithm for suffix stripping, Readings in information retrieval, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1997
• 10: Amit Singhal , Fernando Pereira, Document expansion for speech retrieval, Proceedings of the 22nd annual international ACM SIGIR conference on Research and development in information retrieval, p.34-41, August 15-19, 1999, Berkeley, California, United States [doi>10.1145/312624.312645]
• 11: Alan F. Smeaton , Paul Over , Wessel Kraaij, Evaluation campaigns and TRECVid, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178722]
• 12: Stephen W. Smoliar , HongJiang Zhang, Content-Based Video Indexing and Retrieval, IEEE MultiMedia, v.1 n.2, p.62-72, June 1994 [doi>10.1109/93.311653]
• 13: Cees G. M. Snoek , Marcel Worring , Jan C. van Gemert , Jan-Mark Geusebroek , Arnold W. M. Smeulders, The challenge problem for automated detection of 101 semantic concepts in multimedia, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180727]
• 14: C. G. M. Snoek, B. Huurnink, L. Hollink, M. de Rijke, G. Schreiber, and M. Worring. Adding semantics to detectors for video retrieval. IEEE Transactions on Multimedia, 9(5), August 2007. In press.
• 15: C. G. M. Snoek, J. C. van Gemert, T. Gevers, B. Huurnink, D. C. Koelma, M. V. Liempt, O. D. Rooij, K. E. A. van de Sande, F. J. Seinstra, A. W. M. Smeulders, A. H. Thean, C. J. Veenman, and M. Worring. The MediaMill TRECVID 2006 semantic video search engine. In TREC Video Retrieval Evaluation Proceedings, 2006.
• 16: Tao Tao , Xuanhui Wang , Qiaozhu Mei , ChengXiang Zhai, Language model information retrieval with document expansion, Proceedings of the main conference on Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics, p.407-414, June 04-09, 2006, New York, New York [doi>10.3115/1220835.1220887]
• 17: Jan C. van Gemert , Cees G. M. Snoek , Cor J. Veenman , Arnold W. M. Smeulders, The influence of cross-validation on video classification performance, Proceedings of the 14th annual ACM international conference on Multimedia, October 23-27, 2006, Santa Barbara, CA, USA [doi>10.1145/1180639.1180786]
• 18: Jun Yang , Alexander G. Hauptmann, Exploring temporal consistency for video analysis and retrieval, Proceedings of the 8th ACM international workshop on Multimedia information retrieval, October 26-27, 2006, Santa Barbara, California, USA [doi>10.1145/1178677.1178685]
• 19: J. Yang, M. yu Chen, and A. G. Hauptmann. Finding person X: Correlating names with visual appearances. In CIVR, volume 3115 of Lecture Notes in Computer Science, pages 270--278. Springer, 2004.
• 20: Chengxiang Zhai , John Lafferty, A study of smoothing methods for language models applied to information retrieval, ACM Transactions on Information Systems (TOIS), v.22 n.2, p.179-214, April 2004 [doi>10.1145/984321.984322]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
      H.3.4 Systems and Software
  H.4 INFORMATION SYSTEMS APPLICATIONS
      H.4.2 Types of Systems
      H.4.m Miscellaneous

General Terms:
Algorithms, Experimentation, Measurement, Performance

Keywords:
document expansion, language modeling, redundancy, speech-based video retrieval