Naming every individual in news video monologues

ABSTRACT

Naming every individual person appearing in broadcast news videos with names detected from the video transcript leads to better access of the news video content. In this paper, we approach this challenging problem with a statistical learning method. Two categories of information extracted from multiple video modalities have been explored, namely <i>features</i>, which help distinguish the true name of every person, as well as <i>constraints</i>, which reveal the relationships among the names of different persons. The person-naming problem is formulated into a learning framework which predicts the most likely name for each person based on the features, and refines the predictions using the constraints. Experiments conducted on ABC World New Tonight and CNN Headline News videos demonstrate that this approach outperforms a non-learning alternative by a large amount.

REFERENCES

• 1: Daniel M. Bikel , Scott Miller , Richard Schwartz , Ralph Weischedel, Nymble: a high-performance learning name-finder, Proceedings of the fifth conference on Applied natural language processing, p.194-201, March 31-April 03, 1997, Washington, DC [doi>10.3115/974557.974586]
• 2: Berg, T., Berg, A., Edwards, J., Maire, M., White, R., Teh, Y.W., Miller, E., Foryth, D. Names and Faces in the News. In Proc. of Computer Vision and Pattern Recognition, Vol.2, pp. 848--854, 2004.
• 3: Christopher J. C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, v.2 n.2, p.121-167, June 1998 [doi>10.1023/A:1009715923555]
• 4: Jean-Luc Gauvain , Lori Lamel , Gilles Adda, The LIMSI Broadcast News transcription system, Speech Communication, v.37 n.1-2, p.89-108, May 2002 [doi>10.1016/S0167-6393(01)00061-9]
• 5: Alexander G. Hauptmann , Michael J. Witbrock, Informedia: news-on-demand multimedia information acquisition and retrieval, Intelligent multimedia information retrieval, MIT Press, Cambridge, MA, 1997
• 6: Ricky Houghton, Named Faces: Putting Names to Faces, IEEE Intelligent Systems, v.14 n.5, p.45-50, September 1999 [doi>10.1109/5254.796089]
• 7: Rong, Y., Zhang, J., Yang, J. and Hauptmann, A. A Discriminative Learning Framework with Pair-wise Constraints for Video Object Classification. In Proc. of Computer Vision and Pattern Recognition, Vol.2, pp. 284--291, 2004.
• 8: Toshio Sato , Takeo Kanade , Ellen K. Hughes , Michael A. Smith , Shin'ichi Satoh, Video OCR: indexing digital new libraries by recognition of superimposed captions, Multimedia Systems, v.7 n.5, p.385-395, September 1999 [doi>10.1007/s005300050140]
• 9: Shin'ichi Satoh , Takeo Kanade, Name-It: Association of Face and Name in Video, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.368, June 17-19, 1997
• 10: Henry Schneiderman , Takeo Kanade, Object Detection Using the Statistics of Parts, International Journal of Computer Vision, v.56 n.3, p.151-177, February-March 2004 [doi>10.1023/B:VISI.0000011202.85607.00]
• 11: Snoek, C.G.M. and Hauptmann, A. Learning to identify TV news monologues by style and context. Technical Report, CMU-CS-03-193, Carnegie Mellon University, 2003.
• 12: TRECVID: TREC Video Retrieval Evaluation: http://www-nlpir.nist.gov/projects/trecvid/.
• 13: Yang, J., Chen, M.Y., Hauptmann, A. Finding Person X: Correlating Names with Visual Appearances. Int'l Conf. on Image and Video Retrieval, 2004. (To appear)
• 14: HongJiang Zhang , Shuang Yeo Tan , Stephen W. Smoliar , Gong Yihong, Automatic parsing and indexing of news video, Multimedia Systems, v.2 n.6, p.256-266, Jan. 1995 [doi>10.1007/BF01225243]
• 15: Lei Zhang , Longbin Chen , Mingjing Li , Hongjiang Zhang, Automated annotation of human faces in family albums, Proceedings of the eleventh ACM international conference on Multimedia, November 02-08, 2003, Berkeley, CA, USA [doi>10.1145/957013.957090]
• 16: W. Zhao , R. Chellappa , P. J. Phillips , A. Rosenfeld, Face recognition: A literature survey, ACM Computing Surveys (CSUR), v.35 n.4, p.399-458, December 2003 [doi>10.1145/954339.954342]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Experimentation

Keywords:
broadcast news video, equivalence constraint, machine learning, multi-modality, person naming