Naming faces in broadcast news video by image google

ABSTRACT

Naming faces is important for news videos browsing and indexing. Although some research efforts have been contributed to it, they only use the concurrent information between the face and name or employ some clues as features and use simple heuristic method or machine learning approach to finish the task. They use little extra knowledge about the names and faces. Different from previous work, in this paper we present a novel approach to name the faces by exploring extra knowledge obtained from image google. The behind assumption is that the faces of those important persons will turn out many times in the web images and could be retrieved from image google easily. Firstly, faces are detected in the video frames; and the name entities of candidate persons are extracted from the textual information by automatic speech recognition and close caption detection. Then, these candidate person names are used as queries to find the name related person images through image google. After that, the retrieved result is analyzed and some typical faces are selected through feature density estimation. Finally, the detected faces in the news video are matched with the faces selected from the result returned by image google to label each face. Experimental results on MSNBC news and CNN news demonstrate that the proposed approach is effective.

REFERENCES

• 1: Shin'ichi Satoh , Takeo Kanade, Name-It: Association of Face and Name in Video, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.368, June 17-19, 1997
• 2: T. L. Berg, A. C. Berg, J. Edwards, M. Maire, R. White, Y. W. Teh, E. Miller, D. A. Foryth. Names and Faces in the News. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, pp. 848--854, 2004.
• 3: Derya Ozkan , Pynar Duygulu, A Graph Based Approach for Naming Faces in News Photos, Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, p.1477-1482, June 17-22, 2006 [doi>10.1109/CVPR.2006.29]
• 4: Jun Yang , Alexander G. Hauptmann, Naming every individual in news video monologues, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027666]
• 5: Jun Yang , Rong Yan , Alexander G. Hauptmann, Multiple instance learning for labeling faces in broadcasting news video, Proceedings of the 13th annual ACM international conference on Multimedia, November 06-11, 2005, Hilton, Singapore [doi>10.1145/1101149.1101155]
• 6: L. Chaisorn, T.-S Chua, C.-K Koh, Y.-L Zhao, H. Xu, H. Feng and Q. Tian. A two-level Multi-modal Approach for Story Segmentation of Large News Video Corpus. TRECVID workshop, 2003.
• 7: J. Chen, X. Chen, W. Gao. Expand Training Set for Face Detection by GA Re-sampling. In Proc. IEEE int'l conf. on automatic face and gesture recognition, 2004.
• 8: Alias-i. Lingpipe named entity tagger. In http://www.aliasi.com/lingpipe/.
• 9: Q. Ye and Q. Huang. A New Text Detection Algorithm in Image/Video Frames. Lecture note in computer science, Pacific-Rim conference on Multimedia, pp. 858--865, 2004.
• 10: http://www.hw99.com
• 11: K. Fukunaga and L. Hostetler. The Estimation of the Gradient of a Density Function, with Applications in Pattern Recognition. IEEE Transactions on Information Theory, vol.21, no.1, pp.32--40, 1975.
• 12: Y. Su, S. Shan, X. Chen and W. Gao. Hierarchical Ensemble of Global and Local Classifiers for Face Recognition. In Proc. of IEEE Int'l Conf. on Computer Vision, pp. 1--8, 2007.

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
        I.2.10 Vision and Scene Understanding
            Subjects: Video analysis

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

General Terms:
Algorithms, Design, Experimentation

Keywords:
naming faces, news video analysis, news video browsing and indexing