Multiple instance learning for labeling faces in broadcasting news video

ABSTRACT

Labeling faces in news video with their names is an interesting research problem which was previously solved using supervised methods that demand significant user efforts on labeling training data. In this paper, we investigate a more challenging setting of the problem where there is no complete information on data labels. Specifically, by exploiting the uniqueness of a face's name, we formulate the problem as a special multi-instance learning (MIL) problem, namely exclusive MIL or eMIL problem, so that it can be tackled by a model trained with partial labeling information as the anonymity judgment of faces, which requires less user effort to collect. We propose two discriminative probabilistic learning methods named Exclusive Density (ED) and Iterative ED for eMIL problems. Experiments on the face labeling problem shows that the performance of the proposed approaches are superior to the traditional MIL algorithms and close to the performance achieved by supervised methods trained with complete data labels.

REFERENCES

• 1: S. Andrews, I. Tsochantaridis, and T. Hofmann. Support vector machines for multiple-instance learning. In Advances in Neural Information Processing Systems 15, pages 561--568. MIT Press, 2003.
• 2: T. Berg, A. Berg, J. Edwards, M. Maire, R. White, Y.-W. Teh, E. Learned-Miller, and D. Forsyth. Names and faces in news. In Proc. of Conf. on Computer Vision and Pattern Recognition, pages 848--854. IEEE Computer Society, 2004.
• 3: D. Bikel, S. Miller, R. Schwartz, and R. Weischedel. Nymble: a high-performance learning name-finder, 1997.
• 4: S. F. Chang, R. Manmatha, and T. S. Chua. Combining text and audio-visual features in video indexing. In IEEE ICASSP 2005, 2005.
• 5: M. Chen and A. Hauptmann. Toward robust face recognition from multiple views. In Proc. of Int'l Conference on Multimedia and Expo, 2004.
• 6: Thomas G. Dietterich , Richard H. Lathrop , Tomás Lozano-Pérez, Solving the multiple instance problem with axis-parallel rectangles, Artificial Intelligence, v.89 n.1-2, p.31-71, Jan. 1997 [doi>10.1016/S0004-3702(96)00034-3]
• 7: Ricky Houghton, Named Faces: Putting Names to Faces, IEEE Intelligent Systems, v.14 n.5, p.45-50, September 1999 [doi>10.1109/5254.796089]
• 8: J. Jeon , V. Lavrenko , R. Manmatha, Automatic image annotation and retrieval using cross-media relevance models, Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, July 28-August 01, 2003, Toronto, Canada [doi>10.1145/860435.860459]
• 9: Oded Maron , Tomás Lozano-Pérez, A framework for multiple-instance learning, Proceedings of the 1997 conference on Advances in neural information processing systems 10, p.570-576, July 1998, Denver, Colorado, United States
• 10: Oded Maron , Aparna Lakshmi Ratan, Multiple-Instance Learning for Natural Scene Classification, Proceedings of the Fifteenth International Conference on Machine Learning, p.341-349, July 24-27, 1998
• 11: Shin'ichi Satoh , Takeo Kanade, Name-It: Association of Face and Name in Video, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.368, June 17-19, 1997
• 12: Henry Schneiderman , Takeo Kanade, Object Detection Using the Statistics of Parts, International Journal of Computer Vision, v.56 n.3, p.151-177, February-March 2004 [doi>10.1023/B:VISI.0000011202.85607.00]
• 13: C. Snoek, M. Worring, and A. Hauptmann. Detection ofTVnews monologues by style analysis. In Proc. of theIEEEInt'l Conference on Multimedia & Expo, June 2004.
• 14: Xiaodan Song , Ching-Yung Lin , Ming-Ting Sun, Autonomous visual model building based on image crawling through internet search engines, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026762]
• 15: Xiaodan Song , Ching-Yung Lin , Ming-Ting Sun, Cross-Modality Automatic Face Model Training from Large Video Databases, Proceedings of the 2004 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'04) Volume 5, p.91, June 27-July 02, 2004
• 16: Jun Wang , Jean-Daniel Zucker, Solving the Multiple-Instance Problem: A Lazy Learning Approach, Proceedings of the Seventeenth International Conference on Machine Learning, p.1119-1126, June 29-July 02, 2000
• 17: Yi Wu , Edward Y. Chang , Kevin Chen-Chuan Chang , John R. Smith, Optimal multimodal fusion for multimedia data analysis, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027665]
• 18: Rong Yan , Milind Naphade, Semi-Supervised Cross Feature Learning for Semantic Concept Detection in Videos, Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) - Volume 1, p.657-663, June 20-26, 2005 [doi>10.1109/CVPR.2005.317]
• 19: Image Database Retrieval with Multiple-Instance Learning Techniques, Proceedings of the 16th International Conference on Data Engineering, p.233, February 28-March 03, 2000
• 20: J. Yang, M. Chen, and A. G. Hauptmann. Finding personX: Correlating names with visual appearances. In Proc. of 3rd Int'l Conf. on Image and Video Retrieval, pages 270--278, 2004.
• 21: Jun Yang , Alexander G. Hauptmann, Naming every individual in news video monologues, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027666]
• 22: Q. Zhang and S. Goldman. Em-DD: An improved multiple-instance learning technique. In Advances in Neural Information Processing Systems, pages 1073--1080. TheMITPress, 2001.
• 23: Qi Zhang , Sally A. Goldman , Wei Yu , Jason Fritts, Content-Based Image Retrieval Using Multiple-Instance Learning, Proceedings of the Nineteenth International Conference on Machine Learning, p.682-689, July 08-12, 2002

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Experimentation, Performance

Keywords:
face labeling, machine learning, multiple instance learning, news video