Face annotation for personal photos using context-assisted face recognition

ABSTRACT

Face annotation for personal photos has a number of potential applications in Multimedia Information Retrieval (MIR). We propose a novel face annotation method that systematically combines contextual information of photos with traditional Face Recognition (FR) technologies. To overcome the limitations of conventional FR-based annotation approaches, situation and person clustering methods are proposed based on context of personal photos. By relying on the proposed clustering methods, multiple facial images that belong to the same person are applied to subspace FR methods. This allows traditional FR methods to take the advantages of feature- or measurement-level fusion strategies for automatic annotation of faces on personal photos. The effectiveness of the proposed method was successfully tested on 1,120 personal photos selected from standard MPEG-7 Visual Core Experiment 3 (VCE-3). Extensive experimental results showed that the proposed method improves annotation accuracy by at least an order of magnitude compared with conventional FR-based approaches.

REFERENCES

• 1: Kerry Rodden , Kenneth R. Wood, How do people manage their digital photographs?, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642682]
• 2: Andreas Girgensohn , John Adcock , Lynn Wilcox, Leveraging face recognition technology to find and organize photos, Proceedings of the 6th ACM SIGMM international workshop on Multimedia information retrieval, October 15-16, 2004, New York, NY, USA [doi>10.1145/1026711.1026728]
• 3: R. Chellappa, C. L. Wilson, and S. Sirohey, "Human and Machine Recognition of Faces: A Survey," Proc. IEEE, vol. 83, no. 5, pp. 705--740, 1995.
• 4: Bongwon Suh , Benjamin B. Bederson, Semi-automatic photo annotation strategies using event based clustering and clothing based person recognition, Interacting with Computers, v.19 n.4, p.524-544, July, 2007 [doi>10.1016/j.intcom.2007.02.002]
• 5: Lei Zhang , Yuxiao Hu , Mingjing Li , Weiying Ma , Hongjiang Zhang, Efficient propagation for face annotation in family albums, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027689]
• 6: L. Chen, B. Hu, L. Zhang, M. Li, and H.J. Zhang, "Face annotation for family photo album management," International Journal of Image and Graphics, vol. 3, no. 1, pp. 1--14, 2003.
• 7: S. Cooray et al., "Identifying Person Reoccurrences for Personal Photo Management Applications," Proc. of IEEE conf. on VIE, 2006.
• 8: A. Jain, K. Nandakumar, and A. Ross, "Score normalization in multimodal biometric systems," Pattern Recognit., vol. 38, no. 12, pp. 2270--2285, 2005.
• 9: J. Yang, J.Y. Yang, D. Zhang, and J.F Lu, "Feature Fusion: Parallel Strategy vs. Serial Strategy," Pattern Recognit, vol. 36, no. 2, pp. 1369--1381, 2003.
• 10: Matthew Turk , Alex Pentland, Eigenfaces for recognition, Journal of Cognitive Neuroscience, v.3 n.1, p.71-86, Winter 1991 [doi>10.1162/jocn.1991.3.1.71]
• 11: Peter N. Belhumeur , João P. Hespanha , David J. Kriegman, Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.711-720, July 1997 [doi>10.1109/34.598228]
• 12: B. Moghaddam, T. Jebara, and A. Pentland, "Bayesian face recognition," Pattern Recognit., vol. 33, no. 11, pp. 1771--1782, 2000.
• 13: M. Bober, S.K. Bober, "Description of MPEG-7 Visual Core Experiments," MPEG-7 Visual Group ISO/IEC JTC1/SC29/WG11, N6905, 2005
• 14: Jie Wang , K. N. Plataniotis , Juwei Lu , A. N. Venetsanopoulos, On solving the face recognition problem with one training sample per subject, Pattern Recognition, v.39 n.9, p.1746-1762, September, 2006 [doi>10.1016/j.patcog.2006.03.010]
• 15: Terence Sim , Simon Baker , Maan Bsat, The CMU Pose, Illumination, and Expression Database, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.12, p.1615-1618, December 2003 [doi>10.1109/TPAMI.2003.1251154]
• 16: P. Jonathon Phillips , Hyeonjoon Moon , Syed A. Rizvi , Patrick J. Rauss, The FERET Evaluation Methodology for Face-Recognition Algorithms, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.10, p.1090-1104, October 2000 [doi>10.1109/34.879790]
• 17: "Exchangeable image file format for digital still cameras: Exif Version 2.2 (JEITA CP-3451)," Standard of Japan Electronics and Information Technology Industries Association, 2002.
• 18: S. Yang, S. K. Kim, K. S. Seo, and Y. M. Ro. "Automated Situation Change Clustering of Home Photos for Digital Albuming," Proc. of SPIE on Electronic Imaging, 2005.
• 19: C.-L. Liu, "Classifier combination based on confidence transformation," Pattern Recognit., vol. 38, no. 1, pp. 11--28, 2005.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.4 Applications
          Subjects: Computer vision

General Terms:
Algorithms, Experimentation, Management, Performance

Keywords:
context, face annotation, face recognition, personal photos