ACM Home Page
Please provide us with feedback. Feedback
Face recognition: A literature survey
Full text PdfPdf (4.28 MB)
Source ACM Computing Surveys (CSUR) archive
Volume 35 ,  Issue 4  (December 2003) table of contents
Pages: 399 - 458  
Year of Publication: 2003
ISSN:0360-0300
Authors
W. Zhao  Sarnoff Corporation, Princeton, NJ
R. Chellappa  University of Maryland, MD
P. J. Phillips  National Institute of Standards and Technology, Gaithersburg, MD
A. Rosenfeld  University of Maryland, MD
Publisher
ACM  New York, NY, USA
Bibliometrics
Downloads (6 Weeks): 461,   Downloads (12 Months): 3530,   Citation Count: 161
Additional Information:

abstract   references   cited by   index terms   collaborative colleagues  

Tools and Actions: Request Permissions Request Permissions    Review this Article  
DOI Bookmark: Use this link to bookmark this Article: http://doi.acm.org/10.1145/954339.954342
What is a DOI?

ABSTRACT

As one of the most successful applications of image analysis and understanding, face recognition has recently received significant attention, especially during the past several years. At least two reasons account for this trend: the first is the wide range of commercial and law enforcement applications, and the second is the availability of feasible technologies after 30 years of research. Even though current machine recognition systems have reached a certain level of maturity, their success is limited by the conditions imposed by many real applications. For example, recognition of face images acquired in an outdoor environment with changes in illumination and/or pose remains a largely unsolved problem. In other words, current systems are still far away from the capability of the human perception system.This paper provides an up-to-date critical survey of still- and video-based face recognition research. There are two underlying motivations for us to write this survey paper: the first is to provide an up-to-date review of the existing literature, and the second is to offer some insights into the studies of machine recognition of faces. To provide a comprehensive survey, we not only categorize existing recognition techniques but also present detailed descriptions of representative methods within each category. In addition, relevant topics such as psychophysical studies, system evaluation, and issues of illumination and pose variation are covered.


REFERENCES

Note: OCR errors may be found in this Reference List extracted from the full text article. ACM has opted to expose the complete List rather than only correct and linked references.

 
1
 
2
Akamatsu, S., Sasaki, T., Fukamachi, H., Masui, N., and Suenaga, Y. 1992. An accurate and robust face identification scheme. In Proceedings, International Conference on Pattern Recognition. 217--220.
 
3
Joseph J. Atick , Paul A. Griffin , A. Norman Redlich, Statistical approach to shape from shading: reconstruction of three-dimensional face surfaces from single two-dimensional images, Neural Computation, v.8 n.6, p.1321-1340, Aug. 1996
 
4
 
5
Bachmann, T. 1991. Identification of spatially quantized tachistoscopic images of faces: How many pixels does it take to carry identity? European J. Cog. Psych. 3, 87--103.
 
6
Bailly-Bailliere, E., Bengio, S., Bimbot, F., Hamouz, M., Kittler, J., Mariethoz, J., Matas, J., Messer, K., Popovici, V., Poree, F., Ruiz, B., and Thiran, J. P. 2003. The BANCA database and evaluation protocol. In Proceedings of the International Conference on Audio- and Video-Based Biometric Person Authentication. 625--638.
 
7
Bartlett, J. C. and Searcy, J. 1993. Inversion and configuration of faces. Cog. Psych. 25, 281--316.
 
8
Bartlett, M. S., Lades, H. M., and Sejnowski, T. 1998. Independent component representation for face recognition. In Proceedings, SPIE Symposium on Electronic Imaging: Science and Technology. 528--539.
 
9
Basri, R. and Jacobs, D. W. 2001. Lambertian refelectances and linear subspaces. In Proceedings, International Conference on Computer Vision. Vol. II. 383--390.
 
10
 
11
 
12
 
13
Bell, A. J. and Sejnowski, T. J. 1997. The independent components of natural scenes are edge filters. Vis. Res. 37, 3327--3338.
 
14
Beveridge, J. R., She, K., Draper, B. A., and Givens, G. H. 2001. A nonparametric statisical comparison of principal component and linear discriminant subspaces for face recognition. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition. (An updated version can be found online at http://www.cs.colostate.edu/evalfacerec/news.html.)
 
15
 
16
 
17
 
18
Biederman, I. 1987. Recognition by components: A theory of human image understanding. Psych. Rev. 94, 115--147.
 
19
Biederman, I. and Kalocsai, P. 1998. Neural and psychophysical analysis of object and face recognition. In Face Recognition: From Theory to Applications, H. Wechsler, P. J. Phillips, V. Bruce, F. F. Soulie, and T. S. Huang, Eds. Springer-Verlag, Berlin, Germany, 3--25.
 
20
Bigun, J., Duc, B., Smeraldi, F., Fischer, S., and Makarov, A. 1998. Multi-modal person authentication. In Face Recognition: From Theory to Applications, H. Wechsler, P. J. Phillips, V. Bruce, F. F. Soulie, and T. S. Huang, Eds. Springer-Verlag, Berlin, Germany, 26--50.
 
21
 
22
Black, M. and Yacoob, Y. 1995. Tracking and recognizing facial expressions in image sequences using local parametrized models of image motion. Tech. rep. CS-TR-3401. Center for Automation Research, Unversity of Maryland, College Park, MD.
 
23
Blackburn, D., Bone, M., and Phillips, P. J. 2001. Face recognition vendor test 2000. Tech. rep. http://www.frvt.org.
 
24
 
25
 
26
Bledsoe, W. W. 1964. The model method in facial recognition. Tech. rep. PRI:15, Panoramic research Inc., Palo Alto, CA.
 
27
Brand, M. and Bhotika, R. 2001. Flexible flow for 3D nonrigid tracking and shape recovery. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition.
 
28
Brennan, S. E. 1985. The caricature generator. Leonardo, 18, 170--178.
 
29
Bronstein, A., Bronstein, M., Gordon, E., and Kimmel, R. 2003. 3D face recognition using geometric invariants. In Proceedings, International Conference on Audio- and Video-Based Person Authentication.
 
30
Bruce, V. 1988. Recognizing faces, Lawrence Erlbaum Associates, London, U.K.
 
31
Bruce, V., Burton, M., and Dench, N. 1994. What's distinctive about a distinctive face? Quart. J. Exp. Psych. 47A, 119--141.
 
32
Bruce, V., Hancock, P. J. B., and Burton, A. M. 1998. Human face perception and identification. In Face Recognition: From Theory to Applications, H. Wechsler, P. J. Phillips, V. Bruce, F. F. Soulie, and T. S. Huang, Eds. Springer-Verlag, Berlin, Germany, 51--72.
 
33
Bruner, I. S. and Tagiuri, R. 1954. The perception of people. In Handbook of Social Psychology, Vol. 2, G. Lindzey, Ed., Addison-Wesley, Reading, MA, 634--654.
 
34
Buhmann, J., Lades, M., and Malsburg, C. v. d. 1990. Size and distortion invariant object recognition by hierarchical graph matching. In Proceedings, International Joint Conference on Neural Networks. 411--416.
 
35
Chellappa, R., Wilson, C. L., and Sirohey, S. 1995. Human and machine recognition of faces: A survey. Proc. IEEE, 83, 705--740.
 
36
Choudhury, T., Clarkson, B., Jebara, T., and Pentland, A. 1999. Multimodal person recognition using unconstrained audio and video. In Proceedings, International Conference on Audio- and Video-Based Person Authentication. 176--181.
 
37
 
38
 
39
 
40
 
41
Craw, I. and Cameron, P. 1996. Face recognition by computer. In Proceedings, British Machine Vision Conference. 489--507.
 
42
Darwin, C. 1972. The Expression of the Emotions in Man and Animals. John Murray, London, U.K.
 
43
 
44
 
45
 
46
Ekman, P. Ed., 1998. Charles Darwin's The Expression of the Emotions in Man and Animals, Third Edition, with Introduction, Afterwords and Commentaries by Paul Ekman. HarperCollins/Oxford University Press, New York, NY/London, U.K.
 
47
Ellis, H. D. 1986. Introduction to aspects of face processing: Ten questions in need of answers. In Aspects of Face Processing, H. Ellis, M. Jeeves, F. Newcombe, and A. Young, Eds. Nijhoff, Dordrecht, The Netherlands, 3--13.
 
48
Etemad, K. and Chellappa, R. 1997. Discriminant analysis for recognition of human face images. J. Opt. Soc. Am. A 14, 1724--1733.
 
49
Fisher, R. A. 1938. The statistical utilization of multiple measuremeents. Ann. Eugen. 8, 376--386.
 
50
 
51
 
52
Galton, F. 1888. Personal identification and description. Nature, (June 21), 173--188.
 
53
Gauthier, I., Behrmann, M., and Tarr, M. J. 1999. Can face recognition really be dissociated from object recognition? J. Cogn. Neurosci. 11, 349--370.
 
54
Gauthier, I. and Logothetis, N. K. 2000. Is face recognition so unique after All? J. Cogn. Neuropsych. 17, 125--142.
 
55
 
56
 
57
 
58
Ginsburg, A. G. 1978. Visual information processing based on spatial filters constrained by biological data. AMRL Tech. rep. 78--129.
 
59
 
60
Gordon, G. 1991. Face recognition based on depth maps and surface curvature. In SPIE Proceedings, Vol. 1570: Geometric Methods in Computer Vision. SPIE Press, Bellingham, WA 234--247.
 
61
Gu, L., Li, S. Z., and Zhang, H. J. 2001. Learning probabilistic distribution model for multiview face dectection. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition.
 
62
 
63
Hallinan, P. W. 1991. Recognizing human eyes. In SPIE Proceedings, Vol. 1570: Geometric Methods In Computer Vision. 214--226.
 
64
Hallinan, P. W. 1994. A low-dimensional representation of human faces for arbitrary lighting conditions. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition. 995--999.
 
65
Hancock, P., Bruce, V., and Burton, M. 1998. A comparison of two computer-based face recognition systems with human perceptions of faces. Vis. Res. 38, 2277--2288.
 
66
Harmon, L. D. 1973. The recognition of faces. Sci. Am. 229, 71--82.
 
67
Heisele, B., Serre, T., Pontil, M., and Poggio, T. 2001. Component-based face detection. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition.
 
68
Hill, H. and Bruce, V. 1996. Effects of lighting on matching facial surfaces. J. Exp. Psych.: Human Percept. Perform. 22, 986--1004.
 
69
Hill, H., Schyns, P. G., and Akamatsu, S. 1997. Information and viewpoint dependence in face recognition. Cognition 62, 201--222.
 
70
Hjelmas, E. and Low, B. K. 2001. Face detection: A Survey. Comput. Vis. Image Understand. 83, 236--274.
 
71
 
72
Huang, J., Heisele, B., and Blanz, V. 2003. Component-based face recognition with 3D morphable models. In Proceedings, International Conference on Audio- and Video-Based Person Authentication.
 
73
 
74
 
75
Jebara, T., Russel, K., and Pentland, A. 1998. Mixture of eigenfeatures for real-time structure from texture. Tech. rep. TR-440, MIT Media Lab, Massachusetts Institute of Technology, Cambridge, MA.
 
76
Johnston, A., Hill, H., and Carman, N. 1992. Recognizing faces: Effects of lighting direction, inversion and brightness reversal. Cognition 40, 1--19.
 
77
 
78
Kanade, T. 1973. Computer recognition of human faces. Birkhauser, Basel, Switzerland, and Stuttgart, Germany.
 
79
Kelly, M. D. 1970. Visual identification of people by computer. Tech. rep. AI-130, Stanford AI Project, Stanford, CA.
 
80
 
81
Klasen, L. and Li, H. 1998. Faceless identification. In Face Recognition: From Theory to Applications, H. Wechsler, P. J. Phillips, V. Bruce, F. F. Soulie, and T. S. Huang, Eds. Springer-Verlag, Berlin, Germany, 513--527.
 
82
Knight, B. and Johnston, A. 1997. The role of movement in face recognition. Vis. Cog. 4, 265--274.
 
83
Kruger, N., Potzsch, M., and Malsburg, C. v. d. 1997. Determination of face position and pose with a learned representation based on labelled graphs. Image Vis. Comput. 15, 665--673.
 
84
Kung, S. Y. and Taur, J. S. 1995. Decision-based neural networks with signal/image classification applications. IEEE Trans. Neural Netw. 6, 170--181.
 
85
 
86
Lanitis, A., Taylor, C. J., and Cootes, T. F. 1995. Automatic face identification system using flexible appearance models. Image Vis. Comput. 13, 393--401.
 
87
Lawrence, S., Giles, C. L., Tsoi, A. C., and Back, A. D. 1997. Face recognition: A convolutional neural-network approach. IEEE Trans. Neural Netw. 8, 98--113.
 
88
Li, B. and Chellappa, R. 2001. Face verification through tracking facial features. J. Opt. Soc. Am. 18.
 
89
Li, S. Z. and Lu, J. 1999. Face recognition using the nearest feature line method. IEEE Trans. Neural Netw. 10, 439--443.
 
90
Li, Y., Gong, S., and Liddell, H. 2001a. Constructing facial identity surfaces in a nonlinear discriminating space. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition.
 
91
Li, Y., Gong, S., and Liddell, H. 2001b. Modelling face dynamics across view and over time. In Proceedings, International Conference on Computer Vision.
 
92
Lin, S. H., Kung, S. Y., and Lin, L. J. 1997. Face recognition/detection by probabilistic decision-based neural network. IEEE Trans. Neural Netw. 8, 114--132.
 
93
 
94
Liu, C. and Wechsler, H. 2000b. Robust coding scheme for indexing and retrieval from large face databases. IEEE Trans. Image Process. 9, 132--137.
 
95
Liu, C. and Wechsler, H. 2001. A shape- and texture-based enhanced fisher classifier for face recognition. IEEE Trans. Image Process. 10, 598--608.
 
96
Liu, J. and Chen, R. 1998. Sequential Monte Carlo methods for dynamic systems. J. Am. Stat. Assoc. 93, 1031--1041.
 
97
Manjunath, B. S., Chellappa, R., and Malsburg, C. v. d. 1992. A feature based approach to face recognition. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition. 373--378.
 
98
Marr, D. 1982. Vision. W. H. Freeman, San Francisco, CA.
 
99
 
100
 
101
Maurer, T. and Malsburg, C. v. d. 1996a. Single-view based recognition of faces rotated in depth. In Proceedings, International Workshop on Automatic Face and Gesture Recognition. 176--181.
 
102
 
103
 
104
McKenna, S. and Gong, S. 1998. Recognising moving faces. In Face Recognition: From Theory to Applications, H. Wechsler, P. J. Phillips, V. Bruce, F. F. Soulie, and T. S. Huang, Eds. Springer-Verlag, Berlin, Germany, 578--588.
 
105
 
106
Messer, K., Matas, J., Kittler, J., Luettin, J., and Maitre, G. 1999. XM2VTSDB: The Extended M2VTS Database. In Proceedings, International Conference on Audio- and Video-Based Person Authentication. 72--77.
 
107
Mika, S., Ratsch, G., Weston, J., Scholkopf, B., and Muller, K.-R. 1999. Fisher discriminant analysis with kernels. In Proceedings, IEEE Workshop on Neural Networks for Signal Processing.
 
108
Moghaddam, B., Nastar, C., and Pentland, A. 1996. A Bayesian similarity measure for direct image matching. In Proceedings, International Conference on Pattern Recognition.
 
109
 
110
Moon, H. and Phillips, P. J. 2001. Computational and performance aspects of PCA-based face recognition algorithms. Perception, 30, 301--321.
 
111
 
112
Nefian, A. V. and Hayes III, M. H. 1998. Hidden Markov models for face recognition. In Proceedings, International Conference on Acoustics, Speech and Signal Processing. 2721--2724.
 
113
Okada, K., Steffans, J., Maurer, T., Hong, H., Elagin, E., Neven, H., and Malsburg, C. v. d. 1998. The Bochum/USC Face Recognition System and how it fared in the FERET Phase III Test. In Face Recognition: From Theory to Applications, H. Wechsler, P. J. Phillips, V. Bruce, F. F. Soulie, and T. S. Huang, Eds. Springer-Verlag, Berlin, Germany, 186--205.
 
114
O'Toole, A. J., Roark, D., and Abdi, H. 2002. Recognitizing moving faces. A psychological and neural synthesis. Trends Cogn. Sci. 6, 261--266.
 
115
 
116
 
117
Penev, P. and Atick, J. 1996. Local feature analysis: A general statistical theory for objecct representation. Netw.: Computat. Neural Syst. 7, 477--500.
 
118
Pentland, A., Moghaddam, B., and Starner, T. 1994. View-based and modular eigenspaces for face recognition. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition.
 
119
Perkins, D. 1975. A definition of caricature and recognition. Stud. Anthro. Vis. Commun. 2, 1--24.
 
120
Phillips, P. J., Grother, P. J., Micheals, R. J., Blackburn, D. M., Tabassi, E., and Bone, J. M. 2003. Face recognition vendor test 2002: Evaluation report. NISTIR 6965, 2003. Available online at http://www.frvt.org.
 
121
 
122
Phillips, P. J., McCabe, R. M., and Chellappa, R. 1998. Biometric image processing and recognition. In Proceedings, European Signal Processing Conference.
 
123
 
124
Phillips, P. J., Wechsler, H., Huang, J., and Rauss, P. 1998b. The FERET database and evaluation procedure for face-recognition algorithms. Image Vis. Comput. 16, 295--306.
 
125
 
126
Riklin-Raviv, T. and Shashua, A. 1999. The quotient image: Class based re-rendering and recognition with varying illuminations. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition. 566--571.
 
127
 
128
 
129
 
130
Ruderman, D. L. 1994. The statistics of natural images. Netw.: Comput. Neural Syst. 5, 598--605.
 
131
 
132
 
133
Samaria, F. 1994. Face recognition using hidden markov models. Ph.D. dissertation. University of Cambridge, Cambridge, U.K.
 
134
Samaria, F. and Young, S. 1994. HMM based architecture for face identification. Image Vis. Comput. 12, 537--583.
 
135
Schneiderman, H. and Kanade, T. 2000. Probabilistic modelling of local Appearance and spatial reationships for object recognition. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition. 746--751.
 
136
Sergent, J. 1986. Microgenesis of face perception. In Aspects of Face Processing, H. D. Ellis, M. A. Jeeves, F. Newcombe, and A. Young, Eds. Nijhoff, Dordrecht, The Netherlands.
 
137
 
138
Shepherd, J. W., Davies, G. M., and Ellis, H. D. 1981. Studies of cue saliency. In Perceiving and Remembering Faces, G. M. Davies, H. D. Ellis, and J. W. Shepherd, Eds. Academic Press, London, U.K.
 
139
Shio, A. and Sklansky, J. 1991. Segmentation of people in motion. In Proceedings, IEEE Workshop on Visual Motion. 325--332.
 
140
Sirovich, L. and Kirby, M. 1987. Low-dimensional procedure for the characterization of human face. J. Opt. Soc. Am. 4, 519--524.
 
141
 
142
Strom, J., Jebara, T., Basu, S., and Pentland, A. 1999. Real time tracking and modeling of faces: An EKF-based analysis by synthesis approach. Tech. rep. TR-506, MIT Media Lab, Massachusetts, Institute of Technology, Cambridge, MA.
 
143
 
144
 
145
 
146
Tarr, M. J. and Bulthoff, H. H. 1995. Is human object recognition better described by geon structural descriptions or by multiple views---comment on Biederman and Gerhardstein (1993). J. Exp. Psych.: Hum. Percep. Perf. 21, 71--86.
 
147
 
148
Thompson, P. 1980. Margaret Thatcher---A new illusion. Perception, 9, 483--484.
 
149
Tsai, P. S. and Shah, M. 1994. Shape from shading using linear approximation. Image Vis. Comput. 12, 487--498.
 
150
 
151
Turk, M. and Pentland, A. 1991. Eigenfaces for recognition. J. Cogn. Neurosci. 3, 72--86.
 
152
 
153
 
154
 
155
Viola, P. and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proceedings, IEEE Conference on Computer Vision and Pattern Recognition.
 
156
 
157
Wilder, J. 1994. Face recognition using transform coding of gray scale projection and the neural tree network. In Artificial Neural Networks with Applications in Speech and Vision, R. J. Mammone, Ed. Chapman Hall, New York, NY, 520--536.
 
158
 
159
 
160
Yin, R. K. 1969. Looking at upside-down faces. J. Exp, Psych. 81, 141--151.
 
161
 
162
 
163
 
164
 
165
Zhao, W. and Chellappa, R. 2000. Illumination-insensitive face recognition using symmetric shape-from-shading. In Proceedings, Conference on Computer Vision and Pattern Recognition. 286--293.
 
166
 
167
Zhao, W., Chellappa, R., and Phillips, P. J. 1999. Subspace linear discriminant analysis for face recognition. Tech. rep. CAR-TR-914, Center for Automation Research, University of Maryland, College Park, MD.
 
168

CITED BY  161

Collaborative Colleagues:
W. Zhao: colleagues
R. Chellappa: colleagues
P. J. Phillips: colleagues
A. Rosenfeld: colleagues