Up: No Title
In this paper we have presented an extensive survey of machine
recognition of human faces and a brief review of related
psychological studies. We have considered two types of face recognition
tasks: one from still images and the other from video.
We have categorized the methods used for each type, and discussed their
characteristics and their pros and cons.
In addition to a detailed review of representative work,
we have provided summaries of current developments and of challenging issues.
We have also identified two important
issues in practical face recognition systems: the illumination
problem and the pose problem.
We have categorized proposed methods of
solving these problems and discussed the pros
and cons of these methods.
To emphasize the importance of system evaluation,
three sets of evaluations were described:
FERET, FRVT, and XM2VTS.
Getting started in performing experiments in face recognition is very easy.
The Colorado State University's Evaluation of Face Recognition
Algorithms website, http://www.cs.colostate.edu/evalfacerec/ has an
archive of baseline face recognition algorithms.
Baseline algorithms available are PCA, LDA,
elastic bunch graph matching, and Bayesian Intrapersonal / Extrapersoanl
Image Diffference Classifier.
Source code and scripts for running the algorithms can be downloaded.
The website includes scripts for running the FERET Sep96 evaluation
protocol (the FERET data set needs to obtained from the FERET website).
The baseline algorithms and FERET Sep96 protocol provide a framework
for benchmarking new algorithms.
The scripts can be modified to run different sets of images against
For on-line resources related to face recognition, such as research
papers and databases, see Table .
We give below a concise summary, followed by conclusions,
in the same order as the topics appear in the paper.
- Machine recognition of faces has emerged as an active
research area spanning disciplines such as image processing,
pattern recognition, computer vision, and neural networks.
There are numerous applications of FRT to commercial systems
such as face verification based ATM and access control, as
well as law enforcement applications to video surveillance, etc.
Due to its user-friendly nature, face recognition
will remain a powerful tool in spite of the existence of very
reliable methods of biometric personal identification such as
fingerprint analysis and iris scans.
- Extensive research in psychophysics and the neurosciences
on human recognition of faces is documented in the literature.
We do not feel that machine recognition of faces should
strictly follow what is known about human recognition of faces,
but it is beneficial for engineers who design face recognition
systems to be aware of the relevant findings.
On the other hand, machine systems provide
tools for conducting studies in psychology and neuroscience.
Numerous methods have been proposed for face recognition based on image
Many of these methods have been
successfully applied to the task of face recognition, but they have
advantages and disadvantages. The choice of a method should
be based on the specific requirements of a given task.
For example, the EBGM-based method  has very good
performance, but it requires an image size, e.g., ,
which severely restricts its possible application to video-based
surveillance where the image size of the face area is very small.
On the other hand, the subspace LDA method 
works well for both large and small images, e.g.,
- Recognition of faces from a video sequence
(especially, a surveillance video) is still one of the most
challenging problems in face recognition because video is of
low quality and the images are small.
Often, the subjects of interest are not cooperative, e.g.,
not looking into the camera.
One particular difficulty in these applications is how to
obtain good-quality gallery images.
Nevertheless, video-based face recognition systems
using multiple cues have demonstrated good results in
relatively controlled environments.
- A crucial step in face recognition is the evaluation and
benchmarking of algorithms.
Two of the most important face databases and their associated
evaluation methods are reviewed: the FERET, FRVT, and
The availability of these evaluations has had a
significant impact on progress in the development of face
- Although many face recognition techniques have been proposed
and have shown significant promise, robust face recognition is
There are at least three major challenges:
illumination, pose and recognition in outdoor imagery.
A detailed review of methods proposed to solve these problems
has been presented.
Some basic problems remain to be solved; for example, pose
discrimination is not difficult but accurate pose estimation is hard.
In addition to these two problems, there are other even more difficult
ones, such as recognition of a person from images acquired years apart.
- The impressive face recognition capability of the human perception
system has one limitation: the number and types of faces that can be
Machine systems, on the other hand, can store and potentially
recognize as many people as necessary.
Is it really possible that a machine can be built
that mimics the human perception system
without its limitations on number and types?
To conclude our paper, we present a conjecture about face
recognition based on psychological studies
and lessons learned from designing algorithms.
We conjecture that different mechanisms are involved in human
recognition of familiar and unfamiliar faces.
For example, it is possible that 3D head models are constructed,
by extensive training for familiar faces, but for unfamiliar faces,
multi-view 2D images are stored.
This implies that we have full probability density functions for
familiar faces; while for unfamiliar faces we only have discriminant
Internet resources for research and databases.
Up: No Title
Mon Mar 8 16:07:04 EST 2004