State of the art in face recognition

Question

I have looked a little bit into face recognition recently. However I am overwelmed by the number of algorithm that there are.

PCA, LDA, IDA, Gabor Wavelets, AAM ...

Say you wanted to make something like this... Which algorithms would you use or which papers would you read?

Answer 1

I think the first thing you should do is realize that the algorithms you've listed are used at different stages of face recognition.

First, you need to decide on the representation, i. e. the features to use. These could be raw pixels, Gabor filters, some kind of shape descriptors, deformable models, etc.

Then, you typically want to reduce the dimensionality of your features. This is where algorithms like PCA, ICA, or LDA come in, which project data points into a lower dimensional space trying to preserve most of the variance (PCA) or to ensure the optimal separation of points of different categories (LDA).

Then you probably want to train a classifier on your features to distinguish between the faces of different people. There is a plethora of algorithms to choose from here, such as the Nearest Neighbor, Support Vector Machines, Hidden Markov Models, Bayes Nets, etc.

Note that the choice of an algorithm for a particular stage may or may not depend on the algorithms for other stages. For instances, PCA can be used to reduce dimensionality of almost any type of features. On the other hand, it is not immediately obvious how one can use a support vector machine classifier for faces represented by a deformable mesh.

I guess the first thing you should try to do is to define your problem very precisely. Do you want to distinguish between faces of only a few people, such as recognize your family members in photos? Do you want to recognize people from a huge database? Do you have lots of training images for each face, or only a few? Do you want to handle different orientations and lighting conditions?

The answers to these questions determine how complicated your problem is, and will certainly affect your choice of algorithms.

EDIT: Here's a thesis of somebody who tried solving a similar problem. It is from 2002, but IMHO it is a good place to start.