Match Sketch(Drawing) face photo to digital color photo

Question

I'm going to match the sketch face (drawing photo) in to the color photo. so for the research i want to find out what are the challenges that matching sketch drawing in to color faces. for now i have find out that

1. resolution pixel difference
2. texture difference
3. distance difference
4. and color (not much effect)

I want to know (in technical terms) what are other challenges and what are available OPEN CV and JAVA CV method and algorithms to overcome that challenges?

Here is some example of the sketches and the photos that are known to match them:

Answer 1

This problem is called multi-modal face recognition. There has been a lot of interest in comparing a high quality mugshot (modality 1) to low quality surveillance images (modality 2), another is frontal images to profiles, or pictures to sketches like the OP is interested in. Partial Least Squares (PLS) and Tied Factor Analysis (TFA) have been used for this purpose.

A key difficulty is computing two linear projections from the image in modality 1 (and modality 2) to a space where two points being close means that the individual is the same. This is the key technical step. Here are some papers on this approach:

1. Abhishek Sharma, David W Jacobs : Bypassing Synthesis: PLS for Face Recognition with Pose, Low-Resolution and Sketch. CVPR 2011.
2. S.J.D. Prince, J.H. Elder, J. Warrell, F.M. Felisberti, Tied Factor Analysis for Face Recognition across Large Pose Differences, IEEE Patt. Anal. Mach. Intell, 30(6), 970-984, 2008. Elder is a specialist in this area and has a variety of papers on the topic.
3. B. Klare, Z. Li and A. K. Jain, Matching forensic sketches to mugshot photos, IEEE Pattern Analysis and Machine Intelligence, 29 Sept. 2010.

As you can understand this is an active research area/problem. In terms using OpenCV to overcome the difficulties, let me give you an analogy: you need to build build a house (match sketches to photos) and you're asking how will having a Stanley hammer (OpenCV) will help. Sure, it will probably help. But you'll also need a lot of other resources: wood, time/money, pipes, cable, etc.