Algorithms for 3D face reconstruction

Question

Project: 3D face reconstruction

Input: 2D frontal face Image Output: 3D face Reconstruction and expression simulation Platform: Matlab or Opencv cpp.

I found out after study that 3D Morphable Models (3DMM) algorithm is a good starting point for my project. But I don't have the Basel Face Model (3D face Database) to implement the algorithm. However, I have downloaded GavabDB from http://gavab.escet.urjc.es/recursos_en.html. Can I develop 3DMM using the GavabDB for 3D face reconstruction from frontal image? After reading the dataset description doc, I observed that Gavab doesn't provide Texture data of the 3D scans; is texture data compulsory?

Does the output quality depend on the 3D database used for modeling?

Answer 1

The original morphable model, as well as Basel Face Model, which is from the same principal researcher, do contain texture as well as geometry.

It is perfectly possible to compute the geometry part of a Morphable Model from 3D geometry data without texture. This gives you a model that captures the statistically most significant variations in the shape.

However, matching a pure-geometry model to a 2D face image is more difficult than matching a model with texture. It essentially requires you to ...

1. identify (or annotate) the locations of some landmark points in your 2D face image
2. optimize for the 3D pose and shape parameters that bring the respective landmarks in the 3D Morphable model close to your annotated landmarks after projection with an unknown camera.

This is an interesting problem, but not a trivial one. For step 2 I would recommend to start by assuming orthographic projection.

Regarding your output quality question: If you have a sufficiently large number of 3D models in your database, the individual scans do not have to be of particularly high quality. Noise will not show up in the principal components of the model you will actually use. Holes in the scans, however, are a problem.

Last but not least, let me shamelessly point you to a paper I wrote some years ago. It does not solve your problem, but it contains a section on fitting a 3D morphable model (geometry only) to a 2D face silhouette extracted from a photo.