Unable to set proper orientation of 3d Model in AR augmented face

Question

I am trying to put earings on the face using ARCore Augmented Face. I am editing the 3D model in blender and adjusting the model according to canonical_face_mesh.fbx .

But my 3d model is rendering above the head.

I also tried making the pivot point of my model same as that of canonical_face_mesh.fbx, (as mentioned in this SO question) but still when running in app it's rendering

Answer 1

The problem was

1. I was not placing custom meshes as children of the asset node as mentioned in doc.

   .

To make the custom mesh or the object a child of the asset node:

i) Load the given canonical face mesh in blender file->import->fbx then browse for your downloaded canonical face mesh.

(Note: Every SDK has its own reference face mesh and they differ; make sure you use correct one).

ii) Then load your custom 3D model.

iii) Scale and align your model according to the canonical face mesh wherever you want it to be.

iv) In object mode press ctrl + shift +p and then click your custom model and then click the bones present in canonical face mesh.A selection menu will appear select object in that menu.

This will make your model, the child of the asset node.

2. Now you need to add Vertex Group in your model. To do so:

i) Select all vertex in your model in edit mode.

ii) Then go to context object data menu. Click the + icon and name your Vertex ( NOSE_TIP, FOREHEAD_RIGHT, or FOREHEAD_LEFT ). This you have to assign according to your model. Mine is Nose Pin so i choose NOSE_TIP. Then Click assign.

Now to cross check you can deselect your model and then click your vertex group and click on select button next to assign button you will notice that your model is selected now.

3. You need to add modifier. To add modifier

i) Click on the modifier option.

ii) Then click add modifier you will see a drop down menu in that select armature.

iii) Name it as asset as shown above select object as asset and select Nose_TIP as shown above.

Now Delete only the facemesh from the asset.

and import your model.

Answer 2

There are three possible problems in your case:

• A pivot point of 3D Earrings Model doesn't match Face Anchor position.

  • Face Anchor and pivot point of your model must meet each other.

• A size/proportions of 3D Model doesn't match size/proportions of canonical face mesh.

• A pivot point in your 3D model is shifted in a wrong direction of Y-axis.

P.S.

Read this post if you need more details about Augmented Faces feature and Face Anchor.

Remember that an orientation of the mesh is different for Sceneform, Unity and Unreal.