How to create an AR Face with Face Mesh

This involves creating augmented reality (AR) experiences that track a user's face and apply a 3D mesh or model to it in real time. This is often used for various AR applications, such as adding virtual masks. By using the sample face mesh reference in Easy AR you can create your own ARFace with face mesh. Inside Easy AR you can see our first and third examples related to face mesh. Also those face meshes included blendshapes inside Follow the steps given below to create your own AR face.

Step 1: Go to Easy AR -> Scenes folder and load the Face Filter scene by double-clicking on it.

Step 2: Create an Empty game object, name it as you prefer, your AR Face name. For example, we set it as Dog Face which we are trying to add dog ears and nose part mesh applying to the real face in AR.

Step 3: Reset the transform of Dog Face empty game object and set the tag as “Face” from the tag list.

Step 4: Go to Easy AR -> Meshes -> Face Filter folder and drag and drop sample face mesh into Dog Face game object as a child. Double-click to focus the face mesh. (sample face mesh is used only for reference)

Step 5: Then import your face mesh into the Meshes folder and drag and drop that mesh also into the Dog Face empty game object

Step 6: Now assume the sample face mesh is your real face and according to that you have to fix the mask scale, rotation, and position. Once you apply the adjustment fit to the sample mesh you can delete the sample face mesh.

Step 7: The final result only appears on your real mesh like below. For example, the dog face only with parent object.

Step 8: Click the empty parent object (Ex: Dog) and add the AR Face component to that game object.

If you wish to use face mesh with blendshape, you need to add an animator component with relevant avatar reference and ARKitBlendShapeVisualizer script with a relevant Skinned Mesh Renderer reference. Please consider that blendshapes only work with IOS with only supported devices.

Step 9: Now it’s time to convert our parent game object to prefab. In our case Dog Face. Drag and drop Dog Face object into Easy AR -> Prefabs -> Face Filter folder. Then delete Dog Face from the hierarchy window.

Step 10: One last step. Click AR Session Origin gameobject. Under Switch Face script. Click the plus button to add new Face Prefab to list. Under new list item drag and drop our newly created face prefab (Dog Face) to mask prefab. Also if you have a button icon for dog face drag and drop that sprite into the Mask Icon area.

Step 11: It’s all about creating a face mask with mesh. Go to File -> Build Settings. Choose your build platform and build the app.