Real-Time Modeling of Face Deformation for 3D Head Pose Estimation

We propose a new technique for simultaneously executing face deformation modeling and 3D head pose estimation. Previous methods for estimating 3D head pose require a preliminary training stage for the head model, and cannot start tracking the head pose until this stage is complete. In contrast, our proposed method can acquire and refine a user’s deformable head model in parallel with tracking the user’s head pose. This allows progressive improvement in the accuracy of the estimation of head pose and face deformation.

Our technique consists of three main steps. In the first step we estimate the 3D head pose using a head model that is obtained automatically. The second step finds true positions of feature points by using the resulting poses of the first step. Finally, the basis vectors of face deformation are calculated from the true positions of feature points to acquire a new deformable head model as a linear combination of the basis vectors.

The iteration of the three steps refines the deformable head model, thus improving the accuracy of head pose estimation progressively. The improvement has been successfully demonstrated via experiments.