Study on Different Methods for Recognition of Facial Expressions from the Data Generated by Modern HMDs

One of the non-verbal ways most used by human beings to communicate and convey emotions, often unconsciously, is facial expression. The recognition and tracking of facial expressions are among the main challenges of several companies that intend to enter virtual social environments. Virtual worlds are becoming viable due to the use of head-mounted displays (HMDs) that allow people to interact in these environments with a great deal of realism. However, recognizing and tracking facial expressions on HMDs has been challenging due to optical occlusions. The same device occludes the eyes, which are a fundamental part of facial expressions. In general, the first HMDs did not have cameras or sensors that captured what was happening behind the device. Because of this, previous research has often proposed to work by extracting partial facial features; for example, mouth, cheeks, chin, and so on). However, as of 2021, some of the latest HMDs manufactured have incorporated cameras and/or sensors for face and hand tracking. Among these modern devices, we can mention HTC-Vive-Focus-3 manufactured by HTC, HP-Reverb-G2-Omnicept-Edition manufactured by HP, Meta-Quest-Pro, manufactured by Meta, and Pico-4-Pro manufactured by Pico. This work aims to carry out a study of the main methods of recognition of facial expressions; whether they are traditional, based on deep learning, or hybrid; using as input the complete facial data provided by the new HMD devices that offer cameras and/or sensors for face tracking.