Creating a visualization system for changes in facial shape and color

During the course of treatment in plastic surgery and dermatology, especially aesthetic surgery, it is essential to regularly observe and record any appearance changes of the body. Physicians typically use an interview, photographs of the patient’s face before and after treatment, and anatomical data including size, position, and color. However, it is difficult to photograph the affected area under the same conditions every time because the range of the photograph varies depending on the angle and distance. In addition, measurements with a measuring tape are similarly difficult to reproduce. In this study, we used an infrared depth sensor and RGB sensor built into a smartphone to measure the shape of the face and the color of the skin in three dimensions. The face shape and color were measured before and after modulation, and the changes in volume and skin pigmentation of the skin color were calculated and visualized. It was then possible to analyze the shape and skin tone independent of the viewing angle and direction of illumination light. This research shows the possibility of monitoring changes in facial shape and skin tone with a depth camera built into a smartphone. This technology is expected to contribute to the development of telemedicine, where patients can measure their faces at home and receive treatment consultations remotely.