Application of lightweight 3D mouth localization in vision-guided assistive feeding systems

Assistive feeding robots aim to improve the quality of life for individuals with physical disabilities, yet existing solutions often suffer from bulky perception modules and low system responsiveness. Accurate and lightweight 3D mouth localization is essential to enable real-time, user-aware feeding interaction. To develop a lightweight and efficient vision-based system for real-time 3D mouth detection and adaptive assistive feeding. A pruned YOLOv5n model was employed for efficient and accurate mouth detection. Using an RGB-D camera and hand–eye calibration, 2D detection results were converted into 3D mouth coordinates within the robot’s base frame. A rule-based control strategy was adopted: feeding is initiated if the mouth remains open for more than 3 s and terminated if the mouth is not detected for 3 s. The feeding process consists of a pre-demonstrated scooping trajectory, followed by smooth motion toward the estimated mouth center. The proposed system achieves real-time performance with a significantly reduced model size (2.86 MB), parameter count (1.33 M), and computation cost (3.1 GFLOPs), compared to the original YOLOv5n baseline (3.65 MB, 1.77 M, 4.2 GFLOPs). Despite this compression, the pruned model maintains high detection accuracy with mAP0.5 of 98.8% and mAP0.5: 0.95 of 61.3%. Experimental evaluations confirm robust 3D mouth localization and stable feeding execution under user head movements. The model runs at approximately 16.2 FPS on an Intel N150 CPU, validating its suitability for real-time deployment on low-power edge devices. The control logic enables natural and safe human–robot interaction during assistive feeding.