Graphical Modeling of the Lower-Limb Joint Motion from the Dynamic Floor Responses Under Footstep Forces

This chapter aims to characterize footstep-induced floor vibration to detect lower-limb joint motion during walking, including flexion and extension duration of the ankle, knee, and hip. Quantitative analysis of the lower-limb joint motion outside the laboratory setting is essential for early clinical detection and rehabilitation tracking of gait-related disorders (e.g., diabetes, Parkinson’s) and mitigating trip and fall risks for older adults. Existing approaches involve manual observation and monitoring devices, such as cameras, wearables, and pressure mats. However, they have operational constraints such as requiring professionally trained staff, direct line of sight, carrying devices that patients may discard, and dense deployment. In this chapter, we model the footstep-induced floor vibration during walking to infer the lower-limb joint motions through a physics-informed graph neural network. Our floor vibration sensing approach is contact-less, wide-ranged, and perceived as more privacy-friendly, allowing continuous gait health monitoring in daily life. The primary research challenge is the indirect relationship between floor vibrations and joint motions. Especially, floor vibrations only capture the interaction between the foot and the floor, making it difficult to infer the rotational characteristics of the ankle, knee, and hip joints above the floor. Moreover, there are complex dependencies between the joint motions of the ankle, knee, and hip, making it more challenging to model their characteristics through floor vibrations. To overcome these challenges, we leverage the insights that (1) critical joint motions (e.g., maximum extension and flexion) exert unique footstep forces in terms of direction and magnitude to the floor and (2) these joint motions have dependencies based on their physiological relationships, which are encoded by the floor structure as vibration signals. We characterize and model the relationship between the floor vibration and the critical joint motions to decode such a relation through a state-of-the-art heterogeneous graphical transformer, which represents the physical interaction between the lower limbs and the floor through their heterogeneous motion and vibration information. To incorporate the complex dependencies between ankle, knee, and hip joints, we model their relationship by spatial edges in the graphical model, allowing information sharing among various joints. The output metric of this approach is the duration of critical motion segments. In this chapter, we focus on the ankle extension time during terminal stance, and ankle, knee, and hip flexion time during swing to assess the risk of trips and falls. We evaluate our approach with 10 participants through a real-world experiment. Our approach achieved an average of 2.87% (0.016s) mean absolute error in extracting the duration of critical joint motion segments on four test subjects, reducing \(\sim 50\%\) of the error from the baseline using a Long Short-Term Memory (LSTM) model (0.028s MAE), leading to a comparable accuracy to the existing sensing approaches in medical practices, such as cameras and wearables.