Design of an automatic optical system to measure anthropometric hand parameters

The development of human–machine interfaces customized according to the user needs and anatomical features has proved to be a fundamental element in human–computer interaction. The development of reverse engineering [3] and additive manufacturing technologies has provided the tools for designing and fabricating complex personalized components. Dealing with medical devices, specifically, the possibility of tailoring their dimensions according to the patient’s anatomy plays a major role towards the achievement of a higher quality of treatment [4]. Its importance is even higher in case of patients affected by disabilities, as such medical devices need to cope with strict standards in terms of safety, comfort and wearability. This study dealt with the problem of developing an effective and automatic process for the generation of customized hand exoskeleton systems (HESs) for rehabilitation and assistive purposes. The authors have developed a prototypal version of the HES that has been tested on a limited number of patients with satisfying results [1]. In order to test it in a larger number of cases, an automatic process for the generation of the CAD models of the component personalized w.r.t. the patient’s measures must be developed. The components of the HES have been modelled in a parametric CAD modeller so that their configuration can be updated by tweaking a limited number of parameters. This study focuses on the development of an automatic scanner for the acquisition of the anthropometric measures of the hand required to customize the HES components.

The exoskeleton is composed of a frame, placed upon the back of the hand, where motors and controls are fixed, and four finger mechanisms, actuated by a cable transmission. The thumb finger is not actuated by this model of HES. The only components that need to be tailored to the hand shape and dimensions are the components of the mechanical chains responsible for the movement of the fingers. These must reproduce the kinematic of the patient’s hand. A thorough study on the reconstruction of the kinematic of the patient’s hand and on the performance of the proposed HES architecture in mimicking the original movements of the patient is proposed in [6] (Fig. 1).

Originally, the acquisition of anatomical information was achieved with the optical analysis of the trajectory of the index finger executing ab/adduction movements; the data was processed thanks to open-access software able to track reference points [5].

The introduction of a simplified hand model proved to be effective [2] to guide the modelling process thanks to the reduction in the number of the parameters to be measured. According to a Denavit-Hartenberg (DH) notation, the movement of each long finger can be described using 4 DOFs (one flexion–extension movement per joint, and one ab/adduction rotation about the MetaCarpoPhalangeal (MCP) joint axis)—see Fig. 2. In order to calibrate the model on a specific hand, the lengths of the links (L_1 to L_20) need to be measured. This should be done automatically, maximizing the comfort of the patient during the entire procedure, in a limited time and with a maximum allowed error of 2 mm. In order to guarantee the required specifications, a custom optical scanner has been designed and tested; moreover, an automatic algorithm responsible for the computation of the parameters of interest was studied.

The scanner relies on a commercial RGB-D sensor, i.e. Intel^® RealSense™ Depth Camera D415 to acquire 3D data of the patient’s hand in an open pose. The silhouette observed by the sensor should resemble Fig. 2. Physical circular markers, coloured in green, are placed by an operator on the back of the hand to identify the joints of Fig. 2. The markers can be isolated using RGB data, allowing for an automatic computation of the 3D position of their centres and the following evaluation of the parameters of interest.

A prototype of the device, visible in Fig. 3a, was built using 3D printed joints and standard mechanical components.

The scanner is positioned at 300 mm (Z) from the reference plane where the hand lies. This distance was chosen considering the required viewing frustum and the camera intrinsic parameters. Preliminary tests performed acquiring several hands under different conditions highlighted the need for a bright illumination, constant in intensity and colour, in order to correctly identify markers. This way, any interference introduced by the external light conditions is removed. Accordingly, 4 LED panels characterized by a colour of 4000 K and a total 18,000 lm intensity were installed on the scanner.

The software developed by the authors is responsible for: (1) triggering the acquisition of the data; (2) identifying markers (reference points) within the RGB image acquired by the camera; (3) extract corresponding 3D data for the centres of the markers and compute the distance of the links required to build the DH model of the patient’s hand. In steps 2 and 3 the algorithm detects circular markers on the RGB image using colour separation: a green channel threshold is applied to the original image, resulting in a binary image that is processed through a feature extraction algorithm (Hough Transform [7]) to identify circles. The corresponding depth values (i.e. the 3D coordinates of the marker centres) are then retrieved from the depth stream. Since the RGB camera and the depth camera are intrinsically aligned in the D415, the pixel coordinates on the RGB image are directly related to the xyz values in the depth image. Finally, by analyzing the position of the markers in the cartesian system, the algorithm associates corresponding markers and joints (Fig. 3b). The data is then processed by an algorithm [6] that computes the expected trajectories of the hand and the corresponding dimensions of the components of the HES.

The described procedure is an interactive process in which the patient is actively involved under optimal conditions: the method is fast, ergonomic and suitable for those who have motor difficulties and cannot perform complex movements.