A Fast and Reliable Optical 3D Scanning System for Human Arm

The introduction of personalized medicine procedures in the traditional clinical practice calls for the development of innovative measurement systems capable of acquiring different kind of 3D patient-specific data. Traditional diagnostic tools remain an important data source for the design and production of custom-made medical devices, but some limiting aspects (e.g. CT and MRI administration of radiations to patients) can be identified. Accordingly, the development of optical 3D scanners based on passive technologies or non-invasive active ones, could foster the application of high-quality personalized medicine procedures in new areas (e.g. dentistry [2], monitoring of congenital malformations [3]).

One of the areas that could benefit the most from the introduction of optical non-invasive scanners is orthopedics [2, 5], where they could be applied for the design of different classes of devices. Orthopedic appliances rely on customized geometries that fit the patient’s anatomy to be effective. Accordingly, the development of 3D scanners with specially designed features for the acquisition of specific anatomies could improve the treatment of several injuries and pathologies.

In this context, the authors designed a system for the acquisition of the geometry of the arm-wrist-hand district, called Oplà 1.0, deputed to the fabrication of 3D printed casts for the treatment of wrist fractures. The system is composed of a 3D optical scanner and an acquisition software that is responsible for collecting the reference data and processing it. For the interested reader, all the details pertaining to the development and testing of the first version of the system can be found in [1, 5, 6]. The main features sought in the development of the system were: i) fast acquisition speed, required in order to minimize the incidence of human movements, ii) relative low-cost, iii) a maximum acquisition error of 1 mm. The scanner was successfully applied in a clinical trial that led to the production and application of ten 3D printed orthoses, generated using the CAD procedure detailed in [6].

While the overall results achieved with Oplà 1.0 were satisfactory, several features to be enhanced were identified: 1) a faster acquisition, in order to further reduce the effects of micro-movements on the quality of the result; 2) reduce the variability observed in the quality of the acquisition depending on the type of surface; 3) reduce the incidence of environment light condition; 4) increase the comfort for the patience and 5) improve the easiness of use of the entire system, allowing for a future full-autonomous use of this technology for the medical staff. A new version of the system, Oplà 2.0, has been designed from this specification. The present article discusses the design process and performances of the new device. In detail, Sect. 2 presents the renewed hardware, Sect. 3 discusses the revisions made to the software procedure; finally, Sect. 4 presents a first look on the results provided by Oplà 2.0 and the work’s conclusions. 3D scanner design.

The analysis of the limitations observed in the first version of the device and the imposed medical requirements led to some major changes in the hardware of the 3D scanner. The design process started from the identification of new 3D sensors capable of addressing the major flaws of the previous system.

As previously discussed, the accuracy of Oplà 1.0 was considered sufficient; on the other hand, a further reduction of the scanning time guaranteed by the device was necessary. While the SR300 RGB-D Intel® Realsense™ sensors used in Oplà 1.0 allow a fast acquisition rate (up to 30fps), the integration of eight cameras observing a shared area had imposed a sequential activation in order to avoid interference between the sensors. As discussed in [1], the total time required for the sequential activation of each sensor adds to circa 2.5 s. Accordingly, a new model of low-cost 3D camera, i.e. Intel® RealSense™ Depth Camera D415 [7], has been selected to replace the predecessor. A full metrological characterization of the sensor was performed to evaluate its suitability [8]. As reported in Table 1, which summarizes most notable differences between the SR300 and the D415, the new sensor relies on a different technology for the acquisition of 3D data. Specifically, the 3D information is extrapolated from the comparison between two RGB images acquired by a stereo pair of cameras. An infrared laser projector is also used to enhance the differences between the two frames and ease the computation of corresponding points. Accordingly, the new system makes use of a new procedure where all the sensors are triggered together to acquire a single “image” of the scene. The entire acquisition procedure requires 0.5 s to be performed.

The new sensor was also tested in the acquisition of the different kind of surfaces typically encountered in the considered application. Specifically, it was positively tested the behavior of the sensor w.r.t. human skin, which proved to be source of errors for the SR300 [6].

A new disposition was studied to account for the different FOV and operating range characterizing the D415. Oplà 2.0 maintains the same structure of the first version of the scanner, with two circular arrays of 4 sensors each observing the arm. The radius of the new circle was determined according to the operating range of the D415 and was set to 250 mm. The new setting was computed exploiting a custom optimization procedure [9] tuned to identify the set of positions and orientations for the sensors that: i) maximizes the arm surface observed by all sensors; ii) reduces and possibly removes the portion of the patient’s arm that remain hidden from all sensors; iii) guarantees a sufficient amount of overlapping between the point clouds of the sensors for the registration of the data. This was achieved by means of a visibility analysis able to assess the visibility of each point of a digital model of a target (arm) from the optical centers of the sensors. A set of six digital models of arms of different dimensions and features was used to introduce the desired variability in the analysis.

Finally, the structure of the device was redesigned to maximize the comfort of the patient during the acquisition; to this purpose, a relative movement between the structure supporting the sensors and the elbow support point was introduced (Fig. 1a). This way, by means of a manual adjustment, the distance between the two supports (and the position of the sensors) can be tuned to maximize the patient’s comfort and to assure that the region of interest is fully covered by the sensors. In addition, the new medical requirements foresee the possibility to scan the arm in two different configurations (Fig. 1b). Accordingly, some changes in the general structure of the old system are introduced to address this aspect. The shape of the supports (A and B in Fig. 1a) was edited to maximize the patient’s comfort as well as to allow the rotation of the wrist while maintaining a pivot point in B such to maintain the arm principal axis approximately aligned with the central axis of the system; at the same time, the wrist can assume the indicated position.

The use of the Oplà scanning system is facilitated thanks to the development of a simple and intuitive Graphical User Interface (GUI). In this work, a significant effort has been devoted to the design of a renewed GUI to improve the user experience. The main steps of the procedure are confirmed in 2.0 version of the system, but the graphic design was restyled mainly on the basis of the results of usability tests [5]. As illustrated in Fig. 2, software tools are now divided into two main tabs in the main window, one dedicated to the scanning phase and the other to custom orthosis modelling. This separation is intended to give the user some guidance on the order of operations and reduce the possibility of human errors. Moreover, 3D navigation and interaction tools have been moved in the GUI upon the 3D viewer, following the typical convention of 3D modelling environments. In the new setup, the steps were the human intervention is required are clearly presented to the user; these are: 1) selection and removal of all the points of the arm that will not be covered by the orthosis; 2) provide indications on the position and shape of the thumb opening; 3) indicate where the orthosis closing mechanism features are placed [6]. These operations are now grouped under the “orthosis modelling” tab and presented in the correct order to the user.

The core steps of the software have been revised following two principal requirements: 1) patient’s arm must be acquired in two different positions; the modelling mechanism of the orthosis has been modified accordingly considering the two positions, while remaining transparent to the user, who only needs to select whether the acquisition refers to a right or left arm, as shown in Fig. 2. 2) According to the usability tests performed on Oplà 1.0, the procedure for the selection of the points defining the opening for the thumb on the cast was perceived as the most challenging by the medical staff. Previously, the user was asked to define the entire curve for the aperture; such operation was manually difficult for a person not accustomed to CAD software. The new procedure, depicted in Fig. 1c, asks for the independent selection of a number of points that lay on the desired opening contour. The selected points are then automatically processed to generate the profile required by the procedure.

The scanner has been validated by performing an acquisition of a rigid object (i.e. cylinder of Fig. 3). Ground truth data has been obtained using a Romer RS1 3D scanner mounted on a 7520-SI absolute arm by Hexagon Metrology. Except some local errors, Oplà 2.0 performed well within the limits imposed by the accuracy requirements; all errors measured in the reconstruction were in the range [−2.9, 1.5] mm, the mean error of the signed distance is −0.49 mm with a standard deviation of 0.64 mm. The reference diameter of the cylinder was 59.2 mm, while the reconstructed diameter is 58.8 mm. These values have been considered acceptable for the application as restrictive orthopaedical devices are always characterized by clearance between skin and orthosis that can go up to 2 mm.

The system was tested with positive results in the acquisition of 20 arms of potential patients. The composition of the panel group has allowed the validation of the acquisition system on significantly different hand-wrist-arm anatomies. Figure 4 shows a selection of the obtained results. Positive results were obtained throughout the test, with all the acquisition performed successfully, both with a gauze applied on the arm and without it (Fig. 4). As the acquired anatomy varied, the data produced by the sensors was in any circumstance able to produce a full reconstruction of the arm. The results achieved on this aspect were positive, as all the perturbation of the surface that were present in the SR300 acquisition are not present on the acquisitions performed with the new sensor.