A new metrological characterization strategy for 3D multi-camera systems

Thanks to new low-cost technologies introduced in the optical sensor scenario, the deployment of custom 3D acquisition and processing systems, that are tailored to specific applications in their form and technology is becoming an increasingly widespread reality. This category of acquisition systems, made ad-hoc for their purpose of use, finds application in many fields, among others fashion [4], fitness [5] and medicine [2]. Unlike professional technologies (e.g. Romer Absolute Arm, Konica Minolta Range7, Aicon 3D System StereoScan etc.), in these cases, the accuracy of the acquired data is not certified. Furthermore, one aspect introduced by multi-camera systems is the manipulation of raw data in order to merge the several acquisitions into a single object. Typically, the manipulation of data consists of filtering, to eliminate unnecessary portion of points and to eliminate noise, and the alignment in a global reference system with iterative algorithms. These steps can, however, cause an accumulation of error thus compromising the accuracy of the final result. This can hinder the extensive use of multi-camera systems based on low-cost depth sensors if no data is provided on the accuracy with which the reference geometry is replicated.

In order to overcome this problem, the accuracy of the system in reproducing the object to be acquired must be quantitatively measured. To this end, the metrological characterization of this kind of multi-camera system, highly tailored to specific needs, cannot be addressed using standard state-of-the-art techniques, since the few existing standards are not easily adaptable.

In this paper is presented a new strategy for the metrological characterization, and therefore the evaluation of the accuracy, of interactive multi-camera acquisition systems, and is used in the characterization of a new scanner specifically design for the hand-wrist-arm (HWA) district. The procedure, here tested on the arm scanner, was developed to be adapted to custom made devices by making the appropriate modifications.

The lack of reference standards for the characterization of optical systems has led to the development of multiple hypotheses and guidelines and to date no definitive reference standard was defined. The German VDI/VDE [8, 9] has been very active in defining metrology standards especially for coordinate measuring machines (CMMs) with optical measuring capability. In particular, the standard VDI/VDE 2634 Part 2 is the only recognized standard that offers precise guidelines for the acceptance and verification of an optical acquisition system. According to the standard, measurements are made using pre-established artefacts such as spheres, cylinders and flat objects, in pre-established positions. The guidelines indicate the exact size of the artefact used in accordance with the acquisition volume. For each of such objects, both the calibration test and the verification test are uniquely defined. The results are accepted if the error lies within the limits specified by the manufacturer.

The standard VDI/VDE 2634 Part 3 applies to a multiple view scenario. In this case the guidelines dictated by Part 2 are adopted, with some modifications.

The VDI/VDE Part 2 and Part 3, however, are not easily applicable to custom multi-camera systems as the positions of the cameras around the object must follow the normative and the artefact positions and volumes are often too big to be acquired. At the state of the art the above described standard is used as a reference to perform tests for the characterization of optical devices, however, depending on the specific applications of the scanner under examination, reference artefacts are created differently in each work [1, 6, 7]. Furthermore, the procedure used for the characterization always follow the same steps: once the artefact is defined, it is initially scanned with a high-precision device and then with the optical scanner under examination, to create the ground truth reference object.

The proposed strategy for the characterization of multi-camera optical systems foresees an artefact specifically created in compliance with the VDI/VDE guidelines. The artefact, shown in Fig. 1, can be adapted to the system under examination, i.e. to the volume of work and the resolution of the used device.

It consists of different shapes to characterize the scanner in the acquisition of different geometries. The reference object was created with the idea of measuring the errors described in the VDI/VDE normative, i.e. errors of form, size, length and flatness, and with the possibility to be adapted to different types of multi-camera systems, changing the generating parameters of the shapes. The tests on the cubic artefact involve the flatness error (F) which measures the ability to reproduce planar surfaces, performed on the flat face of the cube as the distance from the best-fit plane. The length error (SS_spheres), which measures the ability of the system to reproduce distances, is measured as the difference between the true (l_gt) and measured (l_m) distance of the centers of the two spheres (the hollow one and the regular one):

Finally the two probing errors (E_f and E_s), which measure the ability of the system to model objects in their actual shape and size, are measured on the sphere (E_{f_sphere}) and on the cylinder (E_{f_cylinder}) as the deviation from the best-fit feature and comparing the obtained diameter (D_{c_sphere}, D_{c_cylinder}) with the ground truth diameter (D_gt) which is the same for both shapes:

To sum up, six evaluation parameters are computed: flatness F, sphere spacing SS_spheres, probing error form E_{f_sphere} and E_{f_cylinder} probing error size E_{s_sphere} and E_{s_cylinder}.

Moreover, to give accurate measurements of non-commercial devices created ad-hoc for specific applications, it was decided to repeat the tests to assess the repeatability of the measurement in three contexts:

In addition to the tests described above, the pyramidal convergent steps on the cubic artefacts are used to assess the minimum resolution perceptible by the scanner, intended as the minimum distance that can be reproduced. This test is performed by extracting the best fit plane on each step, when possible, measuring the distances and comparing the measured values with the ground truth. In the following, the proposed method is used to assess the accuracy of a multi-camera acquisition system specifically design for the HWA district.

The scanner dedicated to the acquisition of the HWA district was realized in collaboration with the Meyer Children Hospital of Florence imposing some fundamental requirements: (1) the minimization of implementation costs; (2) the possibility to adapt to a wide range of variability of the size of the HWA district; (3) the paediatric application and consequently the speed of acquisition to avoid artefacts due to movements in the final result; (4) a simple user interaction with the acquisition system, and (5) sufficient accuracy of the result.

The use of commercial optical devices (the Intel RealSense D415) allows to contain costs; the configuration of the eight cameras and the configuration of arm supports (A and B in Fig. 2a) allows to obtain the necessary modularity and finally the acquisition speed is given by the possibility to activate all the sensors at the same time, for a total of ~ 0.5 s per scan. This satisfies requirements 1, 2 and 3.

A simple user interaction with the scanner (requirement 4) is essential in view of the use of the instrument by clinicians, non-experts in 3D modeling and acquisition. To manage the acquisition, a simple and intuitive software was developed, shown in Fig. 2b. The scan is performed by merging the eight point clouds after adequate filtering and noise elimination on each one.

Finally, the accuracy of the acquisition (requirement 5) was assessed using the proposed reference object. Figure 3 shows the dimensions of the artefact, assigned according to the characteristics of the arm scanner. For the case under examination the cube has a diagonal of 90 mm which is the dimension of the volume that will be considered during the operating phase of the scanner. The dimensions of the sphere and the cylinder were chosen according to the analysis of the Intel RealSense D415 depth sensor [3], therefore the diameter of the two shapes is equal to 25.4 mm (as in [3]). Finally, the resolution of the camera was used as reference for the height of the steps which starts from a maximum height of 5 mm and decrease up to 1.25 mm.

Table 1 reports an average of the obtained results, for each parameter, on the three tests (Light, Repositioning and Scanner motion). Table 2 shows the errors on the pyramidal steps.

The objective of this study was to establish a new methodology for the metrological characterization of multi-camera 3D acquisition systems specifically tailored for specific applications. Literature analysis on the characterization of multi-camera systems has highlighted the lack of standards or guidelines that can be easily followed to assess the accuracy of this class of scanners. Based on the most widely used standard for the characterization of optical devices, the VDI/VDE 2634, and on some specific approaches identified in the literature, this work develops a new methodology based on an adaptable artefact containing several geometric shapes. The shapes generating parameters of the artefact must be modified according to the characteristics of the scanner to be examined. The proposed methodology is used to test a scanner dedicated to acquisition of the hand-wrist-arm district. The scanner under examination is composed of eight low cost commercial sensors placed at a fixed distance from the scene to be captured. The anatomy of the arm is obtained by filtering and merging the eight point clouds.

The proposed approach was successfully used to characterize the arm scanner proving to be a key step in the product engineering process of interactive acquisition systems.