Practical Part-Specific Trajectory Optimization for Robot-Guided Inspection via Computed Tomography

Since the introduction of the first commercially available systems at the beginning of this century [1], industrial computed tomography (CT) has become a valuable and popular inspection tool for the automotive sector. One of its most important use cases in this field is the quality assurance for joining methods, such as riveting, screwing, welding and various adhesive-based variants. While in series production one can typically rely on a constant part quality, this is quite different in the field of prototyping. Suppliers often do not use the final version of their tools to produce the individual components. Instead, suppliers themselves run through a process of prototyping that includes constant improvement and evaluation in order to empower their machines (like punching machines or casting moulds). As a consequence, it has to be expected that individual parts change slightly in size or shape during the prototype production phase. Joining parameters have to be adjusted accordingly, in particular for the sensitive tools of cold joining technology. Thus, the previous validation and fine-tuning through a test series can become invalid. Even if only individual joining sites need to be inspected again, the entire car body would—at least for completion—become unusable. This leads to additional time-wise and financial efforts.

The current evaluation methods for mechanical joining sites are on one hand destructive and on the other hand limited with regard to their ability to provide extensive volumetric information with reasonable effort. Common steps involve (1) forcefully removing the joining element with parts of its structural surroundings (2) cutting it through its longitudinal axis of symmetry (3) polishing it (4) treating it with chemicals in order to remove metallic residue and finally (5) dimensioning its characteristic features with the help of a microscope and a suitable software [2].

While the use of CT can in principle partially replace this costly procedure, it is still limited by the maximum size of the test object and sample parts or destructive cutouts often have to be used. Unfortunately, in typical X-ray and CT systems, only small test objects can be inspected as a whole and up until now there was no practically feasible method to scan major body parts or even entire cars with the required resolution. While special applications are nevertheless available for the scan of such large-volume objects [3], these are still reserved for particular use cases and are by no means readily available or time- and cost-efficient for ever day use. However, the requirements imposed on NDT methods concerning cost, personnel and material resources as well as time spent per evaluation, are becoming ever greater, so that new approaches are required to supplement the previously applied methods. As such, robot-assisted CT systems, where either the X-ray source and detector or the inspected object is connected to a robotic manipulator head, has recently turned into a viable option, due to its increased scanning volume and positioning flexibility. While such methods represent an enormous further development in non-destructive testing, their broad application is unfortunately still impeded due to various restrictive factors and long evaluation times are one of the most crucial ones. For this reason, the technology can only be used for individual tests in the laboratory environment at the moment, instead of being applied to series inspection as well.

Since modern reconstruction algorithms can provide volumetric data almost in real-time, the scan itself remains the bottleneck for the examination of multiple parts [4]. The positioning time of the setup can typically also be disregarded [5], which ultimately leaves the duration of the X-ray exposure required to obtain a single projection as the limiting factor. Higher tube current settings and fast read-out electronics allows a reduction of this acquisition time, but, unfortunately, this method is inherently linked to a decreased image quality as well as a higher noise level. While novel tube designs [6, 7], post-processing algorithms [8‐10] or multi-source setups [11] aim to mitigate this effect, reducing the number of projections required for a sufficient outcome appears still to be the most viable option for the majority of application scenarios.

While reducing the number of projections will typically decrease the resulting image quality and may lead to artifacts, it has also been shown that not every projection carries the same degree of information and the selection of the acquisition trajectory influences the achievable outcome of the reconstructed volume [12‐14] and the reliability of metrological analysis building on this result [15‐18] to a high degree. Consequently, focusing only on the acquisition of such valuable projections enables a significantly increased part throughput and lower cost and might finally even lead to the broader availability of new concepts such as inline-CT that improve future mass inspection beyond the capabilities achievable by conventional CT imaging systems. Trajectory optimization methods are especially valuable for robot-guided systems, due to the typically high distance between X-ray source and detector and the inverse-square law of radiography, which turns the acquisition of each separate projection into a costly endeavour. Also, depending on the requirements, it is not always necessary or reasonable to demand that the best possible scan quality is achieved. The motto “as much as necessary, as little as possible” is applied instead, which gives us room to reduce the number of projections even more. This is analogous to the ALARA-principle [19], which is often heeded in medical applications and stands for “As Low As Reasonably Achievable”. In medicine, however, the objective is to minimize the dose to the patient’s body, whereas in industrial applications it is valuable acquisition time itself, that has to be kept at a minimum.

In order for trajectory optimization methods to reach their full potential, CT systems with as many degrees of freedom as possible are required. While robot-based platforms can offer this versatility, recently a growing number of highly flexible experimental CT systems has been engineered [20, 21] and many conventional setups which can offer the adaptiveness that is necessary for optimized and non-standard orbits are already available. Such installations comprise setups for radiotherapy [22, p. 907f] or angiography, mobile C-arms [22, p. 557f] and laminography systems [23], but are by no means restricted to such. On the other hand, many applications that benefit from said approaches are conceivable. Among such are setups with fixed and limited source positions [24] or adjustable fire sequences [25], the dynamic imaging of fast processes [26] and inline-CT for mass inspection as mentioned above. While medical use cases are beyond the scope of this work, such methods can in principle also help, for instance, to reduce artifacts caused by patient movement due to increased imaging speed or to minimize the radiation delivered to the patient [27]. Particular concerning the latter, it has been shown that sparse projection acquisition holds even more potential than the automatic exposure control mechanisms [28], which have already been applied for decades. While it has been demonstrated that the position of valuable acquisition angles remains constant for different tube parameters, trajectory optimization approaches are considered particularly beneficial for low power setups due to their ability to avoid projections related to a poor signal-to-noise ratio (SNR) [29].

Trajectory optimization can be well included in the concept of a digital twin. The current approach is to scan a part using a universally applicable standard trajectory and manually or automatically classify the part as usable or scrap. However, computed tomography is able to generate a volumetric representation of the actual state of a part or assembly, which can be leveraged to a much more holistic part inspection and tracking concept. To start with, geometrical information concerning the object is typically available, e.g., in form of a CAD file. This data can then be used to optimize the trajectory before the actual acquisition and then to conduct the CT scan using this optimized pattern. Additional information, for instance from optical scans, other non-destructive modalities or historical data from previous imaging can then be used to improve the reconstruction or generate new insights, e.g., by training neural networks on the data, automatically classify the object, track occurring geometric deviations over time, apply predictive maintenance if certain trends can be identified, and much more. Finally, this information can be used to improve the manufacturing processes and the part design, which closes the circle that is also referred to as “cyber-physical loop” [30]. In short, instead of only scanning parts, the addition of a pre-scan stage (e.g., trajectory optimization) in combination with providing additional information (e.g., CAD or historical data), a post-scan stage (e.g., machine learning based evaluation), and linking these to each other has the potential to drastically improve the value chain and make use of currently unused, but available data.

Generally, the CT reconstruction problem can be formulated as solvingwhere \(\mathbf {p}\in \mathbb {R}^{m}\) describes the measurements (i.e., all pixel values of all acquired projections), \(\mathbf {f}\in \mathbb {R}^{n}\) is the inspected object (which is equivalent to the “perfectly” reconstructed image) and \(\mathbf {A}\in \mathbb {R}^{m \times n}\) denotes the system matrix that describes the imaging process with respect to the setup geometry [31, p. 205]. Note that in reality \(\mathbf {f}\) is infinite-dimensional and an infinite amount of line integrals would be required to solve Eq. 1 accurately.

Since the system of equations given by Eq. 1 is ill-posed and the inversion of \(\mathbf {A}\) is typically not feasible, we define the reconstruction operator \(\mathcal {Z}\) as an abstract process that yields an estimate \(\tilde{\mathbf {f}}\) for the true image \(\mathbf {f}\) from the projections:Equation 1 describes the imaging process that is typically performed by industrial CT systems, where the number of measurements is high, so that the system is usually overdetermined. In this context, we consider the projection number to be sufficient if the Nyquist-Shannon criterion or its alternative formulation, the Crowther criterion [32], is met. As a practical rule of thumb, the projection count should be on the order of the number of horizontal detector elements or more [31, p. 261], which can easily comprise thousands of separate measurements. The final goal of trajectory optimization as formulated and discussed in this work is to reduce this number to a smaller subset of measurements \(\mathbf {g}\). This selection is described by the measurement matrix \({\varvec{\Phi }}\) asIt shall be noted that \({\varvec{\Phi }}\) cannot be selected completely arbitrarily, as measuring a single pixel element is equivalent to acquiring all elements in the same projection, since the imaging process for conventional CT is based on projections as the smallest unit of obtainable information. Unsurprisingly, the reconstruction of a subset yields a different estimate for the image:We can now formulate the trajectory optimization task as finding an optimal measurement strategy \({\varvec{\Phi }}_{\mathbf{N}, \text {opt}}\) for a predefined number of projections N so thatwhere \({\varvec{\Phi }}_{\mathbf{N}}\) denotes the set of all feasible measurement matrices that include N projections. The norm \(\Vert \cdot \Vert \) is an appropriately selected metric to quantify the achieved image quality, which has to be chosen in accordance with the imaging task and intention, e.g., visual inspection by a human or further automatized procession by algorithms. Unfortunately, it is typically very difficult to quantify image fidelity for the resulting volume, even if all frame conditions are precisely known. Instead, it is more viable to compare it with a reference image that is considered to be optimal and quantify the difference by use of a suitable metric. The most straightforward choice for such a reference would be \(\mathbf {f}\), i.e., the image itself. In practice, however, \(\mathbf {f}\) is usually not known and can also not be readily obtained, since only the surrogate \(\tilde{\mathbf {f}}\) is available by the reconstruction of a sufficiently high number of images by Eq. 4. Consequently, by relying on this next best option we require thatwhere the notation of the norm \(\Vert A,B\Vert \) denotes the image quality of A with respect to the reference B. Several problems appear that prevent us from solving Eq. 7. For one, it is an in general ill-posed and for real data even inconsistent problem due to the high dimensionality of its terms as well as the non-linear properties and the particular high computational burden of the reconstruction operator. Furthermore, the choice of an appropriate image quality metric is difficult and disproportionately influences the achievable results. Since the size of non-zero entries related to \({\varvec{\Phi }}_{\mathbf{N}}\) increases according to \({k}\atopwithdelims (){N}\), with k being the number of projections in \(\mathbf {p}\) (i.e., a vast number of combinations exist), typically no absolute minimum can be found for higher values of N. Last, Eq. 7 inherently assumes that the image quality achieved by all projections (i.e., the reference) is always better than the subset. It has been shown for medical CT that this is typically not the case and, given a time budget for the entire scan, a range of projections can be identified that yields a particularly good image quality due to the interplay of different noise sources and aliasing artifacts [33]. While this study is not directly comparable to our case due the frame condition of arbitrary scan time and the peculiarities of industrial CT systems with different exposure times, photon energies and detector properties, it has been observed that over a broad range of projections basically no improvements are feasible [4]. Furthermore, exceeding a certain number of projections does not further enhance the image quality [34] and certain part-dependent ranges of particularly valuable projection numbers can be identified [35]. It shall be stressed that these restrictions are only relevant if a solution for a high N is required. Since we focus particularly on very low N in this work, the assumption of an inferior image quality is fulfilled and this point therefore omitted for further discussion.

It has been shown that in general at least two part-dependent projection ranges exist where trajectory optimization can be reasonably applied. For low projection numbers, each image is costly and greatly influences the achievable reconstruction result. Consequently, reducing the number of acquired images in this domain can have a high impact on the required scan time and an appropriate choice of measurement positions can lead to clearly visible improvements. However, if a very high number of measurements is conducted, as is usually the case for metrological applications, it can be beneficial to exclude those related to a poor signal-to-noise ratio, as they might increase the overall noise level and deteriorate the achievable image quality [35]. This work explicitly deals only with the optimization problem for low projection counts and it seems reasonable that Eq. 7 cannot be applied for the high-projection problem. Instead, a reference-less metric (i.e., Eq. 5) has to be used and the computational effort is significantly higher (e.g., due to higher computational burden of the reconstruction of many projections), which makes this task much more difficult to solve [35]. While some approaches that aim to solve Eq. 7 approximately have nevertheless been proposed, other methods have also been developed, which will be outlined in Sect. 3.

Nowadays, the planar (partial) circle is still the most commonly applied trajectory design, owing to its mechanical simplicity, which only requires a single degree of freedom. However, this trajectory does not fulfill the well-known Tuy-Smith sufficiency condition, which leads to the formation of cone beam artifacts and several alternative shapes, such as the helix or the circle-and-line trajectory, have been proposed to overcome this limitation. An overview is provided in the literature [36]. Unfortunately, such designs are still inherently sub-optimal, since they do not consider the individual shape of the inspected object and the concept of part-specific trajectories that are tailored towards the particular imaging task appears to hold the potential for further improvements.

Trajectory optimization is a relatively young field of active research with most methods being developed in the last five years and no commercially available system has yet entered the market. However, the fundamental concepts were proposed much earlier as indicated in Table 1. Since the potential of advances related to computational performance has been exhausted, the advent of highly flexible and robot-based CT platforms that offer new opportunities has recently lead to increasing attention.

So far, most methods have focused on medical applications; however, in the industrial field different frame conditions apply, so that these algorithms are only applicable under certain conditions. The requirements are different in terms of accuracy, properties of the objects to be inspected, possible trajectories, machine parameters and much more. Therefore, different approaches might be necessary to allow for trajectory optimization with respect to industrial CT. In the following, a coarse classification of different optimization methods proposed so far will be suggested, while many algorithms can be counted towards more than one category. A detailed discussion of various optimization realizations [35] and trajectory designs [36] can be found in the literature, while only a brief summary will be provided in the following. It shall be stressed that former industrial approaches which address this thematic complex are to the best of our knowledge, merely based on simulations. As consequence, this work is considered to be the first successful part-specific trajectory optimization using an industrial robot-based setup, while it shall be noted that similar investigations have nevertheless already been performed multiple times using medical equipment and continuous trajectories.

To start with, it is well known that X-rays tangential to the object edges are necessary [66] to resolve these structures sufficiently. While this fact can be directly derived from the Fourier-slice theorem, several methods have been developed in order to adjust X-ray source and detector position accordingly, for instance based on the edge-detecting Hough transform [37] or the wavelet transform [38]. Similarly important, long X-ray paths or those corresponding to particularly dense materials should be avoided in order to obtain a good signal-to-noise ratio in the projections. Gross violation of this condition leads to photon starvation [67], which can culminate in the well-known metal artifacts and facilitates the formation of beam hardening artifacts [39]. As a consequence, multiple approaches have been developed to automatically position the object with respect to these restrictions. While the simplest of such methods accumulate the pathlength or attenuation coefficients within the part for certain possible viewing directions and aim to avoid angles associated with high attenuation, more sophisticated versions also exist that are based, for instance, on the expected signal intensity deviation between a mono- and poly-energetic X-ray source [43, 44].

Recently, several methods have been developed that quantify the extent to which a discrete pendant of the Tuy-Smith sufficiency condition is fulfilled for a particular pixel [68] or volume [69] and related optimization approaches attempt to maximize this property. Tuy-complete trajectories do not suffer from unsampled regions in Fourier space and avoid therefore the formation of the famous cone-beam artifacts that can locally lead to a drastically decreased image quality [70]. Such approaches are typically combined with measures to avoid long X-ray paths, since very noisy images can be considered as missing parts in Fourier space [46].

Another group of optimization algorithms heuristically reconstructs various projection combinations that stem from a pool of feasible acquisition poses and quantify the image quality with respect to a predefined criterion. This is equivalent to directly solving Eq. 7 incrementally and by brute force. While being relatively straightforward, these methods inherently lack computational efficiency due to the expensive reconstruction step and are difficult to handle owing to the critical choice of the quality criterion and the high dimensionality, but can nevertheless be facilitated, e.g., by use of a greedy optimization algorithm or the separation into partial trajectories.

Among the most frequently applied approaches is the use of so-called model observers as figure of merit (optimization criterion), which essentially are an image quality criterion that determines how well a certain feature or object can be detected in the acquired data. To accomplish this, frequency-dependent expressions for noise and signal power are calculated and weighted according to one of several available models with respect to the expected feature expressed in Fourier space. In a certain sense, model observers act here as a matched filter discriminators attempting to quantify the detectability of a certain object. While usually a reconstruction would be necessary to obtain the noise and signal information, efficient approximations have been derived [71, 72] that are based solely on the projection data and allow relatively fast computations. Recently, these computations have been supplemented using a machine learning based approach [61, 62] or the Tuy-Smith sufficiency condition mentioned above [60]. This work is also primarily based on a model observer as outlined in Sect. 4. The detailed discussion of further methods is omitted, since these are typically computationally expensive or restricted to very particular applications and the interested reader is referred to the literature for an overview of such [35].

After generation of the input data, the optimization and evaluation method outlined in Sect. 4 was carried out. The entire reconstructed volume consisted of 512 \(\times \) 512 \(\times \) 512 voxels, the VOI was selected to be of 49 \(\times \) 49 \(\times \) 49 voxels and the isotropic voxel size was determined to be 200 \(\upmu \hbox {m}\). Quantitative evaluation results for the optimized trajectory and both reference designs are provided in Fig. 6. For the planar half circle and the low-discrepancy trajectory, several different possible designs are evaluated for each number of projections and the average image quality is reported together with the standard error of the mean. Due to the limited available poses in the horizontal plane, the sample number also varies with respect to the projection number. Since only 39 projections are available, no rotational shift of the planar reference trajectory is feasible anymore for 20 projections or higher (i.e., no standard error is provided beyond that number), while the sample size was fixed to 10 trajectory variations for each projection number for the low-discrepancy trajectory. For the same reason this benchmark can only be provided for up to 36 projections for the planar design, since it is the next lower sampled step size (see Sect. 4.1).

According to both metrics, the planar half-circle trajectory performs clearly worst, while the low-discrepancy trajectory as the second part-independent design already significantly improves the results. However, the proposed optimized trajectory performs best for all projection numbers, even though the further improvement is comparably lower. We believe this is caused by the particular shape of the object. If we neglect the housing entirely, which is reasonable to assume, since it is almost completely X-ray transparent for the applied acceleration voltage, the inner structure in proximity to the screw nut is in relation almost entirely two-dimensional. The highly absorbing electronic components are arranged horizontally around the VOI and cannot be avoided by a planar trajectory. The three-dimensional LDS and optimized designs, however, are able to avoid these highly attenuating structures and achieve a better signal-to-noise ratio in the related projections. This can be seen well in for the optimized trajectory in Fig. 7, where mostly high inclination angles or planar positions are preferred to cover the horizontal planes of the feature and avoid high penetration lengths. It shall be noted that the upper-level circuit board is smaller than the lower one and consist of fewer electronic components, so that its influence onto the location of the most valuable poses is relatively small. We assume that most three-dimensional trajectories are likely to improve the image quality for this particular part with respect to a purely planar and horizontal design. However, while the acquisition positions related to a good signal intensity are only achieved by chance by the low-discrepancy trajectory, the optimized design can identify and include them more efficiently. In addition, projections tangential to the edges of the screw nut are selected systematically and improve the edge contrast as a direct consequence. Example trajectories for 36 projections of all three designs are provided in Fig. 7.

With respect to the RMSE metric, a similar image quality is achieved, for instance, by a planar half-circle using 36 projections and an optimized trajectory using only 16. Similarly, the RMSE for 36 projections is 0.051 for the planar and 0.031 for the optimized design. Based on these numbers, a reduction related to necessary projections by approx. 55 % or, alternatively, an image quality improvement by approx. 40 % has been achieved by the proposed trajectory optimization algorithm compared to the conventionally applied method. In the light that this work only aims to provide a first proof of concept, these results appear very motivating for future work.

Cross section views through the center of the screw nut for the reconstructions with 12 projections and the reference reconstruction using all 1092 projections are provided in Fig. 8. For the reconstruction, the binned projections with a resolution of 620 \(\times \) 744 pixels were considered, which were identical to the ones used by the optimization stage. The relative perceived image quality difference among the three trajectory designs agrees well with the quantitative findings. This also holds for further cross-sections using more projections and different cross sections at the indicated positions in Fig. 6, which are shown in Fig. 9. It shall be stressed again that the quantitative evaluation was restricted to the VOI, for which approximate size and location are given in Fig. 8. In particular the magnified images show that the optimized trajectory clearly outperforms both benchmark designs, since the edges of the screw nut are resolved well, while e.g., the inner diameter can barely be identified in the planar design. However, the overall noise level of the LDS seems to be slightly lower in the remaining volume than the optimized trajectory. This effect was investigated only qualitatively by visual inspection with respect to the cross-sections in Fig. 9 and the difference decreases for higher projection numbers. This finding comes not unexpectedly, since the optimized trajectory aims to include edge-tangential rays in order to improve the contrast around the surface of the screw nut, even if the selected projections have a relatively low signal-noise ratio. In contrast to this, the LDS is evenly spaced, which can lead to more homogeneous image properties. Furthermore, in a related investigation it was found that improvements in a certain part of the object by a trajectory optimization method can lead to a image quality degradation in other regions of the part. The reason for this is that particular projections which are beneficial for the optimized VOI can nevertheless contain other parts of the object associated with a long X-ray pathlength, so that other regions are affected by increased noise levels. In return, views that show the feature of one VOI very well might be rated poorly by the algorithm, since it results in a low SNR for other VOIs so that projections that are only mediocre for all VOIs are selected in the end as compromise. Because of this effect it was further concluded that as few VOIs as possible (ideally only one as in this case) should be used to increase the obtainable image quality gain [35]. Nevertheless, it shall be noted that the use of several VOIs can be reasonable in some scenarios as performed in earlier work [63]. In such multi-location cases the optimization criterion needs to be adapted to weight the detectability of several features and combine them into a single metric; several possibilities to design this cost function have been suggested [56].

Compared to the quantitative results, the reconstructed images provide further valuable information. Slight distortions (e.g., at the radial surface of the screw nut) are visible over the entire volume, which is even the case for the reference reconstruction. This is most likely caused by geometric deviations between the actual and measured positions of both robots during the scan. While typically such differences are negligible for conventional industrial CT setups, this might not necessarily be the case for robot-based systems, where the actual position of the robots should be measured during the acquisition and taken into account [85]. Unfortunately, no actual positions of the setup were readily available due to the practical limitations of the control software and we were forced to assume the acquisition pattern described in Sect. 4.3 as ground truth. Furthermore, for this investigation no software-based compensation algorithms that could mitigate the effect of this discrepancy were applied. In addition, no other image-enhancing postprocessing methods were used, for instance to compensate for the effect of beam hardening or ring artifacts, and the implementation of such is left for future work.

The inferior image quality for the planar half-circle trajectory becomes once again apparent for the higher-absorbing electronic components at the right part of the shown volumes (arrow a). While these are not visible at all for this design, the low-discrepancy and the optimized trajectory provide significantly better results. A similar effect can be observed at the lower part (arrow b), where the location of the lowermost component appears to be on the right instead of left relative to the higher one and at arrow c, where a round object appears distorted. These findings can turn problematic, since they are likely to lead to false conclusions in case of practical inspection scenarios. Interestingly, the round capacitor parts (arrow d) can be identified in the cross sections in case of the low-discrepancy and optimized trajectory design. Since these are relatively X-ray transparent, resolving them is considered to be a difficult imaging task for a trajectory consisting of only 12 projections. It shall further be noted that the image quality for all provided trajectories would be insufficient if a reconstruction of the backprojection type was used, as it is usually performed in industrial practice. In this case the volume would provide basically zero informative value for each design if only 12 projections were used. The missing edge (arrow e) is no artifact but related to the restricted reconstruction volume that ends at this part of the object. The cross-section for the optimized trajectory shows that the algorithm achieved acquiring projections tangential to the surface of the screw nut. This is illustrated well by the diagonal streak (arrow f), which is caused by X-rays passing almost unhindered through the object in proximity to highly absorbing components. This artifact is aligned in source detector direction for a particular projection, which was acquired as tangentially to the screw nut edge as possible for the given 5\(^\circ \) discretization. Again, we want to stress that the optimization task was to improve the image quality solely within the VOI and enhancements in other parts of the volume—though being desirable as such—are only a side effect and cannot be guaranteed by the proposed method.

Calculating the optimization criterion for a single projection takes around 7 ms on a system with two Xeon E5-2640 processors and an Nvidia GeForce Titan X GTX graphics card, with much space for further improvements. In case of this work with 1092 projections, the runtime of a single optimization loop is on the order of 8 seconds so that the optimization for 150 projections can currently be conducted in approx. 20 min if a suitable regularization parameter for the targeted projection number is known and no evaluation reconstructions for progress monitoring are performed. The optimization towards fewer projections takes almost linearly less time. The calculation of the Fisher information matrices and further preprocessing steps is computed in a few minutes. In its current implementation, most calculations of the actual optimization stage (including all Fourier transforms) are performed on the CPU, while the reconstructions for the evaluation are computed primarily on the GPU. If a continuous evaluation is required, this part of the framework can be performed in parallel to the actual trajectory optimization. Furthermore, while many computations are already parallelized, the optimization criterion is built for each projection sequentially in every optimization run, which could also be improved in future work. In addition, further performance gains, e.g., via distributed or cloud computing are feasible, which would also fit well in an inspection environment that implements concepts of NDT 4.0.

For the course of this work, the signal template was derived from a reconstruction using all available projections, which has to be performed before the optimization can take place. However, it can theoretically be avoided by providing the geometry directly, e.g., via a CAD file. While we optimized directly on the previously acquired projections, which was naturally very time-consuming due to the required imaging process, this step can alternatively be substituted by a simulation framework as in our former work [58, 59]. As we were able to achieve good results in these studies for relatively simple simulation parameters and without taking scatter or advanced noise models into account, it seems reasonable to assume that such computationally efficient approaches can be applied here as well. Simulation-driven methods hold several further benefits in addition. For instance, it is possible to include further parameters, like X-ray tube settings, beam filters, and so on into the framework and optimize these properties as well. Also, while a coarse 5\(^{\circ }\) sampling was used for this work, it is possible to refine the optimized positions using simulation-driven approaches. For example, by iteratively computing new projections in the vicinity of the optimized poses and including them in further optimization steps. Alternatively, if already acquired images are used for the optimization, these can be binned for the optimization stage as done here to reduce the computation time. Since optimization and reconstruction are decoupled, it is possible to reconstruct the optimized trajectory, which was computed using binned input projects, using the unbinned data to achieve full resolution.

We believe that all these steps can easily be automatized, so that an entire trajectory optimization can become clearly feasible in a practically reasonable time (e.g., half an hour). However, an appropriate choice of the regularization parameter needs to be found first, which can be a quite time-consuming task. Alternatively, the proposed brute force method can be applied for the cost of a higher computational burden and runtime due to multiple repetitions of the optimization and the reconstructions and reference evaluations required to identify a suitable parameter set.

One of the greatest problems at the moment is this search for a suitable regularization parameter \(\beta \) for a particular projection number. While a similar regularization strength typically results in the same trajectory, optimization solutions with similar projection number use the same or similar regularization parameters and slight deviations from a suitable value have only limited impact onto the performance of the optimized trajectory, some “incorrect” values can lead to very poor results, which can occasionally be outperformed by both reference trajectories. As result of this, the parameter sweep is definitely required to assure stable results, since appropriate regularization parameters depend on the inspected scenario (e.g., the object, material, projection number). While, unfortunately, this search increases the computational effort of the algorithm linearly, the authors are not aware of any more sophisticated methods to determine a suitable parameter at the moment. Due to these reasons, we plan to remove the model observer based optimization entirely in future work and replace it via a different figure of merit. Such a metric should comprise a SNR-weighting (e.g., analogous to the reconstruction quality based methods of Table 1 and an object-dependent weighting step as illustrated in Fig. 2. We present such a method in subsequent work [35].

An iterative reconstruction approach is strongly recommended for low projection numbers as used in this work. While reconstructing non-uniform or three-dimensional trajectories is efficiently feasible in practice—for instance via the “special reconstruction method” pipeline provided by the CERA library—a clearly better image quality is obtained by iterative approaches. Even though such methods typically come with a higher computational burden, modern workstations are able to provide results sufficiently fast, in particular since fewer projections need to be processed for an optimized trajectory than usually. The reconstruction of fewer than 150 projections with respect to the selected parameters of this work was possible in few seconds or minutes. Since the benefits of trajectory optimization are relativized for higher (at least until a very high projection number is reached, see discussion in Sect. 2) projection numbers [58, 59] and the unproductive dwell time to reach the optimized positions is disproportionally increased due to the complicated mechanical realization, it seems reasonable to apply such methods only for few views, which aids to keep the effort for computing the trajectory and the reconstruction sufficiently low. Furthermore, the scan time itself is still considered the major bottleneck, while the reconstruction can be parallelized and calculation power is typically cheaper than measurement time.

Also, a suitable and fast registration is paramount to properly align the optimized trajectory result to the real world coordinate system before the scan is performed. Assume for example, an optimized trajectory that was calculated with respect to its CAD file and is therefore defined in relation to the CAD coordinate system. However, the real object in the CT system is most probably tilted or positioned differently, so that the trajectory has to be re-defined with respect to the new coordinate system. Otherwise, the individual robots could crash with the scan object causing material damage on both sides. Another scenario that can occur independent of possible crashes, when the registration of reference and object is not sufficient, is the deterioration of the reconstructed volume’s quality due to artifacts caused by incorrect system geometry parameters or discrepancies with the proposed optimized trajectory. However—considering the latter—we believe that the method is tolerant to a certain degree, since the projections are already spread out in coarse 5\(^{\circ }\) pattern in this work, so that errors in a similar range should be acceptable and not lead to relevant disadvantages. It appears likely that for larger deviations the result of the optimized trajectory will tend towards the performance of the LDS trajectory but a detailed sensitivity analysis has not been carried out yet and is planned to be addressed in future work. We want to stress that the accurate position of source and detector is primarily relevant for the reconstruction (which is in general very sensitive to such errors as mentioned in Sect. 5) and to avoid collisions. However, these cases are highly dependent on the scan object and the resulting optimized trajectory. A very nuanced trajectory causes the algorithm to break down if the object is relatively shifted and/or rotated. A trajectory of either robot that passes the object very closely can potentially cause the previously mentioned crash if not properly registered and compensated. For a sufficient registration between object and reference, first the object to be scanned has to be identified by the system and transferred into the real world coordinate system. To do so, an optical 3D scanner poses a viable option to detect the object, which is then fitted against the reference in order to adjust the optimized trajectory. Similar methods based on spherical markers are widely used in image-guided surgery [86]. With the 3D scan method, an additional reference object with known geometry, position, and orientation is captured together with the inspected object. From the optical scan the relative position between both objects can then be derived and the registration is adjusted accordingly. Another possible solution for the registration is projection-based. Here, a few radiographic “scout-projections” could be used to infer the object’s position and orientation. Both approaches—optical 3D scans and radiographic scout-projections [87]—have the potential to be performed by a specifically trained neural network. Furthermore, optical scans can be used in the reconstruction stage to improve the obtainable image quality and to augment the digital twin by additional surface data. However, with precise CAD data and a repeatable fixture system (see Sect. 4.3), the methods mentioned can be avoided. For future industrial practice this could be the preferred option, since it is easier to integrate in productive workflows and avoids the need for additional hardware and scanning steps.

The proposed optimized trajectory for this work was not restricted to follow a smooth path, but was allowed to consist of distinct, unconnected positions in space. While we believe that the side condition of a continuous curve most likely decreases the potential of the suggested method, the positioning time necessary to reach these positions will certainly be lower and allow therefore the acquisition of more projections in the same time. Further advantages comprise an easier mechanical implementation and a reduced parameter space, so that most proposed optimization methods have restricted themselves already to smooth paths and connected splines. In theory, such trajectories might also enable the use of continuous image acquisition with the option of removing their inherent motion blur (e.g., by artificial intelligence methods) instead of distinct imaging poses.

As illustrated in Fig. 2, not all inspected parts bear the identical optimization potential as, e.g., round objects and smooth shapes may be located more widely spread in the Fourier domain and consequently more projections are required to resolve these regions, which has also been demonstrated in previous work [58]. Fortunately, unlike for medical imaging, most industrial parts feature an irregular shape and few distinct planes, so that we believe that the trajectory for many relevant objects can nevertheless be optimized well. Furthermore, a method was proposed that is able to predict how much the trajectory for a particular part can be optimized to obtain improvements [35], so that it can be determined if an optimization approach is worthwhile before actually applying it. While a drawback of this study is the investigation of only a single item, it has been shown via simulations that trajectory optimization is nevertheless feasible for many, greatly different objects and the proposed trajectory always performed similarly well or better than conventional approaches for any projection number [35, 58].

Another restriction can be identified in very small or almost two-dimensional defects, such as cracks, delaminations or interface layers within the part. Such flaws are often just visible from few acquisition directions and it is at least questionable if such will appear in an optimized trajectory if not properly taken into account. Since previous investigations have shown that such features are likely to disappear [88] if the required projections are not included, profound knowledge about the inspected object and occurring errors is required for reliable use of such methods. However, if it can be ensured that such kinds of flaws do not occur, further post-processing algorithms, like advanced filtering or sinogram interpolation methods [89‐91], become an option to further reduce view aliasing artifacts caused by very low projection numbers.

A solid understanding of the inspected part is also necessary to define the signal template \(\mathbf {W}_{{\textbf {Task}}}\) in the first place. Often the location, type or shape of potential flaws cannot easily be determined and expert knowledge and experience will be required to define relevant parts of the object. This is in particular important, since the optimization of one VOI can lead to a decreased image quality in other regions of the inspected part, so that a proper task definition appears critical. However—analogous to the discussion of a suitable registration above—we believe that some deviations are acceptable and plan to address the question of only partially correct task definitions in future work. In addition, the selection of a suitable termination criterion of the optimization framework (i.e., “how many projections are enough?”) needs to be performed with respect to the achieved probability of detection before the optimized trajectory is applied in a larger scale. It shall be noted that the related question of the ideal number of projections for metrological approaches has been investigated by several studies [4, 34, 92]. If geometries are produced via generative methods, like additive manufacturing, it is imaginable to relate production parameters to defects appearing in the CT scan and use this information to improve the process. This communication among several systems within the value creation chain represents an important core property of future smart factories, which can be augmented by the use of artificial intelligence and further concepts of Industry 4.0 [93]. Similarly, information about the part stemming from its digital twin, a priori knowledge or other modalities can be used to improve the reconstruction or generate a deeper insight into the involved processes.

A relevant question for the efficient use of trajectory optimization methods in practice is the definition of a suitable termination criterion as mentioned above. The easiest method is to evaluate the image quality at predefined iterations and stop once a certain threshold that corresponds to a sufficient fidelity (e.g., at a SSIM of 0.5) is reached. Alternatively, one could stop the optimization process once the image quality gain for each additional projection becomes small (in relative or absolute numbers), which equals a plateau in the curves shown in Fig. 6. Both approaches require a profound understanding of the part and the required image quality and rely on frequent use of the evaluation stage, which increases the computational burden. A practically highly relevant approach would be to link the image quality measures used in this work (i.e., SSIM and RMSE) or similar quantities to the probability of detection (POD) for given defects. However, we consider this connection to be currently a very challenging task, in particular since very few work has been performed in this domain and a multitude of influence factors being relevant. Last, as mentioned above, a computationally efficient prediction algorithm has been developed, which is able to predict the optimization potential of a given geometry. This method will be published as a separate work [35], where the mentioned termination criteria are also discussed in more detail. Given some experience with different geometries, one could use this method (and maybe further input parameters) to predict the projection number that is necessary to achieve a certain image quality. This is practically interesting, since the number of required images (which would then be used as termination criterion) could be determined before the actual optimization takes place.

As mentioned above, knowing of specific defects before the scan, and linking their characteristics to image quality measures should improve the performance and output quality of an optimization algorithm similar to the one proposed in this work. Continuing this idea and introducing machine learning to this concept could potentially pose an elegant way of automatizing the linking of image quality measure and defect characteristics. In use-cases, where certain defects are already visible in a simple radiographic projection, a machine learning approach could be trained in a fashion that allows segmentation of very obvious defects in real time in 2D; see for example the YOLO network with a processing speed of 45 frames per second [94]. Here, attention should also be focused on determining beneficial views in 2D radiographic projections, as was shown in [95]. The main limiting factor, as is the case with most machine learning endeavors, is the amount and the quality of available data (see [96, 97] for the case of porosities). It directly impacts the quality of the network’s output and, in the case of far too little data, can even prevent the network from converging at all [98]. It shall also be noted that, even without the proposed optimization method, 2D projection-based sample testing appears to be still a viable option, which is, however, highly dependent on the exact cycle time and the object that has to be scanned. For instance, every n-th object could be automatically removed from the assembly line, scanned next to it and then returned in the case that no indication of defects was found by the 2D radiographic analysis. In the opposite case (a potential defect was detected) the network could then link the characteristics of the found defects to the image quality measures and therefore impact the number of projections as well as directions that are essential to detect the previously found defects in a 3D scan that was produced with an optimized trajectory.

Typical cycle times for modern assembly lines in the automotive industry can be as short as 60 s. In rare cases, like custom manufacturing, they can span over about half an hour, which is, however, the clear exception. This economically imposed temporal requirement represents—at first—a disqualifying limitation for efficient robot-based inline computed tomography. With that in mind, as long as there is no technological development regarding the object-dependent long imaging times per projection, a 100 % inline testing via this method might remain unfeasible in the automotive industry. The most problematic factor at the moment is the accumulation of exposure times that are needed for every projection in order to reconstruct and evaluate the volume with sufficient quality.

In this work we proposed, based on previous approaches [36, 58], a trajectory optimization framework which is capable of significantly reducing typical CT scan times by acquiring fewer projections than required by conventional methods. The functionality presented in this work can also be well incorporated in systems and workflows that are associated with the currently growing fields of industry 4.0 and machine learning. In the former case, a practical implementation could consist of an inline-CT system that (1) recognizes the object which is to be scanned, for example via a QR-code or an RFID device, (2) automatically pulls the respective CAD file or reconstruction from a previously scanned identical object from a server, (3) starts the presented trajectory-optimization technique and, finally, (4) performs the CT scan. An industry 4.0 environment is made up of a network of machines and devices which are digitally connected and can share data with each other. Limiting factors for this technology are therefore the network’s up- and download speeds as well as the available machine-to-machine interfaces.

This example illustrates, that more than only scan speed enhancement is necessary to enable a practical applicable inline-CT method for the mass inspection of separate parts. For one, the sheer quantity of generated data needs to be evaluated in real-time, which should be automated (e.g., by neural networks) to be able to provide practical value [99]. Furthermore, at least a fraction of this data needs to be stored to allow long-term process control. Recently, some compression methods have been proposed to facilitate this endeavour [100]. A higher degree of automation is required for many more parts of the pipeline to finally allow reasonable use of the digital twin concept, for instance practically feasible adjustments of the acquisition trajectory in a shop floor environment, part tracking mechanism and—probably most important—the use of the generated insights, e.g., in form of a feedback loop connecting detected flaws to the related production machines and procedures and thereby enabling the automated adaption of the respective process parameters.

While the investigation conducted here aims towards providing a first proof of concept and to demonstrate the practical potential of the method, significantly more work will be necessary to obtain a stable and universally applicable framework that can be employed for industrial mass inspection. However, we believe that this technique can be a remedy for the current limitations concerning scan times and might represent a first step towards a wider and more economical use of robot-based CT systems and a more sustainable method of non-destructive testing.