Contour segmentation of the intima, media, and adventitia layers in intracoronary OCT images: application to fully automatic detection of healthy wall regions

The purpose of the proposed segmentation method is to simultaneously extract the contours of the three anatomical interfaces, namely the intima–media (IM), media–adventitia (MA), and adventitia–periadventitia (AP) boundaries (Fig. 1a–c). This operation is performed along the entire circumference of the vessel wall, regardless of the state of the artery in the current image plane (i.e., fully healthy, fully diseased, or partly healthy and diseased). The three contours are expected to be nearly parallel lines located in regions of extremum intensity gradient (i.e., strongly positive or negative values, Fig. 1d). This a priori information is exploited by a dynamic programming approach to extract, in a 4D multi-parametric space, a unique and globally optimal solution that corresponds, in the 2D image plane, to three smooth paths that do not cross each other.

The present segmentation framework shares two aspects with a previous method proposed by our team [28]. Namely, the lumen-intima (LI) interface is segmented using the already existing method as a preprocessing step, and the initial cost map used for the front propagation is identical and defined as the gradient image. The essential aspects of the framework that define the originality of the present work are the following: (i) three anatomical interfaces (IM, MA, and AP) are targeted (as opposed to only two, namely the LI and abluminal plaque boundary), (ii) these boundaries are very close to each other and the IM and AP can easily be mistaken as they are located on ambiguous image regions with similar features, namely a strong negative gradient (as opposed to the LI and abluminal boundaries being unambiguous and located on positive and negative gradient regions, respectively), (iii) these three interfaces are extracted simultaneously (as opposed to iteratively), and (iv) a specific 4D search space is introduced (as opposed to 2D) to allow a unique front propagation scheme to localize the three target interfaces at once. All these points are further detailed below.

The segmentation framework is composed of a set of nine fully automatic operations (i.e., no user interaction is required), as described hereafter.

Forty OCT pullbacks, from 40 patients, were included in this study. To evaluate the performances of the framework on images acquired with different scanners and protocols, data were gathered from two different medical centers, namely the Thoraxcenter of Erasmus MC (Rotterdam, The Netherlands) and the University Hospital of Auvergne (Clermont-Ferrand, France). Twenty patients were included from both medical centers. All patients were suffering from coronary artery disease.

Pullbacks from Erasmus MC were acquired using the C7XR frequency-domain system and the Dragonfly intracoronary imaging catheter (Lightlab/St Jude Medical, Minneapolis, MN, USA), hereafter referred to as “SJM-OCT.” Pullbacks from University Hospital of Auvergne were acquired using the Lunawave coronary imaging console and the Fastview coronary imaging catheter (Terumo Corporation, Ashitaka, Japan), hereafter referred to as “Terumo-OCT.” The characteristics of both OCT scanners are provided in Table 2. All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation, and informed consent was acquired from all patients.

For all parameters, the optimal setting was empirically determined using the training set (\(n=140\)), then the method was applied once to all the images of the testing set (\(n=260\)). The following configuration was used: standard deviation of the Gaussian function \(G_\sigma \), \(\sigma =30~\upmu \text {m}\); weight of the interfaces, \(\omega _1 = 0.2\), \(\omega _2 = 1\), and \(\omega _3 = 1\); minimal gap between the contours, \(g=45~\upmu \text {m}\); number of reachable neighbors, \(2N+1=7\); smoothness coefficient, \(\kappa =0.1\). The number of AdaBoost iterations was equal to 100. The size of the morphological kernels was \(K_E=4\) and \(K_D=9\), respectively.

The segmentation method was successfully applied to all the 400 images to extract the contours of the IM, MA, and AP interfaces. Representative results are provided in Fig. 4. The mean absolute errors (± standard deviation) of the contour segmentation method are displayed in Table 3. It is also relevant to describe the segmentation errors in relation to the thickness of the segmented layers: the mean absolute error to assess the intima–media thickness was \(33 \pm 51~\upmu \text {m}\), with a bias of \(+8~\upmu \text {m}\) and \(95\%\) limits of agreement of \([-109, 125~\upmu \text {m}]\). For the 260 images used to assess the segmentation method, the average reference thickness of the layers was \(131 \pm 67~\upmu \text {m}\) for the intima, \(94 \pm 38~\upmu \text {m}\) for the media, and \(103 \pm 64~\upmu \text {m}\) for the adventitia. For these images, the relative segmentation error, compared to the thickness of the corresponding layer, was \(29 \pm 73~\%\) for the IM interface, \(35 \pm 64~\%\) for the MA interface, \(54 \pm 74~\%\) for the AP interface. When quantifying the thickness of the intima, media, and adventitia layers, the error introduced by the proposed method was very close to the inter- and intra-analyst variability, as displayed in Fig. 5.

Subsequent to the contour extraction, the healthy region classification method was successfully applied to all the 260 images of the testing set to detect healthy regions of the wall from diseased regions. Representative classification results are displayed in Fig. 6. The healthy and diseased labels resulting from the automatic classification method were in good accordance with the manual annotations, as displayed in Fig. 7. For all the 260 analyzed images, the median value (and interquartile range) of the accuracy, sensitivity, and specificity was 0.91 (\([0.75 - 0.98]\)), 0.92 (\([0.71 - 1.00]\)), and 1.00 (\([0.92 - 1.00]\)), respectively. The median Dice coefficient was 0.93, with an interquartile range of \([0.78 - 0.98]\) and an average (±SD) value of \(0.83 \pm 0.25\). More specifically, the median, interquartile range, and average was 0.92, \([0.74 - 0.98]\), and \(0.79 \pm 0.30\) for the SJM-OCT images, and 0.92, \([0.81 - 0.98]\), and \(0.86 \pm 0.19\) for the Terumo-OCT images, respectively. For each of the 260 images of the testing set, the average percentage of healthy regions compared to the entire circumference of the wall, as annotated by the analyst \(\mathcal {A}_1\), was \(69 \pm 31\%\). This is to be compared with the corresponding ratio derived from the automatic method, which was \(61 \pm 29\%\). To assess the usefulness of feature selection, these results were confronted to those obtained when using the full set of 17 features. Similar results were systematically found: the median value (and interquartile range) of the Dice coefficient, accuracy, sensitivity, and specificity were  0.92 (\([0.79 - 0.99]\)), 0.89 (\([0.76 - 0.98]\)), 0.93 (\([0.69 - 1.00]\)), 1.00 (\([0.93 - 1.00]\)), respectively. The healthy region classification method was finally applied to the first 200 frames of a single pullback as a proof of concept to automatically highlight healthy regions, as shown in Fig. 8.

The present framework was implemented in MATLAB (MATLAB 7.14, The MathWorks Inc., Natick, MA, USA, 2011), on a desktop computer with 2.40 GHz processor with 12 Gb RAM. The average time required by the method to process one frame was 5 s to extract the contours of the 3 anatomical interfaces and 2 s to classify healthy and diseased regions. This is to be compared with the time required by manual annotations, corresponding to 180 s and 8 s, respectively.

Extraction of the anatomical interfaces is a critical step in the overall analysis, as the resulting contours are directly used to generate the classification features. Therefore, errors in the segmentation process would have a substantial impact in the subsequent classification process. Nevertheless, the performances of the segmentation method were validated and demonstrated an accuracy similar to the inter- and intra-analyst variability, as displayed in Table 3. Moreover, the segmentation errors were similar between the two cohorts, which contributes to demonstrate the robustness of the method. For both cohorts, the AP interface—generally less well anatomically defined—was more challenging to extract with accuracy, as demonstrated by the higher error rate. Nevertheless, accurate determination of the thickness of the intima–media complex is generally the most relevant clinical application.

When designing the front propagation scheme to simultaneously extract the three contours, the balance between low guidance (i.e., image data only) and high guidance (i.e., image data, and a number of geometrical a priori) is crucial. In the present framework, the resulting contours are subsequently exploited in the classification step to detect healthy and diseased regions. Therefore, a low guidance would result in irregular contours that are often erroneous in challenging image sections corrupted by noise, whereas a high guidance would result in contours that systematically respect a smooth and parallel structure and would compromise the classification process. In the present implementation, the guidance level is intermediate: first, a sub-image I is generated, where the LI interface is a horizontal line (Fig. 2), then the non-horizontal solutions are penalized by a smoothing term in the front propagation scheme (Eq. 4).

The problem of segmenting multiple layers was addressed in several studies, in the field of vascular OCT or intravascular ultrasound (IVUS), as well as retinal OCT as the structure of the human eye also exhibits several interfaces. These methods can be classified in two categories. The first category concerns the iterative extraction of the layers (i.e., one after the other) that cannot perform a simultaneous segmentation of all contours. A combination of region-growing and clustering was used to segment the LI and MA interfaces with a two-step approach [6]. A Viterbi-based dynamic programming implementation was proposed to iteratively segment multiple layers, where the final solutions are obtained by uncrossing all the extracted contours [14]. A fast-marching method was presented to segment the LI and MA contours [3], although a drawback of this approach is the initialization procedure, since two initial contours must be placed around each interface to be segmented. A coarse-to-fine approach, based on a combination of active contours and Markov random fields, and following three successive steps where the large structures are extracted first and the fine structures last, was also introduced [7]. A method based on support vector machine was proposed to segment the LI and MA interfaces, using a set of features based on texture, geometry, edge presence, and shadow [2]. The second category includes methods devised to simultaneously extract multiple contours, similarly to the approach presented in this manuscript. Six intra-retinal layers were segmented using a method based on multi-phase active contours [27]. Nevertheless, this approach is hindered by two limitations: it requires a manual initialization (three points per contour), and convergence relies on multiple iterations that are not always guaranteed to yield the global optimal result. Graph optimization was exploited under the min-flow theory to simultaneously segment the LI and MA interfaces [17], as well as seven retinal layers (that is, 3 layers were first simultaneously segmented, then the remaining 4 also simultaneously) in ophthalmic OCT [16]. The segmentation method proposed here is comparable to these two latter approaches [16, 17], in the sense that a combinatorial analysis is performed to extract the optimal solution corresponding to the simultaneous delineation of several contours, the difference being that the algorithm adopted in the present framework is based on front propagation rather than graph cut.

The accuracy of the segmentation framework may be penalized by the presence of artifacts. The method is not devised to detect the presence of stents and extract their contours and is expected to fail in such cases. Future perspectives will focus on a cascading approach, where stented regions are first detected using a specifically devised method [10, 12, 19], before the tissues in the remaining of the pullback are segmented using the present approach. Thrombi were not present in our dataset, and such images were therefore not evaluated; however, they are expected to yield erroneous contours with irregular shapes (from our experience, we also expect the classification method to provide correct results, as the features that will be derived from the contours would be substantially different from those of the healthy wall structure). It is, however, noteworthy that the method performed well when flushing was not optimal.

The main originality of the proposed classification method is to directly exploit the tentative contours resulting from the previous segmentation step to partition the image in healthy and diseased regions. A collection of local features is extracted from specific image regions delimited by the contours (e.g, image statistics at the contours location). This approach is different from previous studies that used a unique and more global feature such as the backscattering coefficient [26] or the attenuation coefficient [21].

A limitation of the proposed approach is its dependency to a clear layered architecture, with visible intima, media, and adventitia tunicas. Tangential penetration is the artifact that most hinders the method: in case of an eccentric catheter, the sharp angle of incidence of the beam would result in an image where part of the healthy layers are not clearly defined [22]. In such a situation, the classification algorithm may provide inaccurate results, as illustrated in Fig. 6d. The healthy region classification process also failed in some cases where all three human analysts correctly annotated the region. This misclassification concerned two types of failures: false negatives and false positives. False negatives corresponded to healthy images that could not be correctly identified when the structure was not well defined, with blurred and poorly perceptible layers, as depicted in Fig. 6f. False positives corresponded to diseased regions that were wrongly labeled as healthy because of image artifacts or plaque formation. Such cases include fibrotic plaques with a layer-like structure, or calcific plaques showing a well contrasted pattern, as shown Fig. 6c). Misclassification issues could be solved in a future implementation of the AdaBoost model, by specifically integrating a large number of such challenging cases in the learning process. However, it is noteworthy that false positives were less frequent than false negatives. This is represented in Fig. 7 with the specificity being substantially higher than the sensitivity. From a clinical perspective, this would have the following implication: the classification results for regions labeled as healthy are trustworthy, and only regions labeled as diseased—that must be analyzed anyway—should be visually inspected to confirm the classification result. Let us also note that the accuracy (Dice coefficient, accuracy, sensitivity and specificity) is very close to the inter-observer variability (Fig. 7), which contributes to validate the performances of the method.

The proposed classification scheme is binary, namely a healthy or diseased label is applied to different regions of the image. In contrast to several previous studies [1, 20, 21, 24, 26], the present method is not capable of characterizing different tissue types, such as lipid, fibrous, or calcium. To address this limitation, future work will focus on a cascade approach, where the artery is first roughly partitioned in healthy and diseased regions using the present framework, before a second classifier based on different image features is applied only to diseased regions for finer tissue and plaque type characterization. However, the present method can potentially be used as a preprocessing step to guide existing classification approaches in diseased regions.

A contour segmentation method based on dynamic programming was introduced to localize the contours of the intima, media, and adventitia layers in intracoronary OCT images. A unique and globally optimal multi-parametric path was extracted to describe the set of three 2D contours. The tentative contours resulting from the segmentation method were then exploited by a machine learning algorithm to partition the images in healthy and diseased regions. To thoroughly evaluate the method and assess its clinical applicability potential, the framework was trained on 140 images and validated on 260 other images, gathered from two different medical centers with two different OCT scanners. The clinical significance of the present study is twofold. First, segmenting the arterial layers in healthy regions enables accurate and automatic quantification the intima–media thickness, one of the most relevant clinical parameters. Second, fully automatic identification of the diseased regions provides valuable information for the clinician because it indicates the regions that must be further analyzed. Such an approach, allowing a distinction of healthy and diseased regions, is an essential step for plaque characterization. In conclusion, the introduced framework has potential to assist clinicians with quantified information in an automatic, accurate, and reproducible fashion.