A review on lung boundary detection in chest X-rays

The algorithms in this group set sequential steps and heuristic assumptions to locate the lung region. They are generally used as initialization of more robust segmentation algorithms. For example, in [23], researchers propose using level sets which combine the global statistics, prior shape, and edge information. The level set is initialized at a seed mask which is computed using rule-based steps such as thresholding, morphology, and connected component analysis. In [24], lung region is extracted using Euler number method and refined through morphological operations. In [25], before applying fuzzy C-means clustering algorithms, sequential steps are applied such as Gaussian derivative filtering, thresholding, border cleaning, noise removal, and clavicle elimination. Several earlier approaches in this group are mentioned in [11] and in [26]. The algorithms in this group have an easier implementation. However, the output boundaries obtained with this algorithms may not be optimal due to sequential steps, e.g., applying morphological operations, resulting in cascaded accumulation of errors.

In these algorithms, each pixel is labeled as a lung or a non-lung pixel using a classifier (e.g., support vector machines, neural networks) that is trained with example CXRs and their corresponding lung masks. For example in [11], the proposed method employs multiscale filter bank of Gaussian derivatives and k-nearest neighbor (k-NN) classifier. The limitation of the conventional classification approaches is the lack of model constraint to keep the boundary in the expected lung shape. The classifier might fail at segmenting lung with lesions or other pathology without a reference model due to the difference in imaging characteristics in these areas.

The algorithms in this group use both low level appearance and shape priors. The earliest model-based algorithms are Active Shape Model (ASM) [27] and Active Appearance Model (AAM) [28] in which the shape is modeled with the distribution of landmark points on training images and is fitted to the test image by adjusting the distribution parameters. They are applied to lung region detection in [11, 29]. Despite their broad applicability due to shape flexibility, ASM and AAM do not perform well at widely varying shapes, require proper initialization for a successful convergence, and output boundary strongly rely on tuning the parameters. For lung region segmentation, the algorithm can get trapped at local minima due to strong rib cage and clavicle bone edges.

Several studies have been proposed as an extension of ASM and AAM to cope with their disadvantages by incorporating prior shape statistics in objective functions [30‐33]. For example, in [22] the lung boundary is characterized by a scale-invariant feature transform, and ASM is constrained by statistics collected from previous CXRs of same and other patient’s CXRs. In [34], a shape particle filtering approach is used to prevent getting trapped at a local minimum. In [35], global edge and region forces are added as additional terms to the objective function to reach the global minimum.

In these methods, the best parts of the schemes are combined to produce a better approach to overcome the challenges of lung boundary detection. For instance, in [11], deformable models and pixel classification approach are combined with majority voting, and a better boundary detection performance is reported. In [36], an atlas-based approach is used in which the model atlases are registered to the patient CXR using the SIFT-flow algorithm [37] and combined with graph cut boundary detection.

With advances in GPU technology, computer vision systems designed with deep neural networks trained on a massive amount of data have been shown to produce more accurate results than conventional approaches. In deep neural networks, input data is processed through deep convolutional layers, which learn feature representation hierarchically, starting from low-level to more abstract representations. In particular, convolutional neural networks (CNNs) have received considerable attention in image analysis problems, since they preserve the spatial relationship between the image pixels.

Despite the popularity of deep learning algorithms in medical imaging, only a few studies have been reported in the literature for lung boundary detection in CXRs. A recent study uses semantic segmentation approach [38] in which the input is a CXR image and output is a map indicating lung region probability of each pixel. In [39], researchers proposed using fully convolutional networks (FCN) [40] for segmenting lung, clavicle and heart regions. FCN is an encoder-decoder architecture. The encoder models the semantic information in the image; the decoder recovers the location information which is lost during the pooling process and produces a map contains lung region probability of each pixel. FCN produces rough map due to its basic decoder architecture. Therefore, researchers [39] applied architectural modifications by adding a drop out layer after every convolutional layer, by re-ordering the feature maps and by replacing pooling layers with convolutional layers. In [41] SegNet [42], performance is investigated for lung region detection in CXRs. SegNet is a semantic segmentation approach which has similar encoder-decoder architecture as in FCN. However, each deconvolutional layer in the decoder stage corresponds to a convolutional layer at the same level; upsampling is performed based on the pooling indices in the corresponding encoder stage which provides more accurate segmentation map compared to FCN. In [43], researchers proposed using generative adversarial network (GAN) [44] for lung boundary detection in CXRs. GANs consist of two networks: a generator and a discriminator. For segmentation problem, the generator produces artificial lung masks using manually delineated lung regions; the discriminator produces probability if the mask is synthetic or it is from ground-truth mask set. Based on the probability, the discriminator guides the generator to generate masks more similar to the ground-truth masks.

All proposed DNN-based approaches perform as good as inter-observer performance for lung regions detection. The advantages and disadvantages of algorithms (as in groups) are summarized in Table 1. Quantitative comparisons of lung boundary detection algorithms are given in Table 2.

In addition to intra-thoracic pathology, lungs appearance is often distorted by external objects that may be present due to poor quality assurance, e.g., jewelry, buttons [61, 62], body piercings, or external elements due to patient age, e.g., cardiac pacemaker, tubes [61]. Examples of some of these distortions are shown in Fig. 1c–e. Such a distorted appearance can distract the algorithm and lead to inaccurate segmentation. Although there are articles recognize the importance of such distortions [61, 63, 64], to our knowledge, there is not any methodical inclusion of these challenges into a lung segmentation algorithm that is robust to such real-world image artifacts.

A significant problem in developing robust lung segmentation algorithms is patient positioning. Most algorithms found in the literature assume that the patient is upright with appropriately inflated lungs and properly positioned without rotation. However, real-world CXR images, particularly those from hospital settings or of physically disabled subjects, have these problems. Subject positioning lead to deformed lung appearance, thus adversely impact the lung segmentation stage and subsequent decision-support algorithms. Some articles in the literature [25, 63, 64] attempt to correct planar rotation, which is important for image analysis, but we do not find articles that detect patient rotation to aid in improved imaging quality assurance.

One of the main challenges in medical image analysis is access to suitable datasets. It is usually difficult to avail of appropriately sized de-identified data that can be used for algorithm development. Further, curated datasets are generally clean and may not reflect normal variations in image acquisition characteristics (e.g., device, subject positioning, exposure, resolution), appropriate distribution of diseases that reflect their prevalence, adequate distribution among various age groups, or reflect the gender diversity. Further, the images are modified such that they are windowed or leveled for human visual analysis. They are rarely accompanied with full clinical reports or at least pertinent sections of the reports such as the radiologists’ impressions and readings. Finally, image datasets are often not in the original DICOM format as acquired at the clinical sites. It is desirable that datasets be available that address the above and include expert delineation of important organs and zonal markup data indicating the location of disease. All of these characteristics are partially addressed in the datasets identified below, but each lacks some key element that could hamper advances in the field.

In order to train and evaluate the system performance of automated lung boundary detection algorithms, reference lung boundaries are needed. However, expert delineation which is a task that is unnatural for domain experts, i.e., radiologists, is cumbersome, slow, and prone-to-error. User-friendly interactive annotation toolboxes such as Firefly [80, 81] or LabelMe [82] may ease the delineation and speed up the process. For instance, in [36], reference lung boundaries are manually delineated by an expert by clicking points along the lung border (by considering the lung anatomy) through Firefly which is web-based interactive labeling tool [80, 81].

Although reference boundaries are used for training and evaluation, expert delineation introduces high inter- and intra-observer variabilities because of the subjective nature of the delineation process [78]. For instance, in study [5], two radiologists delineate the lung boundaries on the same CXRs. The inter-observer agreement is measured from the delineations. For normal lungs, inter-observer agreement is \((\mu , \sigma ) = (86\%, 13.6)\). However, for deformed lungs, the inter-observer agreement is \((\mu , \sigma ) = (73\%, 18.1)\), slightly lower than lung agreement for normal cases, mainly because of the invisible border occurred due to pathology. With the standardization of annotation guidelines and with the help of the interactive tools, the subjectivity of the delineation process may decrease.

To our knowledge, there are few publicly available CXR datasets along with expert annotated lung boundaries and other characteristics identified above. Following are the list of these datasets.

JSRT dataset [60] is compiled by the Japanese Society of Radiological Technology (JSRT) which contains 247 CXRs (154 CXRs with lung nodules and 93 CXRs without lung nodules). All CXRs have a size of \(2048 \times 2048\) pixels, the spatial resolution of 0.175 mm/pixel and 12-bit grayscale color depth. The CXRs are publicly available at [83]. In addition, patient age, gender, diagnosis, and the location of the anomalies are provided as text files. The reference lung boundaries (along with heart and clavicle boundaries) are available at [11, 84]. This dataset is collected for developing lung nodule detection algorithms. Therefore the only abnormality in this set is lung nodules which do not cause any shape and texture deformations on the lungs.

NLM Sets [59]: The U.S. National Library of Medicine has made two CXR datasets available: the Montgomery and Shenzhen datasets. The Montgomery set contains 138 frontal CXRs from Montgomery County’s Tuberculosis screening program. Eighty of the X-rays are normal, and 58 of X-rays have manifestations of TB. The size of the X-rays is either \(4020 \times 4892\) or \(4892 \times 4020\) with 12 bit grayscale color depth. The reference lung regions of CXRs are manually delineated by an expert radiologist [36]. The Shenzhen set is collected in collaboration with Shenzhen No.3 People’s Hospital, Guangdong Medical College, Shenzhen, China. The set contains 662 CXRs. Three hundred twenty-six of X-rays belong to normal cases, and 336 cases have manifestations of TB. CXR sizes vary but approximately \(3\,\mathrm{K} \times 3\,\mathrm{K}\). The datasets are publicly available at [85].

Belarus Set [86] is collected for a drug resistance study initiated by the National Institute of Allergy and Infectious Diseases, the United Institute of Informatics Problems of the National Academy of Sciences of Belarus, and the Republican Research and Practical Center for Pulmonology and Tuberculosis, Ministry of Health, Republic of Belarus. Much of the data collected for this study is publicly available [86]. The set contains both CXRs and CTs of 169 patients. Chest radiographs were taken using the Kodak Point-of-Care 260 system with \(2248 \times 2248\) pixel resolution. Reference boundaries of the lung regions are available for each CXR.

The literature has several other publicly available CXR databases such as NIH-CXR dataset [87], NLM Indiana CXR collection [88], and New Delhi dataset [89]. However, there are no reference lung boundaries for the CXRs in these sets.