Head and Neck Tumor Segmentation and Outcome Prediction

This paper presents an overview of the third edition of the HEad and neCK TumOR segmentation and outcome prediction (HECKTOR) challenge, organized as a satellite event of the 25th International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) 2022. The challenge comprises two tasks related to the automatic analysis of FDG-PET/CT images for patients with Head and Neck cancer (H &N), focusing on the oropharynx region. Task 1 is the fully automatic segmentation of H &N primary Gross Tumor Volume (GTVp) and metastatic lymph nodes (GTVn) from FDG-PET/CT images. Task 2 is the fully automatic prediction of Recurrence-Free Survival (RFS) from the same FDG-PET/CT and clinical data. The data were collected from nine centers for a total of 883 cases consisting of FDG-PET/CT images and clinical information, split into 524 training and 359 test cases. The best methods obtained an aggregated Dice Similarity Coefficient (\(DSC_{agg}\)) of 0.788 in Task 1, and a Concordance index (C-index) of 0.682 in Task 2.

Head and neck tumor segmentation challenge (HECKTOR) 2022 offers a platform for researchers to compare their solutions to segmentation of tumors and lymph nodes from 3D CT and PET images. In this work, we describe our solution to HECKTOR 2022 segmentation task. We re-sample all images to a common resolution, crop around head and neck region, and train SegResNet semantic segmentation network from MONAI. We use 5-fold cross validation to select best model checkpoint. The final submission is an ensemble of 15 models from 3 runs. Our solution (team name NVAUTO) achieves the 1st place on the HECKTOR22 challenge leaderboard with an aggregated dice score of 0.78802 (https://​hecktor.​grand-challenge.​org/​evaluation/​segmentation/​leaderboard/​.). It is implemented with Auto3DSeg (https://​monai.​io/​apps/​auto3dseg.).

Head and neck (H &N) cancer is one of the most prevalent cancers [1]. In its treatment and prognosis analysis, tumors and metastatic lymph nodes may play an important role but their manual segmentations are time-consuming and laborious. In this paper, we propose a coarse-to-fine ensembling framework to segment the H &N tumor and metastatic lymph nodes automatically from Positron Emission Tomography (PET) and Computed Tomography (CT) images. The framework consists of three steps. The first step is to locate the head region in CT images. The second step is a coarse segmentation, to locate the tumor and lymph region of interest (ROI) from the head region. The last step is a fine segmentation, to get the final precise predictions of tumors and metastatic lymph nodes, where we proposed a ensembling refinement model. This framework is evaluated quantitatively with aggregated Dice Similarity Coefficient (DSC) of 0.77782 in the task 1 of the HECKTOR 2022 challenge[2, 3] as team SJTU426.

Delineation of head and neck lesions are crucial for radiation treatment planning and follow-up studies. In this paper we developed an automated segmentation method for head and neck primary and nodal gross tumor volumes (GTVp and GTVn) segmentation in positron emission tomography/computed tomography (PET/CT) provided by the MICCAI 2022 Head and Neck Tumor Segmentation Challenge (HECKTOR 2022). Our segmentation algorithm takes nnU-Net as the backbone and uses dedicated pre- and post-processing to improve the auto-segmentation performance. The pipeline described achieved DSC results of 0.77212 (GTVp 0.77485 and GTVn 0.76938) in the testing dataset of HECTOR 2022. The developed auto-segmentation method is further extensively developed to a web-based platform to permit easy access and facilitate clinical workflow.

In this paper, we create a fully autonomous system that segments primary head and neck tumors as well as lymph node tumors given only FDG-PET and CT scans without contrast enhancers. Given only these two modalities, the typical Dice score for the state-of-the-art (SOTA) models lies below 0.8, below what it would be when including other modalities due to the low resolution of PET scans and noisy non-enhanced CT images. Thus, we seek to improve tumor segmentation accuracy while working with the limitation of only having these two modalities. We introduce the Transfiner, a novel octree-based refinement system to harness the fidelity of transformers while keeping computation and memory costs low for fast inferencing. The observation behind our method is that segmentation errors almost always occur at the edges of a mask for predictions from a well-trained model. The Transfiner utilizes base network feature maps in addition to the raw modalities as input and selects regions of interest from these. These are then processed with a transformer network and decoded with a CNN. We evaluated our framework with Dice Similarity Coefficient (DSC) 0.76426 for the first task of the Head and Neck Tumor Segmentation Challenge (HECKTOR) and ranked 6th.

Segmentation of head and neck (H &N) cancer primary tumor and lymph nodes on medical imaging is a routine part of radiation treatment planning for patients and may lead to improved response assessment and quantitative imaging analysis. Manual segmentation is a difficult and time-intensive task, requiring specialist knowledge. In the area of computer vision, deep learning-based architectures have achieved state-of-the-art (SOTA) performances for many downstream tasks, including medical image segmentation. Deep learning-based auto-segmentation tools may improve efficiency and robustness of H &N cancer segmentation. For the purpose of encouraging high performing methods for lesion segmentation while utilizing the bi-modal information of PET and CT images, the HEad and neCK TumOR (HECKTOR) challenge is offered annually. In this paper, we preprocess PET/CT images and train and evaluate several deep learning frameworks, including 3D U-Net, MNet, Swin Transformer, and nnU-Net (both 2D and 3D), to segment CT and PET images of primary tumors (GTVp) and cancerous lymph nodes (GTVn) automatically. Our investigations led us to three promising models for submission. Via 5-fold cross validation with ensembling and testing on a blinded hold-out set, we received an average of 0.77 and 0.70 using the aggregated Dice Similarity Coefficient (DSC) metric for primary and node, respectively, for task 1 of the HECKTOR2022 challenge. Herein, we describe in detail the methodology and results for our top three performing models that were submitted to the challenge. Our investigations demonstrate the versatility and robustness of such deep learning models on automatic tumor segmentation to improve H &N cancer treatment. Our full implementation based on the PyTorch framework and the trained models are available at https://​github.​com/​xmuyzz/​HECKTOR2022 (Team name: AIMERS).

Machine learning, especially deep learning, has achieved state-of-the-art performance on various computer vision tasks. For computer vision tasks in the medical domain, it remains as challenging tasks since medical data is heterogeneous, multi-level, and multi-scale. Head and Neck Tumor Segmentation Challenge (HECKTOR) provides a platform to apply machine learning techniques to the medical image domain. HECKTOR 2022 provides positron emission tomography/computed tomography (PET/CT) images which includes useful metabolic and anatomical information to sufficiently make an accurate tumor segmentation. In this paper, we proposed a stacked-multi-scaled medical image segmentation framework to automatically segment the Head and Neck tumor using PET/CT images. The main idea of our network was to generate various low-resolution feature maps of PET/CT images to make a better contour of Head and Neck tumors. We used multi-scaled PET/CT images as inputs, and stacked different intermediate feature maps by resolution for a better inference result. In addition, we evaluated our model on the HECKTOR challenge test dataset. Overall, we achieved a 0.69786, 0.66730 mean Dice score on GTVp and GTVn respectively. Our team’s name is HPCAS.

Head and Neck (HN) cancer has the sixth highest incidence rate of all malignancies worldwide. One of the two main curative treatments for this malignancy is radiotherapy, whose delivery depends on accurate contouring of the primary tumor and affected lymph nodes among other structures. In this study, we present a transfer learning-based approach for the automatic primary tumor and lymph nodes segmentation in fused positron emission tomography (PET) and computed tomography (CT) images belonging to the HECKTOR challenge dataset. Transfer learning is performed from the Genesis Chest CT model, a publicly available 3D U-net, pre-trained on chest CT scans. Three-fold cross-validation is employed during training, so that, on each fold, two different binary segmentation models are chosen, one for the primary tumor and one for the lymph nodes. During testing, majority voting is applied. Our results show promising performance on the training and validation cohorts, while moderate performance was observed in the test cohort.

We implemented a 2D U-Net model with an ImageNet-pretrained ResNet50 encoder for performing segmentation of primary tumors (GTVp) and metastatic lymph nodes (GTVn) from PET/CT images provided by the HEad and neCK TumOR segmentation challenge (HECKTOR) 2022. We utilized a multiclass Dice Loss for model training which was minimized using the AMSGrad variant of the Adam algorithm optimizer. We trained our 2D models on the axial slices of the images in a 5-fold cross-validation setting and stacked the 2D predictions axially to obtain the predicted 3D segmentation masks. We obtained mean aggregate Dice similarity coefficients (mean DSC\(_{\text {agg}}\)) of 0.6865, 0.6689, 0.6768, 0.6792, and 0.6726 on the 5 validation sets respectively. The model with the best performance on the validation set (validation split 1) was chosen for evaluating segmentation masks on the test set for submission to the challenge. Our model achieved a mean DSC\(_{\text {agg}}\) = 0.6345 on the test set, with DSC\(_{\text {agg}}\)(GTVp) = 0.6955 and DSC\(_{\text {agg}}\)(GTVn) = 0.5734. The implementation can be found under our Github repository.

Head and Neck (H &N) organ-at-risk (OAR) and tumor segmentations are an essential component of radiation therapy planning. The varying anatomic locations and dimensions of H &N nodal Gross Tumor Volumes (GTVn) and H &N primary gross tumor volume (GTVp) are difficult to obtain due to lack of accurate and reliable delineation methods. The downstream effect of incorrect segmentation can result in unnecessary irradiation of normal organs. Towards a fully automated radiation therapy planning algorithm, we explore the efficacy of multi-scale fusion based deep learning architectures for accurately segmenting H &N tumors from medical scans. Team Name: M &H_lab_NU.

Delineation of Gross Tumor Volume (GTV) is essential for the treatment of cancer with radiotherapy. GTV contouring is a time-consuming specialized manual task performed by radiation oncologists. Deep Learning (DL) algorithms have shown potential in creating automatic segmentations, reducing delineation time and inter-observer variation. The aim of this work was to create automatic segmentations of primary tumors (GTVp) and pathological lymph nodes (GTVn) in oropharyngeal cancer patients using DL. The organizers of the HECKTOR 2022 challenge provided 3D Computed Tomography (CT) and Positron Emission Tomography (PET) scans with ground-truth GTV segmentations acquired from nine different centers. Bounding box cropping was applied to obtain an anatomic based region of interest. We used the Swin UNETR model in combination with transfer learning. The Swin UNETR encoder weights were initialized by pre-trained weights of a self-supervised Swin UNETR model. An average Dice score of 0.656 was achieved on a test set of 359 patients from the HECKTOR 2022 challenge. Code is available at: https://​github.​com/​HC94/​swin_​unetr_​hecktor_​2022.

Automated lesion detection and segmentation might assist radiation therapy planning and contribute to the identification of prognostic image-based biomarkers towards personalized medicine. In this paper, we propose a pipeline to segment the primary and metastatic lymph nodes from fluorodeoxyglucose (FDG) positron emission tomography and computed tomography (PET/CT) head and neck (H &N) images and then predict recurrence free survival (RFS) based on the segmentation results. For segmentation, an out-of-the-box nnUNet-based deep learning method was trained and labelled the two lesion types as primary gross tumor volume (GTVp) and metastatic nodes (GTVn). For RFS prediction, 2421 radiomic features were extracted from the merged GTVp and GTVn using the pyradiomics package. The ability of each feature to predict RFS was measured using the C-index. Only the features with a C-index greater than \(C_{min}\), hyperparameter of the model, were selected and assigned a +1 or –1 weight as a function of how they varied with the recurrence time. The final RFS probability was calculated as the mean across all selected feature z-scores weighted by their +/–1 weight. The fully automated pipeline was applied to the data provided through the HECKTOR 2022 MICCAI challenge. On the test data, the fully automated segmentation model achieved 0.777 and 0.763 Dice scores on the primary tumor and lymph nodes respectively (0.770 on average). The binary-weighted radiomic model yielded a 0.682 C-index. These results allowed us to rank first for outcome prediction and fourth for segmentation in the challenge. We conclude that the proposed fully-automated pipeline from segmentation to outcome prediction using a binary-weighted radiomic model competes well with more complicated models. Team: LITO.

Outcome prediction is crucial for head and neck cancer patients as it can provide prognostic information for early treatment planning. Radiomics methods have been widely used for outcome prediction from medical images. However, these methods are limited by their reliance on intractable manual segmentation of tumor regions. Recently, deep learning methods have been proposed to perform end-to-end outcome prediction so as to remove the reliance on manual segmentation. Unfortunately, without segmentation masks, these methods will take the whole image as input, such that makes them difficult to focus on tumor regions and potentially unable to fully leverage the prognostic information within the tumor regions. In this study, we propose a radiomics-enhanced deep multi-task framework for outcome prediction from PET/CT images, in the context of HEad and neCK TumOR segmentation and outcome prediction challenge (HECKTOR 2022). In our framework, our novelty is to incorporate radiomics as an enhancement to our recently proposed Deep Multi-task Survival model (DeepMTS). The DeepMTS jointly learns to predict the survival risk scores of patients and the segmentation masks of tumor regions. Radiomics features are extracted from the predicted tumor regions and combined with the predicted survival risk scores for final outcome prediction, through which the prognostic information in tumor regions can be further leveraged. Our method achieved a C-index of 0.681 on the testing set, placing the 2^nd on the leaderboard with only 0.00068 lower in C-index than the 1^st place.

For Head and Neck Cancers (HNC) patient management, automatic gross tumor volume (GTV) segmentation and accurate pre-treatment cancer recurrence prediction are of great importance to assist physicians in designing personalized management plans, which have the potential to improve the treatment outcome and quality of life for HNC patients. In this paper, we developed an automated primary tumor (GTVp) and lymph nodes (GTVn) segmentation method based on combined pre-treatment positron emission tomography/computed tomography (PET/CT) scans of HNC patients. We extracted radiomics features from the segmented tumor volume and constructed a multi-modality tumor recurrence-free survival (RFS) prediction model, which fused the prediction results from separate CT radiomics, PET radiomics, and clinical models. We performed 5-fold cross-validation to train and evaluate our methods on the MICCAI 2022 HEad and neCK TumOR segmentation and outcome prediction challenge (HECKTOR) dataset. The ensemble prediction results on the testing cohort achieved aggregated Dice scores of 0.77 and 0.73 for GTVp and GTVn segmentation, respectively, and a C-index value of 0.67 for RFS prediction. The code is publicly available (https://​github.​com/​wangkaiwan/​HECKTOR-2022-AIRT). Our team’s name is AIRT.

Automatic segmentation of head and neck cancer (HNC) tumors and lymph nodes plays a crucial role in the optimization treatment strategy and prognosis analysis. This study aims to employ nnU-Net for automatic segmentation and radiomics for recurrence-free survival (RFS) prediction using pretreatment PET/CT images in a multi-center HNC cohort of 883 patients (524 patients for training, 359 for testing) provided within the context of the HECKTOR MICCAI challenge 2022. A bounding box of the extended oropharyngeal region was retrieved for each patient with fixed size of 224 \(\times \) 224 \(\times \) 224 mm\(^{3}\). Then the 3D nnU-Net architecture was adopted to carry out automatic segmentation of both primary tumor and lymph nodes. From the predicted segmentation mask, ten conventional features and 346 standardized radiomics features were extracted for each patient. Three prognostic models were constructed containing conventional and radiomics features alone, and their combinations by multivariate CoxPH modelling. The statistical harmonization method, ComBat, was explored towards reducing multicenter variations. Dice score and C-index were used as evaluation metrics for segmentation and prognosis task, respectively. For segmentation task, we achieved a mean dice score of 0.7 for primary tumor and lymph nodes. For recurrence-free survival prediction, conventional and radiomics models obtained C-index values of 0.66 and 0.65 in the test set, respectively, while the combined model did not improve the prognostic performance (0.65).

Head and neck cancers are the fifth most common cancer worldwide, and recently, analysis of Positron Emission Tomography (PET) and Computed Tomography (CT) images has been proposed to identify patients with a prognosis. Even though the results look promising, more research is needed to further validate and improve the results. This paper presents the work done by team MLC for the 2022 version of the HECKTOR grand challenge held at MICCAI 2022. For Task 1, the automatic segmentation task, our approach was, in contrast to earlier solutions using 3D segmentation, to keep it as simple as possible using a 2D model, analyzing every slice as a standalone image. In addition, we were interested in understanding how different modalities influence the results. We proposed two approaches; one using only the CT scans to make predictions and another using a combination of the CT and PET scans. For Task 2, the prediction of recurrence-free survival, we first proposed two approaches, one where we only use patient data and one where we combined the patient data with segmentations from the image model. For the prediction of the first two approaches, we used Random Forest. In our third approach, we combined patient data and image data using XGBoost. Low kidney function might worsen cancer prognosis. In this approach, we therefore estimated the kidney function of the patients and included it as a feature. Overall, we conclude that our simple methods were not able to compete with the highest-ranking submissions, but we still obtained reasonably good scores. We also got interesting insights into how the combination of different modalities can influence the segmentation and predictions.

Head and neck (H &N) tumor segmentation from FDG-PET/CT images has a significant impact on radiotherapy diagnosis. However, manual delineation of primary tumor and lymph nodes is a time-consuming and labor-intensive process. Also, patients that underwent radiotherapy have a high risk of regional recurrence. In this work, we used 3D nnU-Net with DiceTopK loss function to achieve automatic segmentation for head and neck primary tumor and lymph nodes. With the average ensemble of five cross-validation models, our approach got an aggregated Dice Similarity Coefficient of 0.70427 on the test set. Furthermore, we extracted radiomics features from PET/CT images combined with the clinical data to predict patients’ Recurrence-Free Survival (RFS) using a weighted ensemble predictor by AutoGluon. This led to a Concordance index of 0.63896 on the test set. The code is publicly available at  https://​github.​com/​amylyu1123/​HECKTOR-2022.

Auto-segmentation of the primary tumor (GTVp) and the associated lymph nodes (GTVn) of head and neck cancer (HNC) is deemed beneficial for precise radiotherapy. However, previous studies were mostly focusing on the auto segmentation of GTVp. It remains difficult to additionally segment GTVn automatically. Moreover, current methods also face challenges in handling whole-body scans due to memory limitations. In this study, the Retina Unet has been utilized for the first time to localize the HNC from whole-body positron emission tomography/computed tomography (PET/CT) scans. Cropped (based on the predicted tumor center) PET/CTs were then input to a multi-label Unet to produce the auto segmentation of GTVp and GTVn. Several time-to-event analysis models, including a segmentation-free model, have also been explored for relapse free survival (RFS) prognosis. The proposed models achieved encouraging cross validation (testing) Dice coefficient scores for GTVp/GTVn segmentation of 0.66 (0.70), indicating a promising first step towards fully automated HNC localization and segmentation from whole-body PET/CT scans. Furthermore, the segmentation-free PET-only RFS prognosis model produced the best average cross-validation (testing) Harrell’s Concordance Index of 0.70 (0.635), verifying our previous observation that GTV segmentation might be less relevant for PET-based prognosis.

Head and neck cancer is one of the most prevalent cancers in the world. Automatic delineation of primary tumors and lymph nodes is important for cancer diagnosis and treatment. In this paper, we develop a deep learning-based model for automatic tumor segmentation, HNT-AI, using PET/CT images provided by the MICCAI 2022 Head and Neck Tumor (HECKTOR) segmentation Challenge. We investigate the effect of residual blocks, squeeze-and-excitation normalization, and grid-attention gates on the performance of 3D-UNET. We project the predicted masks on the z-axis and apply k-means clustering to reduce the number of false positive predictions. Our proposed HNT-AI segmentation framework achieves an aggregated dice score of 0.774 and 0.759 for primary tumors and lymph nodes, respectively, on the unseen external test set. Qualitative analysis of the predicted segmentation masks shows that the predicted segmentation mask tends to follow the high standardized uptake value (SUV) area on the PET scans more closely than the ground truth masks. The largest tumor volume, the larget lymph node volume, and the total number of lymph nodes derived from the segmentation proved to be potential biomarkers for recurrence-free survival with a C-index of 0.627 on the test set.

Head and Neck Squamous Cell Carcinoma (HNSCC) is a group of malignancies arising in the squamous cells of the head and neck region. As a group, HNSCC accounts for around 4.5% of cancer incidences and deaths worldwide. Radiotherapy is part of the standard care for HNSCC cancers and accurate delineation of tumors is important for treatment quality. Imaging features of Computed Tomography (CT) and Positron Emission Tomography (PET) scans have been shown to be correlated with survival of HNSCC patients. In this paper we present our solutions to the segmentation task and recurrence-free survival prediction task of the HECKTOR 2022 challenge. We trained a 3D UNet model for the segmentation of primary tumors and lymph node metastases based on CT images. Three sets of models with different combinations of loss functions were ensembled to generate a more robust model. The softmax output of the ensembled model was fused with co-registered PET scans and post-processed to generate our submission to task 1 of the challenge, which achieved a 0.716 aggregated Dice score on the test data. Our segmentation model outputs were used to extract radiomic features of individual tumors on test data. Clinical variables and location of the tumors were also encoded and concatenated with radiomic features as additional inputs. We trained a multiple instance neural network to aggregate features of individual tumors into patient-level representations and predict recurrence-free survival rates of patients. Our method achieved an AUC of 0.619 for task 2 on the test data (Team name: SMIAL).

Aim: The development and evaluation of deep learning (DL) and radiomics based models for recurrence-free survival (RFS) prediction in oropharyngeal squamous cell carcinoma (OPSCC) patients based on clinical features, positron emission tomography (PET) and computed tomography (CT) scans and GTV (Gross Tumor Volume) contours of primary tumors and pathological lymph nodes.

Methods: A DL auto-segmentation algorithm generated the GTV contours (task 1) that were used for imaging biomarkers (IBMs) extraction and as input for the DL model. Multivariable cox regression analysis was used to develop radiomics models based on clinical and IBMs features. Clinical features with a significant correlation with the endpoint in a univariable analysis were selected. The most promising IBMs were selected by forward selection in 1000 times bootstrap resampling in five-fold cross validation. To optimize the DL models, different combinations of clinical features, PET/CT imaging, GTV contours, the selected radiomics features and the radiomics model predictions were used as input. The combination with the best average performance in five-fold cross validation was taken as the final input for the DL model. The final prediction in the test set, was an ensemble average of the predictions from the five models for the different folds.

Results: The average C-index in the five-fold cross validation of the radiomics model and the DL model were 0.7069 and 0.7575, respectively. The radiomics and final DL models showed C-indexes of 0.6683 and 0.6455, respectively in the test set.

Conclusion: The radiomics model for recurrence free survival prediction based on clinical, GTV and CT image features showed the best predictive performance in the test set with a C-index of 0.6683.