Hybrid Aggregation Network for Survival Analysis from Whole Slide Histopathological Images

Understanding of prognosis and mortality is crucial for evaluating the treatment plans for patients. Recent developments of digital pathology and deep learning bring the possibility of predicting survival time using histopathology whole slide images (WSIs). However, most prevalent methods usually rely on a small set of patches sampled from a WSI and are unable to directly learn from an entire WSI. We argue that a small patch set cannot fully represent patients’ survival risks due to the heterogeneity of tumors; moreover, multiple WSIs from one patient need to be evaluated together. In this paper, we propose a Hybrid Aggregation Network (HANet) to adaptively aggregate information from multiple WSIs of one patient for survival analysis. Specifically, we first extract features from WSIs using a convolutional neural network trained in a self-supervised manner, and further aggregate feature maps using two proposed aggregation modules. The self-aggregation module propagates informative features to the entire WSI, and further abstract features to region representations. The WSI-aggregation module fuses all the region representations from different WSIs of one patient to predict patient-level survival risk. We conduct experiments on two WSI datasets that have accompanying survival data, i.e., NLST and TCGA-LUSC. The proposed method achieves state-of-the-art performances with concordance indices of 0.734 for NLST and 0.668 for TCGA-LUSC, outperforming existing approaches.