Survival Prediction Based on Histopathology Imaging and Clinical Data: A Novel, Whole Slide CNN Approach

Current methods for whole slide image (WSI) histopathology subregion classification and survival prediction rely on phenotype clustering from randomly sampled image tiles or on analyzing key tiles selected by experts from the much larger in size WSIs. These approaches do not capture the whole tissue region present in a histopathology image, also missing the spatial distribution of features that could be critical for good survival predictors. We propose a novel method that extracts a whole slide feature map (WSFM) in the first step and then uses it to train the survival prediction model. Specifically, we partition the WSI into tiles, and for each tile extract InceptionV3 features followed by PCA dimension reduction. The low dimension features of each tile are stored as the channel information in the WSFM. The resulting WSFM preserves the tile adjacency information and captures the entire tissue in the WSI. To overcome the small-size data set concern inherent to previous methods, we design a siamese survival convolutional neural network (SSCNN) that takes the WSFM and multivariate clinical features as input and predicts the survival score. We train the SSCNN using a novel loss function that combines a modified pairwise ranking loss and a bounded inverse term. The key advantages of the proposed method are that it does not require pixel-level annotations, a notorious bottleneck, and it can be easily adapted for any type of tumor without performance dependence on other parameters like the number of clusters. Experimental results demonstrate the effectiveness of the proposed SSCNN over other state-of-the-art survival analysis approaches.