Multi-task Learning Network for Automatic Pancreatic Tumor Segmentation and Classification with Inter-Network Channel Feature Fusion

Pancreatic cancer is a malignant tumor with a high mortality rate. Therefore, accurately identifying pancreatic cancer is of great significance for early diagnosis and treatment. Currently, several methods have been developed using network structures based on multi-task learning to address tumor recognition issues. One common approach is to use the encoding part of a segmentation network as shared features for both segmentation and classification tasks. However, due to the focus on detailed features in segmentation tasks and the requirement for more global features in classification tasks, the shared features may not provide more discriminatory feature representation for the classification task. To address above challenges, we propose a novel multi-task learning network that leverages the correlation between the segmentation and classification networks to enhance the performance of both tasks. Specifically, the classification task takes the tumor region images extracted from the segmentation network’s output as input, effectively capturing the shape and internal texture features of the tumor. Additionally, a feature fusion module is added between the networks to facilitate information exchange and fusion. We evaluated our model on 82 clinical CT image samples. Experimental results demonstrate that our proposed multi-task network achieves excellent performance with a Dice similarity coefficient (DSC) of 88.42% and a classification accuracy of 85.71%.