Detection of Glands and Villi by Collaboration of Domain Knowledge and Deep Learning

Architecture distortions of glands and villi are indication of chronic inflammation. However, the “duality” nature of these two structures causes lots of ambiguity for their detection in H&E histology tissue images, especially when multiple instances are clustered together. Based on the observation that once such an object is detected for certain, the ambiguity in the neighborhood of the detected object can be reduced considerably, we propose to combine deep learning and domain knowledge in a unified framework, to simultaneously detect (the closely related) glands and villi in H&E histology tissue images. Our method iterates between exploring domain knowledge and performing deep learning classification, and the two components benefit from each other. (1) By exploring domain knowledge, the generated object proposals (to be fed to deep learning) form a more complete coverage of the true objects and the segmentation of object proposals can be more accurate, thus improving deep learning’s performance on classification. (2) Deep learning can help verify the class of each object proposal, and provide feedback to repeatedly “refresh” and enhance domain knowledge so that more reliable object proposals can be generated later on. Experiments on clinical data validate our ideas and show that our method improves the state-of-the-art for gland detection in H&E histology tissue images (to our best knowledge, we are not aware of any method for villi detection).