Pattern-Anchored Adaptive Prototype Learning for Gastroscopic Lesion Detection and Beyond

Gastroscopic Lesion Detection (GLD) is one of the critical tasks within computer-assisted gastroscopic diagnostics. Each category of gastroscopic lesions have several sub-category patterns and endoscopists adopt a pattern-based philosophy for GLD: they identify and summarize typical sub-category patterns with specific medical meanings and conduct GLD based on these patterns. The current gastroscopic lesion detectors follow the classical data-driven deep-learning-based training paradigm, which differs from the endoscopists’ diagnosis process and leads to low interpretability, limiting their performance and potential for daily clinical practice and patient care. Directly integrating the philosophy into GLD tasks requires re-annotating the whole GLD datasets with sub-category labels, which is prohibitively expensive. An affordable solution is to annotate limited sub-category labels and train the detectors in a few-shot manner. To address the challenge, we propose a Pattern-Anchored Adaptive Prototype Learning (PAAPL) for Gastroscopic Lesion Detection. Specifically, to integrate the pattern-based philosophy, PAAPL introduces a Prototype-based Gastroscopic Lesion Detector (PGLD) for prototype-based sub-category pattern detection and a Pattern-Anchored Adaptive Learning (PAAL) method to learn prototypes from GLD datasets with limited sub-category pattern annotations. The PAAL can soft-anchor prototypes to limited-annotated patterns with specific medical meanings and adaptively learn pattern characteristics from unannotated data samples in GLD datasets. We evaluate PAAPL on the LGLDD and Endo21 datasets, demonstrating its ability to learn and detect sub-category patterns trained with limited annotated examples. By doing this, PAAPL enhances detector interpretability and yields significant performance improvement (+3.7AP on LGLDD/+5.4AP on Endo21).