Follow Sonographers’ Visual Scan-path: Adjusting CNN Model for Diagnosing Gout from Musculoskeletal Ultrasound

The current models for automatic gout diagnosis train convolutional neural network (CNN) using musculoskeletal ultrasound (MSKUS) images paired with classification labels, which are annotated by skilled sonographers. However, this prevalent diagnostic model overlooks valuable supplementary information derived from sonographers’ annotations, such as the visual scan-path followed by sonographers. We notice that this annotation procedure offers valuable insight into human attention, aiding the CNN model in focusing on crucial features in gouty MSKUS scans, including the double contour sign, tophus, and snowstorm, which play a crucial role in sonographers’ diagnostic decisions. To verify this, we create a gout MSKUS dataset that enriched with sonographers’ annotation byproduct visual scan-path. Furthermore, we introduce a scan path based fine-tuning training mechanism (SFT) for gout diagnosis models, leveraging the annotation byproduct scan-paths for enhanced learning. The experimental results demonstrate the superiority of our SFT method over several SOTA CNNs.