STAN-LOC: Visual Query-based Video Clip Localization for Fetal Ultrasound Sweep Videos

Detecting standard frame clips in fetal ultrasound videos is crucial for accurate clinical assessment and diagnosis. It enables health- care professionals to evaluate fetal development, identify abnormalities, and monitor overall health with clarity and standardization. To aug- ment sonographer workflow and to detect standard frame clips, we in- troduce the task of Visual Query-based Video Clip Localization in med- ical video understanding. It aims to retrieve a video clip from a given ultrasound sweep that contains frames similar to a given exemplar frame of the required standard anatomical view. To solve the task, we propose STAN-LOC that consists of three main components: (a) a Query-Aware Spatio-Temporal Fusion Transformer that fuses information available in the visual query with the input video. This results in visual query-aware video features which we model temporally to understand spatio-temporal relationship between them. (b) a Multi-Anchor, View-Aware Contrastive loss to reduce the influence of inherent noise in manual annotations es- pecially at event boundaries and in videos featuring highly similar ob- jects. (c) a query selection algorithm during inference that selects the best visual query for a given video to reduce model’s sensitivity to the quality of visual queries. We apply STAN-LOC to the task of detect- ing standard-frame clips in fetal ultrasound heart sweeps given four- chamber view queries. Additionally, we assess the performance of our best model on PULSE [2] data for retrieving standard transventricular plane (TVP) in fetal head videos. STAN-LOC surpasses the state-of-the- art method by 22% in mtIoU. The code will be available upon acceptance at xxx.github.com.