Multiple-Instance Contrastive Alignment for Pose-Based Sign Spotting
Abstract
AbstractSign spotting, the task of deciding whether an isolated query sign occurs inside a clip of continuous signing, enables sign language video retrieval without gloss or text annotation. We describe our system for the WSLP@EMNLP-2026 Word/Sign Presence Prediction shared task on Indian Sign Language: a lightweight (2.86M-parameter) pose-based Siamese encoder that embeds both clips in one space, forms an explicit frame-to-frame cosine similarity map, and reduces it with multiple-instance pooling over temporal windows. Training combines binary cross-entropy with a multiple-instance contrastive objective (MIL-NCE). The system reaches 72.69 accuracy and 71.95 macro-F1 on the blind test set, against 61.66 and 58.42 for the published pose baseline on the same data. Ablations identify contrastive alignment as the component that makes the architecture work: the MIL-NCE objective is what places the two clips in a comparable metric space, and training does not leave chance without it. We further document a label-balance property of the released splits that fixes 76.3% of blind test labels without a model, and recommend a change for future editions.