Spatial Audio Question Answering and Reasoning on Dynamic Source Movements
Abstract
AbstractSpatial audio understanding aims to enable machines to interpret complex auditory scenes, particularly when sound sources move over time. In this work, we study Spatial Audio Question Answering (SAQA) with a focus on movement reasoning, where a model must infer object motion, position and directional changes directly from stereo audio. First, we introduce a movement-centric spatial audio augmentation framework that synthesizes diverse motion patterns from isolated mono audio events, enabling controlled and scalable training data generation. Second, we propose an end-to-end multimodal finetuning approach with a thinking mode, which allows audio-language models to produce explicit intermediate reasoning steps before predicting an answer. Third, we investigate the impact of query-conditioned source separation as a preprocessing stage and compare three inference regimes: no masking, an audio grounding model (AGM), and ground-truth masks. Our results show that the benefits of thinking mode are most pronounced when paired with high-quality source separation rather than in isolation, with an improvement of +5.1% when a single event is present in the question. These findings highlight the relationship between movement modeling, reasoning, and separation quality, offering new insights for advancing spatial audio understanding.