David Harwath

Hear you are: Teaching LLMs Spatial Reasoning with Vision and Spatial Sound CVPR 2026 MMAU-Pro: A Challenging and Comprehensive Benchmark for Holistic Evaluation of Audio General Intelligence AAAI 2026 Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks ICLR 2025 Probing the Robustness Properties of Neural Speech Codecs Interspeech 2025 Scaling Rich Style-Prompted Text-to-Speech Datasets EMNLP 2025 SyllableLM: Learning Coarse Semantic Units for Speech Language Models ICLR 2025 Temporally Streaming Audio-Visual Synchronization for Real-World Videos WACV 2025 VoiceCraft-Dub: Automated Video Dubbing with Neural Codec Language Models ICCV 2025 VoiceCraft-X: Unifying Multilingual, Voice-Cloning Speech Synthesis and Speech Editing EMNLP 2025 Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos ECCV 2024 BAT: Learning to Reason about Spatial Sounds with Large Language Models ICML 2024 Direct Speech Synthesis from Non-Invasive, Neuromagnetic Signals Interspeech 2024 Improving Audio Classification with Low-Sampled Microphone Input: An Empirical Study Using Model Self-Distillation Interspeech 2024 Interface Design for Self-Supervised Speech Models Interspeech 2024 Multimodal Contextualized Semantic Parsing from Speech ACL 2024 Neural Codec Language Models for Disentangled and Textless Voice Conversion Interspeech 2024 SoundingActions: Learning How Actions Sound from Narrated Egocentric Videos CVPR 2024 Textless Speech-to-Speech Translation With Limited Parallel Data EMNLP 2024 VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild ACL 2024 Comparison of Multilingual Self-Supervised and Weakly-Supervised Speech Pre-Training for Adaptation to Unseen Languages Interspeech 2023 Contrastive Audio-Visual Masked Autoencoder ICLR 2023 Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalization Interspeech 2023 Style-transfer based Speech and Audio-visual Scene understanding for Robot Action Sequence Acquisition from Videos Interspeech 2023 Syllable Discovery and Cross-Lingual Generalization in a Visually Grounded, Self-Supervised Speech Model Interspeech 2023 When to Use Efficient Self Attention? Profiling Text, Speech and Image Transformer Variants ACL 2023 Everything at Once - Multi-Modal Fusion Transformer for Video Retrieval CVPR 2022 Exploring Few-Shot Fine-Tuning Strategies for Models of Visually Grounded Speech Interspeech 2022 MAE-AST: Masked Autoencoding Audio Spectrogram Transformer Interspeech 2022 Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality EMNLP 2022 Word Discovery in Visually Grounded, Self-Supervised Speech Models Interspeech 2022 AVLnet: Learning Audio-Visual Language Representations from Instructional Videos Interspeech 2021 Cascaded Multilingual Audio-Visual Learning from Videos Interspeech 2021 Multimodal Clustering Networks for Self-Supervised Learning From Unlabeled Videos ICCV 2021 Spoken Moments: Learning Joint Audio-Visual Representations From Video Descriptions CVPR 2021 Text-Free Image-to-Speech Synthesis Using Learned Segmental Units ACL 2021 Pair Expansion for Learning Multilingual Semantic Embeddings Using Disjoint Visually-Grounded Speech Audio Datasets Interspeech 2020 Learning Words by Drawing Images CVPR 2019 Towards Bilingual Lexicon Discovery From Visually Grounded Speech Audio Interspeech 2019 Transfer Learning from Audio-Visual Grounding to Speech Recognition Interspeech 2019 Jointly Discovering Visual Objects and Spoken Words from Raw Sensory Input ECCV 2018 Learning Word-Like Units from Joint Audio-Visual Analysis ACL 2017 Unsupervised Learning of Spoken Language with Visual Context NeurIPS 2016