Wei-Ning Hsu

Pushing the Frontier of Audiovisual Perception with Large-Scale Multimodal Correspondence Learning CVPR 2026 Audiobox TTA-RAG: Improving Zero-Shot and Few-Shot Text-To-Audio with Retrieval-Augmented Generation Interspeech 2025 FlowDec: A flow-based full-band general audio codec with high perceptual quality ICLR 2025 Action2Sound: Ambient-Aware Generation of Action Sounds from Egocentric Videos ECCV 2024 Generative Pre-training for Speech with Flow Matching ICLR 2024 Learning Fine-Grained Controllability on Speech Generation via Efficient Fine-Tuning Interspeech 2024 MusicFlow: Cascaded Flow Matching for Text Guided Music Generation ICML 2024 Scaling Speech Technology to 1,000+ Languages JMLR 2024 XLAVS-R: Cross-Lingual Audio-Visual Speech Representation Learning for Noise-Robust Speech Perception ACL 2024 DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning NeurIPS 2023 Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language ICML 2023 Expresso: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis Interspeech 2023 MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation Interspeech 2023 ReVISE: Self-Supervised Speech Resynthesis With Visual Input for Universal and Generalized Speech Regeneration CVPR 2023 Scaling Laws for Generative Mixed-Modal Language Models ICML 2023 Simple and Effective Unsupervised Speech Translation ACL 2023 Speech-to-Speech Translation for a Real-world Unwritten Language ACL 2023 Toward Joint Language Modeling for Speech Units and Text EMNLP 2023 Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale NeurIPS 2023 data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language ICML 2022 Direct Speech-to-Speech Translation With Discrete Units ACL 2022 Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation Interspeech 2022 Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction ICLR 2022 Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT Interspeech 2022 On-demand compute reduction with stochastic wav2vec 2.0 Interspeech 2022 Robust Self-Supervised Audio-Visual Speech Recognition Interspeech 2022 Simple and Effective Unsupervised Speech Synthesis Interspeech 2022 Text-Free Prosody-Aware Generative Spoken Language Modeling ACL 2022 textless-lib: a Library for Textless Spoken Language Processing NAACL 2022 Textless Speech Emotion Conversion using Discrete & Decomposed Representations EMNLP 2022 Textless Speech-to-Speech Translation on Real Data NAACL 2022 u-HuBERT: Unified Mixed-Modal Speech Pretraining And Zero-Shot Transfer to Unlabeled Modality NeurIPS 2022 Unified Speech-Text Pre-training for Speech Translation and Recognition ACL 2022 fairseq Sˆ2: A Scalable and Integrable Speech Synthesis Toolkit EMNLP 2021 Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training Interspeech 2021 Speech Resynthesis from Discrete Disentangled Self-Supervised Representations Interspeech 2021 Text-Free Image-to-Speech Synthesis Using Learned Segmental Units ACL 2021 Unsupervised Speech Recognition NeurIPS 2021 A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning Interspeech 2020 Unsupervised Methods for Evaluating Speech Representations Interspeech 2020 An Unsupervised Autoregressive Model for Speech Representation Learning Interspeech 2019 Hierarchical Generative Modeling for Controllable Speech Synthesis ICLR 2019 Transfer Learning from Audio-Visual Grounding to Speech Recognition Interspeech 2019 A Study of Enhancement, Augmentation and Autoencoder Methods for Domain Adaptation in Distant Speech Recognition Interspeech 2018 Scalable Factorized Hierarchical Variational Autoencoder Training Interspeech 2018 Unsupervised Adaptation with Interpretable Disentangled Representations for Distant Conversational Speech Recognition Interspeech 2018 Learning Latent Representations for Speech Generation and Transformation Interspeech 2017 Unsupervised Learning of Disentangled and Interpretable Representations from Sequential Data NeurIPS 2017 Exploiting Depth and Highway Connections in Convolutional Recurrent Deep Neural Networks for Speech Recognition Interspeech 2016 Neural Attention for Learning to Rank Questions in Community Question Answering COLING 2016 SLS at SemEval-2016 Task 3: Neural-based Approaches for Ranking in Community Question Answering SemEval 2016