Interspeech 2024

1,065 papers

A Contrastive Learning Approach to Mitigate Bias in Speech Models

Alkis Koudounas, Flavio Giobergia, Eliana Pastor, Elena Baralis

Using Large Language Model for End-to-End Chinese ASR and NER

Yuang Li, Jiawei Yu, Min Zhang, Mengxin Ren, Yanqing Zhao

Speech-MASSIVE: A Multilingual Speech Dataset for SLU and Beyond

Beomseok Lee, Ioan Calapodescu, Marco Gaido, Matteo Negri, Laurent Besacier

A dual task learning approach to fine-tune a multilingual semantic speech encoder for Spoken Language Understanding

Gaëlle Laperrière, Sahar Ghannay, Bassam Jabaian, Yannick Estève

Out-of-distribution generalisation in spoken language understanding

Dejan Porjazovski, Anssi Moisio, Mikko Kurimo

Finding Task-specific Subnetworks in Multi-task Spoken Language Understanding Model

Hayato Futami, Siddhant Arora, Yosuke Kashiwagi, Emiru Tsunoo, Shinji Watanabe

Domain-Aware Data Selection for Speech Classification via Meta-Reweighting

Junghun Kim, Ka Hyun Park, Hoyoung Yoon, U Kang

Speculative Speech Recognition by Audio-Prefixed Low-Rank Adaptation of Language Models

Bolaji Yusuf, Murali Karthick Baskar, Andrew Rosenberg, Bhuvana Ramabhadran

Dual-Pipeline with Low-Rank Adaptation for New Language Integration in Multilingual ASR

Yerbolat Khassanov, Zhipeng Chen, Tianfeng Chen, Tze Yuang Chong, Wei Li

Factor-Conditioned Speaking-Style Captioning

Atsushi Ando, Takafumi Moriya, Shota Horiguchi, Ryo Masumura

SAML: Speaker Adaptive Mixture of LoRA Experts for End-to-End ASR

Qiuming Zhao, Guangzhi Sun, Chao Zhang, Mingxing Xu, Thomas Fang Zheng

Modality Translation Learning for Joint Speech-Text Model

Pin-Yen Liu, Jen-Tzung Chien

Improved Factorized Neural Transducer Model For Text-only Domain Adaptation

Junzhe Liu, Jianwei Yu, Xie Chen

Prompt Tuning for Speech Recognition on Unknown Spoken Name Entities

Xizi Wei, Stephen McGregor

Fast Context-Biasing for CTC and Transducer ASR models with CTC-based Word Spotter

Andrei Andrusenko, Aleksandr Laptev, Vladimir Bataev, Vitaly Lavrukhin, Boris Ginsburg

Improving Neural Biasing for Contextual Speech Recognition by Early Context Injection and Text Perturbation

Ruizhe Huang, Mahsa Yarmohammadi, Sanjeev Khudanpur, Daniel Povey

Contextual Biasing with Confidence-based Homophone Detector for Mandarin End-to-End Speech Recognition

Chengxu Yang, Lin Zheng, Sanli Tian, Gaofeng Cheng, Sujie Xiao

Incorporating Class-based Language Model for Named Entity Recognition in Factorized Neural Transducer

Peng Wang, Yifan Yang, Zheng Liang, Tian Tan, Shiliang Zhang

Improving Speech Recognition with Prompt-based Contextualized ASR and LLM-based Re-predictor

Nguyen Manh Tien Anh, Thach Ho Sy

Keyword-Guided Adaptation of Automatic Speech Recognition

Aviv Shamsian, Aviv Navon, Neta Glazer, Gill Hetz, Joseph Keshet

Unified Multi-Talker ASR with and without Target-speaker Enrollment

Ryo Masumura, Naoki Makishima, Tomohiro Tanaka, Mana Ihori, Naotaka Kawata

Neural Blind Source Separation and Diarization for Distant Speech Recognition

Yoshiaki Bando, Tomohiko Nakamura, Shinji Watanabe

SOT Triggered Neural Clustering for Speaker Attributed ASR

Xianrui Zheng, Guangzhi Sun, Chao Zhang, Philip C. Woodland

Serialized Output Training by Learned Dominance

Ying Shi, Lantian Li, Shi Yin, Dong Wang, Jiqing Han

A Joint Noise Disentanglement and Adversarial Training Framework for Robust Speaker Verification

Xujiang Xing, Mingxing Xu, Thomas Fang Zheng

LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarization

Zengrui Jin, Yifan Yang, Mohan Shi, Wei Kang, Xiaoyu Yang

DINO-VITS: Data-Efficient Zero-Shot TTS with Self-Supervised Speaker Verification Loss for Noise Robustness

Vikentii Pankov, Valeria Pronina, Alexander Kuzmin, Maksim Borisov, Nikita Usoltsev

Lightweight Zero-shot Text-to-Speech with Mixture of Adapters

Kenichi Fujita, Takanori Ashihara, Marc Delcroix, Yusuke Ijima

An Investigation of Noise Robustness for Flow-Matching-Based Zero-Shot TTS

Xiaofei Wang, Sefik Emre Eskimez, Manthan Thakker, Hemin Yang, Zirun Zhu

Improving Audio Codec-based Zero-Shot Text-to-Speech Synthesis with Multi-Modal Context and Large Language Model

Jinlong Xue, Yayue Deng, Yicheng Han, Yingming Gao, Ya Li

Dynamic Gated Recurrent Neural Network for Compute-efficient Speech Enhancement

Longbiao Cheng, Ashutosh Pandey, Buye Xu, Tobi Delbruck, Shih-Chii Liu

MUSE: Flexible Voiceprint Receptive Fields and Multi-Path Fusion Enhanced Taylor Transformer for U-Net-based Speech Enhancement

Zizhen Lin, Xiaoting Chen, Junyu Wang

Lightweight Dynamic Sparse Transformer for Monaural Speech Enhancement

Zehua Zhang, Xuyi Zhuang, Yukun Qian, Mingjiang Wang

Streamlining Speech Enhancement DNNs: an Automated Pruning Method Based on Dependency Graph with Advanced Regularized Loss Strategies

Zugang Zhao, Jinghong Zhang, Yonghui Liu, Jianbing Liu, Kai Niu

Sub-PNWR: Speech Enhancement Based on Signal Sub-Band Splitting and Pseudo Noisy Waveform Reconstruction Loss

Yuewei Zhang, Huanbin Zou, Jie Zhu

Knowledge Distillation for Tiny Speech Enhancement with Latent Feature Augmentation

Behnam Gholami, Mostafa El-Khamy, KeeBong Song

Speech Boosting: Low-Latency Live Speech Enhancement for TWS Earbuds

Hanbin Bae, Pavel Andreev, Azat Saginbaev, Nicholas Babaev, WonJun Lee

QMixCAT: Unsupervised Speech Enhancement Using Quality-guided Signal Mixing and Competitive Alternating Model Training

Shilin Wang, Haixin Guan, Yanhua Long

Elucidating Clock-drift Using Real-world Audios In Wireless Mode For Time-offset Insensitive End-to-End Asynchronous Acoustic Echo Cancellation

Premanand Nayak, M. Ali Basha Shaik

ANIMAL-CLEAN – A Deep Denoising Toolkit for Animal-Independent Signal Enhancement

Alexander Barnhill, Elmar Noeth, Andreas Maier, Christian Bergler

DeWinder: Single-Channel Wind Noise Reduction using Ultrasound Sensing

Kuang Yuan, Shuo Han, Swarun Kumar, Bhiksha Raj

Speech dereverberation constrained on room impulse response characteristics

Louis Bahrman, Mathieu Fontaine, Jonathan Le Roux, Gaël RICHARD

Graph Attention Based Multi-Channel U-Net for Speech Dereverberation With Ad-Hoc Microphone Arrays

Hongmei Guo, Yijiang Chen, Xiao-Lei Zhang, Xuelong Li

Deep Echo Path Modeling for Acoustic Echo Cancellation

Fei Zhao, Chenggang Zhang, Shulin He, Jinjiang Liu, Xueliang Zhang

Enhanced Reverberation as Supervision for Unsupervised Speech Separation

Kohei Saijo, Gordon Wichern, François G. Germain, Zexu Pan, Jonathan Le Roux

TSE-PI: Target Sound Extraction under Reverberant Environments with Pitch Information

Yiwen Wang, Xihong Wu

SA-WavLM: Speaker-Aware Self-Supervised Pre-training for Mixture Speech

Jingru Lin, Meng Ge, Junyi Ao, Liqun Deng, Haizhou Li

Multimodal Representation Loss Between Timed Text and Audio for Regularized Speech Separation

Tsun-An Hsieh, Heeyoul Choi, Minje Kim

OR-TSE: An Overlap-Robust Speaker Encoder for Target Speech Extraction

Yiru Zhang, Linyu Yao, Qun Yang

PARIS: Pseudo-AutoRegressIve Siamese Training for Online Speech Separation

Zexu Pan, Gordon Wichern, François G. Germain, Kohei Saijo, Jonathan Le Roux

Does the Lombard Effect Matter in Speech Separation? Introducing the Lombard-GRID-2mix Dataset

Iva Ewert, Marvin Borsdorf, Haizhou Li, Tanja Schultz

Towards Explainable Monaural Speaker Separation with Auditory-based Training

Hassan Taherian, Vahid Ahmadi Kalkhorani, Ashutosh Pandey, Daniel Wong, Buye Xu

Low-Complexity Acoustic Scene Classification Using Parallel Attention-Convolution Network

Yanxiong Li, Jiaxin Tan, Guoqing Chen, Jialong Li, Yongjie Si

Sound Event Bounding Boxes

Janek Ebbers, François G. Germain, Gordon Wichern, Jonathan Le Roux

MAT-SED: A Masked Audio Transformer with Masked-Reconstruction Based Pre-training for Sound Event Detection

Pengfei Cai, Yan Song, Kang Li, Haoyu Song, Ian McLoughlin

Audio Mamba: Selective State Spaces for Self-Supervised Audio Representations

Sarthak Yadav, Zheng-Hua Tan

Scaling up masked audio encoder learning for general audio classification

Heinrich Dinkel, Zhiyong Yan, Yongqing Wang, Junbo Zhang, Yujun Wang

Can Synthetic Audio From Generative Foundation Models Assist Audio Recognition and Speech Modeling?

Tiantian Feng, Dimitrios Dimitriadis, Shrikanth S. Narayanan

Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detection

Duc-Tuan Truong, Ruijie Tao, Tuan Nguyen, Hieu-Thi Luong, Kong Aik Lee

Boosting the Transferability of Adversarial Examples with Gradient-Aligned Ensemble Attack for Speaker Recognition

Zhuhai Li, Jie Zhang, Wu Guo, Haochen Wu

Textual-Driven Adversarial Purification for Speaker Verification

Sizhou Chen, Yibo Bai, Jiadi Yao, Xiao-Lei Zhang, Xuelong Li

Neural Codec-based Adversarial Sample Detection for Speaker Verification

Xuanjun Chen, JIAWEI DU, Haibin Wu, Jyh-Shing Roger Jang, Hung-yi Lee

VoiceDefense: Protecting Automatic Speaker Verification Models Against Black-box Adversarial Attacks

Yip Keng Kan, Ke Xu, Hao Li, Jie Shi

Improving Copy-Synthesis Anti-Spoofing Training Method with Rhythm and Speaker Perturbation

Jingze Lu, Yuxiang Zhang, Zhuo Li, Zengqiang Shang, Wenchao Wang

Spoofing Speech Detection by Modeling Local Spectro-Temporal and Long-term Dependency

Haochen Wu, Wu Guo, Zhentao Zhang, Wenting Zhao, Shengyu Peng

Spoof Diarization: "What Spoofed When" in Partially Spoofed Audio

Lin Zhang, Xin Wang, Erica Cooper, Mireia Diez, Federico Landini

Anti-spoofing Ensembling Model: Dynamic Weight Allocation in Ensemble Models for Improved Voice Biometrics Security

Eros Rosello, Angel M. Gomez, Iván López-Espejo, Antonio M. Peinado, Juan M. Martín-Doñas

Switching Tongues, Sharing Hearts: Identifying the Relationship between Empathy and Code-switching in Speech

Debasmita Bhattacharya, Eleanor Lin, Run Chen, Julia Hirschberg

Real-world PTSD Recognition: A Cross-corpus and Cross-linguistic Evaluation

Alexander Kathan, Martin Bürger, Andreas Triantafyllopoulos, Sabrina Milkus, Jonas Hohmann

Fine-tuning of Pre-trained Models for Classification of Vocal Intensity Category from Speech Signals

Manila Kodali, Sudarsana Reddy Kadiri, Paavo Alku

Cross-transfer Knowledge between Speech and Text Encoders to Evaluate Customer Satisfaction

Luis Felipe Parra-Gallego, Tilak Purohit, Bogdan Vlasenko, Juan Rafael Orozco-Arroyave, Mathew Magimai.-Doss

Familiar and Unfamiliar Speaker Identification in Speech and Singing

Katelyn Taylor, Amelia Gully, Helena Daffern

Exploring the anatomy of articulation rate in spontaneous English speech: relationships between utterance length effects and social factors

James Tanner, Morgan Sonderegger, Jane Stuart-Smith, Tyler Kendall, Jeff Mielke

Entrainment Analysis and Prosody Prediction of Subsequent Interlocutor’s Backchannels in Dialogue

Keiko Ochi, Koji Inoue, Divesh Lala, Tatsuya Kawahara

Modelled Multivariate Overlap: A method for measuring vowel merger

Irene Smith, Morgan Sonderegger, The Spade Consortium

In search of structure and correspondence in intra-speaker trial-to-trial variability

Vivian G. Li

Echoes of Implicit Bias Exploring Aesthetics and Social Meanings of Swiss German Dialect Features

Tillmann Pistor, Adrian Leemann

Leveraging Graphic and Convolutional Neural Networks for Auditory Attention Detection with EEG

Saurav Pahuja, Gabriel Ivucic, Pascal Himmelmann, Siqi Cai, Tanja Schultz

ASA: An Auditory Spatial Attention Dataset with Multiple Speaking Locations

Zijie Lin, Tianyu He, Siqi Cai, Haizhou Li

Auditory Attention Decoding in Four-Talker Environment with EEG

Yujie Yan, Xiran Xu, Haolin Zhu, Pei Tian, Zhongshu Ge

Articulatory synthesis using representations learnt through phonetic label-aware contrastive loss

Jesuraj Bandekar, Sathvik Udupa, Prasanta Kumar Ghosh

Multimodal Segmentation for Vocal Tract Modeling

Rishi Jain, Bohan Yu, Peter Wu, Tejas Prabhune, Gopala Anumanchipalli

Optical Flow Guided Tongue Trajectory Generation for Diffusion-based Acoustic to Articulatory Inversion

Yudong Yang, Rongfeng Su, Rukiye Ruzi, Manwa Ng, Shaofeng Zhao

Direct Speech Synthesis from Non-Invasive, Neuromagnetic Signals

Jinuk Kwon, David Harwath, Debadatta Dash, Paul Ferrari, Jun Wang

Using articulated speech EEG signals for imagined speech decoding

Chris Bras, Tanvina Patel, Odette Scharenborg

Towards EMG-to-Speech with Necklace Form Factor

Peter Wu, Ryan Kaveh, Raghav Nautiyal, Christine Zhang, Albert Guo

A multimodal approach to study the nature of coordinative patterns underlying speech rhythm

Jinyu Li, Leonardo Lancia

ZeroST: Zero-Shot Speech Translation

Sameer Khurana, Chiori Hori, Antoine Laurent, Gordon Wichern, Jonathan Le Roux

Translating speech with just images

Dan Oneata, Herman Kamper

Soft Language Identification for Language-Agnostic Many-to-One End-to-End Speech Translation

Peidong Wang, Jian Xue, Jinyu Li, Junkun Chen, Aswin Shanmugam Subramanian

Navigating the Minefield of MT Beam Search in Cascaded Streaming Speech Translation

Rastislav Rabatin, Frank Seide, Ernie Chang

Knowledge-Preserving Pluggable Modules for Multilingual Speech Translation Tasks

Nan Chen, Yonghe Wang, Feilong Bao

Wave to Interlingua: Analyzing Representations of Multilingual Speech Transformers for Spoken Language Translation

Badr M. Abdullah, Mohammed Maqsood Shaik, Dietrich Klakow

Parameter-Efficient Adapter Based on Pre-trained Models for Speech Translation

Nan Chen, Yonghe Wang, Feilong Bao

OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on E-Branchformer

Yifan Peng, Jinchuan Tian, William Chen, Siddhant Arora, Brian Yan

Adding User Feedback To Enhance CB-Whisper

Raul Monteiro

Language-Universal Speech Attributes Modeling for Zero-Shot Multilingual Spoken Keyword Recognition

Hao Yen, Pin-Jui Ku, Sabato Marco Siniscalchi, Chin-Hui Lee

Text-aware Speech Separation for Multi-talker Keyword Spotting

Haoyu Li, Baochen Yang, Yu Xi, Linfeng Yu, Tian Tan

CTC-aligned Audio-Text Embedding for Streaming Open-vocabulary Keyword Spotting

Sichen Jin, Youngmoon Jung, Seungjin Lee, Jaeyoung Roh, Changwoo Han

Relational Proxy Loss for Audio-Text based Keyword Spotting

Youngmoon Jung, Seungjin Lee, Joon-Young Yang, Jaeyoung Roh, Chang Woo Han