Interspeech 2024

1,065 papers

Pre-training Neural Transducer-based Streaming Voice Conversion for Faster Convergence and Alignment-free Training

Hiroki Kanagawa, Takafumi Moriya, Yusuke Ijima

Noise-Robust Voice Conversion by Conditional Denoising Training Using Latent Variables of Recording Quality and Environment

Takuto Igarashi, Yuki Saito, Kentaro Seki, Shinnosuke Takamichi, Ryuichi Yamamoto

Vec-Tok-VC+: Residual-enhanced Robust Zero-shot Voice Conversion with Progressive Constraints in a Dual-mode Training Strategy

Linhan Ma, Xinfa Zhu, Yuanjun Lv, Zhichao Wang, Ziqian Wang

Utilizing Adaptive Global Response Normalization and Cluster-Based Pseudo Labels for Zero-Shot Voice Conversion

Ji Sub Um, Hoirin Kim

Improvement Speaker Similarity for Zero-Shot Any-to-Any Voice Conversion of Whispered and Regular Speech

Aleksei Gusev, Anastasia Avdeeva

RW-VoiceShield: Raw Waveform-based Adversarial Attack on One-shot Voice Conversion

Ching-Yu Yang, Shreya G. Upadhyay, Ya-Tse Wu, Bo-Hao Su, Chi-Chun Lee

How Private is Low-Frequency Speech Audio in the Wild? An Analysis of Verbal Intelligibility by Humans and Machines

Ailin Liu, Pepijn Vunderink, Jose Vargas Quiros, Chirag Raman, Hayley Hung

RawBMamba: End-to-End Bidirectional State Space Model for Audio Deepfake Detection

yujie Chen, Jiangyan Yi, Jun Xue, Chenglong Wang, XiaoHui Zhang

Robust spread spectrum speech watermarking using linear prediction and deep spectral shaping

David Looney, Nikolay D. Gaubitch

Prompt Tuning for Audio Deepfake Detection: Computationally Efficient Test-time Domain Adaptation with Limited Target Dataset

Hideyuki Oiso, Yuto Matsunaga, Kazuya Kakizaki, Taiki Miyagawa

Harder or Different? Understanding Generalization of Audio Deepfake Detection

Nicolas M. Müller, Nicholas Evans, Hemlata Tak, Philip Sperl, Konstantin Böttinger

Enhancing No-Reference Speech Quality Assessment with Pairwise, Triplet Ranking Losses, and ASR Pretraining

Bao Thang Ta, Minh Tu Le, Van Hai Do, Huynh Thi Thanh Binh

Experimental evaluation of MOS, AB and BWS listening test designs

Dan Wells, Andrea Lorena Aldana Blanco, Cassia Valentini, Erica Cooper, Aidan Pine

IndicMOS: Multilingual MOS Prediction for 7 Indian languages

Sathvik Udupa, Soumi Maiti, Prasanta Kumar Ghosh

Embedding Learning for Preference-based Speech Quality Assessment

ChengHung Hu, Yusuke Yasuda, Tomoki Toda

Whisper-PMFA: Partial Multi-Scale Feature Aggregation for Speaker Verification using Whisper Models

Yiyang Zhao, Shuai Wang, Guangzhi Sun, Zehua Chen, Chao Zhang

Parameter-efficient Fine-tuning of Speaker-Aware Dynamic Prompts for Speaker Verification

Zhe Li, Man-Wai Mak, Hung-yi Lee, Helen Meng

On the impact of several regularization techniques on label noise robustness of self-supervised speaker verification systems

Abderrahim Fathan, Xiaolin Zhu, Jahangir Alam

Improving Noise Robustness in Self-supervised Pre-trained Model for Speaker Verification

Chan-yeong Lim, Hyun-seo Shin, Ju-ho Kim, Jungwoo Heo, Kyo-Won Koo

Towards Supervised Performance on Speaker Verification with Self-Supervised Learning by Leveraging Large-Scale ASR Models

Victor Miara, Theo Lepage, Reda Dehak

Self-supervised speaker verification with relational mask prediction

Ju-ho Kim, Hee-soo Heo, Bong-Jin Lee, Youngki Kwon, Minjae Lee

From Text to Emotion: Unveiling the Emotion Annotation Capabilities of LLMs

Minxue Niu, Mimansa Jaiswal, Emily Mower Provost

DropFormer: A Dynamic Noise-Dropping Transformer for Speech Emotion Recognition

Jialong Mai, Xiaofen Xing, Weidong Chen, Xiangmin Xu

Dataset-Distillation Generative Model for Speech Emotion Recognition

Fabian Ritter-Gutierrez, Kuan-Po Huang, Jeremy H. M. Wong, Dianwen Ng, Hung-yi Lee

ExHuBERT: Enhancing HuBERT Through Block Extension and Fine-Tuning on 37 Emotion Datasets

Shahin Amiriparian, Filip Packań, Maurice Gerczuk, Bjorn W. Schuller

Magnitude and timing of acceleration peaks in stressed and unstressed syllables

Malin Svensson Lundmark

What if HAL breathed? Enhancing Empathy in Human-AI Interactions with Breathing Speech Synthesis

Nicolò Loddo, Francisca Pessanha, Almila Akdag

Cross-Attention-Guided WaveNet for EEG-to-MEL Spectrogram Reconstruction

Hao Li, Yuan Fang, Xueliang Zhang, Fei Chen, Guanglai Gao

A novel experimental design for the study of listener-to-listener convergence in phoneme categorization

Qingye Shen, Leonardo Lancia, Noël Nguyen

Behavioral evidence for higher speech rate convergence following natural than artificial time altered speech

Jérémy Giroud, Jessica Lei, Kirsty Phillips, Matthew H. Davis

Decoding Human Language Acquisition: EEG Evidence for Predictive Probabilistic Statistics in Word Segmentation

Bin Zhao, Mingxuan Huang, Chenlu Ma, Jinyi Xue, Aijun Li

Leveraging Large Language Models to Refine Automatic Feedback Generation at Articulatory Level in Computer Aided Pronunciation Training

Huihang Zhong, Yanlu Xie, ZiJin Yao

DBD-CI: Doubling the Band Density for Bilateral Cochlear Implants

Mingyue Shi, Huali Zhou, Qinglin Meng, Nengheng Zheng

State-of-the-art speech production MRI protocol for new 0.55 Tesla scanners

Prakash Kumar, Ye Tian, Yongwan Lim, Sophia X. Cui, Christina Hagedorn

UY/CH-CHILD -- A Public Chinese L2 Speech Database of Uyghur Children

Mewlude Nijat, Chen Chen, Dong Wang, Askar Hamdulla

VoxSim: A perceptual voice similarity dataset

Junseok Ahn, Youkyum Kim, Yeunju Choi, Doyeop Kwak, Ji-Hoon Kim

Analysis of Pathological Speech – Pitfalls along the Way

Elmar Noeth

Codec-ASR: Training Performant Automatic Speech Recognition Systems with Discrete Speech Representations

Kunal Dhawan, Nithin Rao Koluguri, Ante Jukić, Ryan Langman, Jagadeesh Balam

MMM: Multi-Layer Multi-Residual Multi-Stream Discrete Speech Representation from Self-supervised Learning Model

Jiatong Shi, Xutai Ma, Hirofumi Inaguma, Anna Sun, Shinji Watanabe

SingOMD: Singing Oriented Multi-resolution Discrete Representation Construction from Speech Models

Yuxun Tang, Yuning Wu, Jiatong Shi, Qin Jin

The Interspeech 2024 Challenge on Speech Processing Using Discrete Units

Xuankai Chang, Jiatong Shi, Jinchuan Tian, Yuning Wu, Yuxun Tang

How Should We Extract Discrete Audio Tokens from Self-Supervised Models?

Pooneh Mousavi, Jarod Duret, Salah Zaiem, Luca Della Libera, Artem Ploujnikov

TokSing: Singing Voice Synthesis based on Discrete Tokens

Yuning Wu, Chunlei Zhang, Jiatong Shi, Yuxun Tang, Shan Yang

Learn and Don't Forget: Adding a New Language to ASR Foundation Models

Mengjie Qian, Siyuan Tang, Rao Ma, Kate M. Knill, Mark J.F. Gales

Exploring adaptation techniques of large speech foundation models for low-resource ASR: a case study on Northern Sámi

Yaroslav Getman, Tamás Grósz, Katri Hiovain-Asikainen, Mikko Kurimo

Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whisper

Tianyi Xu, Kaixun Huang, Pengcheng Guo, Yu Zhou, Longtao Huang

Adapter pre-training for improved speech recognition in unseen domains using low resource adapter tuning of self-supervised models

Sathvik Udupa, Jesuraj Bandekar, Saurabh Kumar, Deekshitha G, Sandhya B

Interleaved Audio/Audiovisual Transfer Learning for AV-ASR in Low-Resourced Languages

Zhengyang Li, Patrick Blumenberg, Jing Liu, Thomas Graave, Timo Lohrenz

Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling

Kaushal Santosh Bhogale, Deovrat Mehendale, Niharika Parasa, Sathish Kumar Reddy G, Tahir Javed

Improving Whisper's Recognition Performance for Under-Represented Language Kazakh Leveraging Unpaired Speech and Text

Jinpeng Li, Yu Pu, Qi Sun, Wei-Qiang Zhang

Comparing Discrete and Continuous Space LLMs for Speech Recognition

Yaoxun Xu, Shi-Xiong Zhang, Jianwei Yu, Zhiyong Wu, Dong Yu

Interface Design for Self-Supervised Speech Models

Yi-Jen Shih, David Harwath

Wav2vec 2.0 Embeddings Are No Swiss Army Knife -- A Case Study for Multiple Sclerosis

Gábor Gosztolya, Mercedes Vetráb, Veronika Svindt, Judit Bóna, Ildikó Hoffmann

Training Data Augmentation for Dysarthric Automatic Speech Recognition by Text-to-Dysarthric-Speech Synthesis

Wing-Zin Leung, Mattias Cross, Anton Ragni, Stefan Goetze

Learnings from curating a trustworthy, well-annotated, and useful dataset of disordered English speech

Pan-Pan Jiang, Jimmy Tobin, Katrin Tomanek, Robert MacDonald, Katie Seaver

Fine-Tuning Automatic Speech Recognition for People with Parkinson's: An Effective Strategy for Enhancing Speech Technology Accessibility

Xiuwen Zheng, Bornali Phukon, Mark Hasegawa-Johnson

Acoustic changes in speech prosody produced by children with autism after robot-assisted speech training

Si Chen, Bruce Xiao Wang, Yitian Hong, Fang Zhou, Angel Chan

PARAN: Variational Autoencoder-based End-to-End Articulation-to-Speech System for Speech Intelligibility

Seyun Um, Doyeon Kim, Hong-Goo Kang

Towards Improving NAM-to-Speech Synthesis Intelligibility using Self-Supervised Speech Models

Neil Shah, Shirish Karande, Vineet Gandhi

Enhancing Voice Wake-Up for Dysarthria: Mandarin Dysarthria Speech Corpus Release and Customized System Design

Ming Gao, Hang Chen, Jun Du, Xin Xu, Hongxiao Guo

Multimodal Digital Biomarkers for Longitudinal Tracking of Speech Impairment Severity in ALS: An Investigation of Clinically Important Differences

Michael Neumann, Hardik Kothare, Jackson Liscombe, Emma C.L. Leschly, Oliver Roesler

A Comparative Analysis of Federated Learning for Speech-Based Cognitive Decline Detection

Stefan Kalabakov, Monica Gonzalez-Machorro, Florian Eyben, Bjorn W. Schuller, Bert Arnrich

A Cluster-based Personalized Federated Learning Strategy for End-to-End ASR of Dementia Patients

Wei-Tung Hsu, Chin-Po Chen, Yun-Shao Lin, Chi-Chun Lee

Developing vocal system impaired patient-aimed voice quality assessment approach using ASR representation-included multiple features

Shaoxiang Dang, Tetsuya Matsumoto, Yoshinori Takeuchi, Takashi Tsuboi, Yasuhiro Tanaka

Leveraging Phonemic Transcription and Whisper toward Clinically Significant Indices for Automatic Child Speech Assessment

Yeh-Sheng Lin, Shu-Chuan Tseng, Jyh-Shing Roger Jang

Clever Hans Effect Found in Automatic Detection of Alzheimer's Disease through Speech

Yin-Long Liu, Rui Feng, Jia-Hong Yuan, Zhen-Hua Ling

Towards Self-Attention Understanding for Automatic Articulatory Processes Analysis in Cleft Lip and Palate Speech

Ilja Baumann, Dominik Wagner, Maria Schuster, Korbinian Riedhammer, Elmar Noeth

Speech Recognition Models are Strong Lip-readers

K R Prajwal, Triantafyllos Afouras, Andrew Zisserman

Whisper-Flamingo: Integrating Visual Features into Whisper for Audio-Visual Speech Recognition and Translation

Andrew Rouditchenko, Yuan Gong, Samuel Thomas, Leonid Karlinsky, Hilde Kuehne

MM-KWS: Multi-modal Prompts for Multilingual User-defined Keyword Spotting

Zhiqi Ai, Zhiyong Chen, Shugong Xu

Spoken-to-written text conversion with Large Language Model

HyunJung Choi, Muyeol Choi, Yohan Lim, MinKyu Lee, Seonhui Kim

MaLa-ASR: Multimedia-Assisted LLM-Based ASR

Guanrou Yang, Ziyang Ma, Fan Yu, Zhifu Gao, Shiliang Zhang

HuBERT-EE: Early Exiting HuBERT for Efficient Speech Recognition

Ji Won Yoon, Beom Jun Woo, Nam Soo Kim

Qifusion-Net: Layer-adapted Stream/Non-stream Model for End-to-End Multi-Accent Speech Recognition

Jinming Chen, Jingyi Fang, Yuanzhong Zheng, Yaoxuan Wang, Haojun Fei

Speaker-Smoothed kNN Speaker Adaptation for End-to-End ASR

Shaojun Li, Daimeng Wei, Hengchao Shang, Jiaxin Guo, Zongyao Li

Whisper Multilingual Downstream Task Tuning Using Task Vectors

Ji-Hun Kang, Jae-Hong Lee, Mun-Hak Lee, Joon-Hyuk Chang

Leveraging Adapter for Parameter-Efficient ASR Encoder

Kyuhong Shim, Jinkyu Lee, Hyunjae Kim

AdaRA: Adaptive Rank Allocation of Residual Adapters for Speech Foundation Model

Zhouyuan Huo, Dongseong Hwang, Gan Song, Khe Chai Sim, Weiran Wang

Shared-Adapters: A Novel Transformer-based Parameter Efficient Transfer Learning Approach For Children’s Automatic Speech Recognition

Thomas Rolland, Alberto Abad

Towards End-to-End Unified Recognition for Mandarin and Cantonese

Meiling Chen, Pengjie Liu, Heng Yang, Haofeng Wang

Learning from memory-based models

Rhiannon Mogridge, Anton Ragni

LingWav2Vec2: Linguistic-augmented wav2vec 2.0 for Vietnamese Mispronunciation Detection

Tuan Nguyen, Huy Dat Tran

The Processing of Stress in End-to-End Automatic Speech Recognition Models

Martijn Bentum, Louis ten Bosch, Tom Lentz

SELM: Enhancing Speech Emotion Recognition for Out-of-Domain Scenarios

Hazim Bukhari, Soham Deshmukh, Hira Dhamyal, Bhiksha Raj, Rita Singh

Exploring Multilingual Unseen Speaker Emotion Recognition: Leveraging Co-Attention Cues in Multitask Learning

Arnav Goel, Medha Hira, Anubha Gupta

Cross-modal Features Interaction-and-Aggregation Network with Self-consistency Training for Speech Emotion Recognition

Ying Hu, Huamin Yang, Hao Huang, Liang He

MinSpeech: A Corpus of Southern Min Dialect for Automatic Speech Recognition

Jiayan Lin, Shenghui Lu, Hukai Huang, Wenhao Guan, Binbin Xu

LearnerVoice: A Dataset of Non-Native English Learners’ Spontaneous Speech

Haechan Kim, Junho Myung, Seoyoung Kim, Sungpah Lee, Dongyeop Kang

LAHAJA: A Robust Multi-accent Benchmark for Evaluating Hindi ASR Systems

Tahir Javed, Janki Nawale, Sakshi Joshi, Eldho George, Kaushal Bhogale

Performant ASR Models for Medical Entities in Accented Speech

Tejumade Afonja, Tobi Olatunji, Sewade Ogun, Naome A. Etori, Abraham Owodunni

Improving Self-supervised Pre-training using Accent-Specific Codebooks

Darshan Prabhu, Abhishek Gupta, Omkar Nitsure, Preethi Jyothi, Sriram Ganapathy

Multi-Modal Automatic Prosody Annotation with Contrastive Pretraining of Speech-Silence and Word-Punctuation

Jinzuomu Zhong, Yang Li, Hui Huang, Korin Richmond, Jie Liu

Towards Expressive Zero-Shot Speech Synthesis with Hierarchical Prosody Modeling

Yuepeng Jiang, Tao Li, Fengyu Yang, Lei Xie, Meng Meng

A Human-in-the-Loop Approach to Improving Cross-Text Prosody Transfer

Himanshu Maurya, Atli Sigurgeirsson

Total-Duration-Aware Duration Modeling for Text-to-Speech Systems

Sefik Emre Eskimez, Xiaofei Wang, Manthan Thakker, Chung-Hsien Tsai, Canrun Li

Should you use a probabilistic duration model in TTS? Probably! Especially for spontaneous speech

Shivam Mehta, Harm Lameris, Rajiv Punmiya, Jonas Beskow, Eva Szekely

Word-level Text Markup for Prosody Control in Speech Synthesis

Yuliya Korotkova, Ilya Kalinovskiy, Tatiana Vakhrusheva

MaskSR: Masked Language Model for Full-band Speech Restoration

Xu Li, Qirui Wang, Xiaoyu Liu

MultiStage Speech Bandwidth Extension with Flexible Sampling Rate Control

Ye-Xin Lu, Yang Ai, Zheng-Yan Sheng, Zhen-Hua Ling

SWiBE: A Parameterized Stochastic Diffusion Process for Noise-Robust Bandwidth Expansion

Yin-Tse Lin, Shreya G. Upadhyay, Bo-Hao Su, Chi-Chun Lee