Interspeech 2024

1,065 papers

CaptainA self-study mobile app for practising speaking: task completion assessment and feedback with generative AI

Nhan Phan, Anna von Zansen, Maria Kautonen, Tamás Grósz, Mikko Kurimo

Visualization for improving foreign language pronunciation

Charlotte Yoder, Karrie Karahalios, Mark Hasegawa-Johnson, Shreyansh Agrawal

The speech motor chaining web app for speech motor learning

Jonathan L Preston, Nina R Benway, Nathan Prestopnik, Nathan Preston

CALL system using pitch-accent feature representations reflecting listeners’ subjective adequacy

Ikuyo Masuda-Katsuse, Ayako Shirose

Visual scene display application for augmentative and alternative communication

Karthik Venkat Sridaran, Raja Praveen, Reuben T Varghese, Ajish K Abraham, Shankar R

A ChatGPT-based oral Q&A practice system for first-time student participants in international conferences

Mayuko Aiba, Daisuke Saito, Nobuaki Minematsu

Speech enabled visual acuity test

Boon Peng Yap, Kok Liang Tan, Zhenghao Li, Rong Tong

IIITH Ucchar e-Sudharak: an automatic English pronunciation corrector for school-going children with a teacher in the loop

Meenakshi Sirigiraju, Arjun Rajasekar, Abhishikth Meejuri, Chiranjeevi Yarra

Exploring Speech Foundation Models for Speaker Diarization in Child-Adult Dyadic Interactions

Anfeng Xu, Kevin Huang, Tiantian Feng, Lue Shen, Helen Tager-Flusberg

Mixed Children/Adult/Childrenized Fine-Tuning for Children’s ASR: How to Reduce Age Mismatch and Speaking Style Mismatch

Thomas Graave, Zhengyang Li, Timo Lohrenz, Tim Fingscheidt

Improving child speech recognition with augmented child-like speech

Yuanyuan Zhang, Zhengjun Yue, Tanvina Patel, Odette Scharenborg

Introduction To Partial Fine-tuning: A Comprehensive Evaluation Of End-to-end Children’s Automatic Speech Recognition Adaptation

Thomas Rolland, Alberto Abad

Benchmarking Children's ASR with Supervised and Self-supervised Speech Foundation Models

Ruchao Fan, Natarajan Balaji Shankar, Abeer Alwan

Self-Supervised Models for Phoneme Recognition: Applications in Children's Speech for Reading Learning

Lucas Block Medin, Thomas Pellegrini, Lucile Gelin

Enhancing Child Vocalization Classification with Phonetically-Tuned Embeddings for Assisting Autism Diagnosis

Jialu Li, Mark Hasegawa-Johnson, Karrie Karahalios

Automatic Evaluation of a Sentence Memory Test for Preschool Children

Ilja Baumann, Nicole Unger, Dominik Wagner, Korbinian Riedhammer, Tobias Bocklet

Reading Miscue Detection in Primary School through Automatic Speech Recognition

Lingyun Gao, Cristian Tejedor-García, Helmer Strik, Catia Cucchiarini

Bridging Child-Centered Speech Language Identification and Language Diarization via Phonetics

Yujia Wang, Hexin Liu, Leibny Paola Garcia

Children’s Speech Recognition through Discrete Token Enhancement

Vrunda N. Sukhadia, Shammur Absar Chowdhury

Examining Vocal Tract Coordination in Childhood Apraxia of Speech with Acoustic-to-Articulatory Speech Inversion Feature Sets

Nina R. Benway, Jonathan L. Preston, Carol Espy-Wilson

How Does Alignment Error Affect Automated Pronunciation Scoring in Children's Speech?

Prad Kadambi, Tristan Mahr, Lucas Annear, Henry Nomeland, Julie Liss

Training speech-breathing coordination in computer-assisted reading

Delphine Charuau, Andrea Briglia, Erika Godde, Gérard Bailly

Preliminary Investigation of Psychometric Properties of a Novel Multimodal Dialog Based Affect Production Task in Children and Adolescents with Autism

Carly Demopoulos, Linnea Lampinen, Cristian Preciado, Hardik Kothare, Vikram Ramanarayanan

Large Language Models for Dysfluency Detection in Stuttered Speech

Dominik Wagner, Sebastian P. Bayerl, Ilja Baumann, Elmar Noeth, Korbinian Riedhammer

MMSD-Net: Towards Multi-modal Stuttering Detection

Liangyu Nie, Sudarsana Reddy Kadiri, Ruchit Agrawal

Prosody of speech production in latent post-stroke aphasia

Cong Zhang, Tong Li, Gayle DeDe, Christos Salis

Analyzing Speech Motor Movement using Surface Electromyography in Minimally Verbal Adults with Autism Spectrum Disorder

Wazeer Zulfikar, Nishat Protyasha, Camila Canales, Heli Patel, James Williamson

AS-70: A Mandarin stuttered speech dataset for automatic speech recognition and stuttering event detection

Rong Gong, Hongfei Xue, Lezhi Wang, Xin Xu, Qisheng Li

Missingness-resilient Video-enhanced Multimodal Disfluency Detection

Payal Mohapatra, Shamika Likhite, Subrata Biswas, Bashima Islam, Qi Zhu

Leveraging Speech Data Diversity to Document Indigenous Heritage and Culture

Allahsera Tapo, Eric Le Ferrand, Zoey Liu, Christopher Homan, Emily Prud'hommeaux

Few-Shot Keyword-Incremental Learning with Total Calibration

Ilseok Kim, Ju-Seok Seong, Joon-Hyuk Chang

GPA: Global and Prototype Alignment for Audio-Text Retrieval

Yuxin Xie, Zhihong Zhu, Xianwei Zhuang, Liming Liang, Zhichang Wang

2DP-2MRC: 2-Dimensional Pointer-based Machine Reading Comprehension Method for Multimodal Moment Retrieval

Jiajun He, Tomoki Toda

Pretraining End-to-End Keyword Search with Automatically Discovered Acoustic Units

Bolaji Yusuf, Jan Honza Cernocky, Murat Saraclar

Few-Shot Keyword Spotting from Mixed Speech

Junming Yuan, Ying Shi, Lantian Li, Dong Wang, Askar Hamdulla

Balanced-Wav2Vec: Enhancing Stability and Robustness of Representation Learning Through Sample Reweighting Techniques

Mun-Hak Lee, Jae-Hong Lee, Dohee Kim, Ye-Eun Ko, Joon-Hyuk Chang

MS-HuBERT: Mitigating Pre-training and Inference Mismatch in Masked Language Modelling methods for learning Speech Representations

Hemant Yadav, Sunayana Sitaram, Rajiv Ratn Shah

Self-Supervised Learning for ASR Pre-Training with Uniquely Determined Target Labels and Controlling Cepstrum Truncation for Speech Augmentation

Akihiro Kato, Hiroyuki Nagano, Kohei Chike, Masaki Nose

What happens in continued pre-training? Analysis of self-supervised speech models with continued pre-training for colloquial Finnish ASR

Yaroslav Getman, Tamás Grósz, Mikko Kurimo

wTIMIT2mix: A Cocktail Party Mixtures Database to Study Target Speaker Extraction for Normal and Whispered Speech

Marvin Borsdorf, Zexu Pan, Haizhou Li, Tanja Schultz

SpeakerBeam-SS: Real-time Target Speaker Extraction with Lightweight Conv-TasNet and State Space Modeling

Hiroshi Sato, Takafumi Moriya, Masato Mimura, Shota Horiguchi, Tsubasa Ochiai

Bird Whisperer: Leveraging Large Pre-trained Acoustic Model for Bird Call Classification

Muhammad Umer Sheikh, Hassan Abid, Bhuiyan Sanjid Shafique, Asif Hanif, Muhammad Haris Khan

Towards Robust Few-shot Class Incremental Learning in Audio Classification using Contrastive Representation

Riyansha Singh, Parinita Nema, Vinod K Kurmi

DGSRN: Noise-Robust Speech Recognition Method with Dual-Path Gated Spectral Refinement Network

Wenjun Wang, Shangbin Mo, Ling Dong, Zhengtao Yu, Junjun Guo

Hold Me Tight: Stable Encoder-Decoder Design for Speech Enhancement

Daniel Haider, Felix Perfler, Vincent Lostanlen, Martin Ehler, Peter Balazs

Enhanced ASR Robustness to Packet Loss with a Front-End Adaptation Network

Yehoshua Dissen, Shiry Yonash, Israel Cohen, Joseph Keshet

NOTSOFAR-1 Challenge: New Datasets, Baseline, and Tasks for Distant Meeting Transcription

Alon Vinnikov, Amir Ivry, Aviv Hurvitz, Igor Abramovski, Sharon Koubi

Transcription-Free Fine-Tuning of Speech Separation Models for Noisy and Reverberant Multi-Speaker Automatic Speech Recognition

William Ravenscroft, George Close, Stefan Goetze, Thomas Hain, Mohammad Soleymanpour

Multi-Channel Multi-Speaker ASR Using Target Speaker’s Solo Segment

Yiwen Shao, Shi-Xiong Zhang, Yong Xu, Meng Yu, Dong Yu

RIR-SF: Room Impulse Response Based Spatial Feature for Target Speech Recognition in Multi-Channel Multi-Speaker Scenarios

Yiwen Shao, Shi-Xiong Zhang, Dong Yu

X-E-Speech: Joint Training Framework of Non-Autoregressive Cross-lingual Emotional Text-to-Speech and Voice Conversion

Houjian Guo, Chaoran Liu, Carlos Toshinori Ishi, Hiroshi Ishiguro

XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model

Edresson Casanova, Kelly Davis, Eren Gölge, Görkem Göknar, Iulian Gulea

Seamless Language Expansion: Enhancing Multilingual Mastery in Self-Supervised Models

Jing Xu, Minglin Wu, Xixin Wu, Helen Meng

Improving Multilingual Text-to-Speech with Mixture-of-Language-Experts and Accent Disentanglement

Jing Wu, Ting Chen, Minchuan Chen, Wei Hu, Shaojun Wang

An Initial Investigation of Language Adaptation for TTS Systems under Low-resource Scenarios

Cheng Gong, Erica Cooper, Xin Wang, Chunyu Qiang, Mengzhe Geng

Meta Learning Text-to-Speech Synthesis in over 7000 Languages

Florian Lux, Sarina Meyer, Lyonel Behringer, Frank Zalkow, Phat Do

FVTTS : Face Based Voice Synthesis for Text-to-Speech

Minyoung Lee, Eunil Park, Sungeun Hong

UNIQUE : Unsupervised Network for Integrated Speech Quality Evaluation

Juhwan Yoon, WooSeok Ko, Seyun Um, Sungwoong Hwang, Soojoong Hwang

SpeechBERTScore: Reference-Aware Automatic Evaluation of Speech Generation Leveraging NLP Evaluation Metrics

Takaaki Saeki, Soumi Maiti, Shinnosuke Takamichi, Shinji Watanabe, Hiroshi Saruwatari

Bilingual and Code-switching TTS Enhanced with Denoising Diffusion Model and GAN

Huai-Zhe Yang, Chia-Ping Chen, Shan-Yun He, Cheng-Ruei Li

H4C-TTS: Leveraging Multi-Modal Historical Context for Conversational Text-to-Speech

Donghyun Seong, Joon-Hyuk Chang

Frame-Wise Breath Detection with Self-Training: An Exploration of Enhancing Breath Naturalness in Text-to-Speech

Dong Yang, Tomoki Koriyama, Yuki Saito

Enabling Conversational Speech Synthesis using Noisy Spontaneous Data

Liisa Rätsep, Rasmus Lellep, Mark Fishel

Towards a General-Purpose Model of Perceived Pragmatic Similarity

Nigel G. Ward, Andres Segura, Alejandro Ceballos, Divette Marco

Zero-shot Out-of-domain is No Joke: Lessons Learned in the VoiceMOS 2023 MOS Prediction Challenge

Marie Kunešová, Jan Lehečka, Josef Michálek, Jindřich Matoušek, Jan Švec

FLY-TTS: Fast, Lightweight and High-Quality End-to-End Text-to-Speech Synthesis

Yinlin Guo, Yening Lv, Jinqiao Dou, Yan Zhang, Yuehai Wang

Neural ATSM: Fully Neural Network-based Adaptive Time-Scale Modification Using Sentence-Specific Dynamic Control

Jaeuk Lee, Sohee Jang, Joon-Hyuk Chang

PPPR: Portable Plug-in Prompt Refiner for Text to Audio Generation

Shuchen Shi, Ruibo Fu, Zhengqi Wen, Jianhua Tao, Tao Wang

Towards realtime co-speech gestures synthesis using STARGATE

Louis Abel, Vincent Colotte, Slim Ouni

PL-TTS: A Generalizable Prompt-based Diffusion TTS Augmented by Large Language Model

Shuhua Li, Qirong Mao, Jiatong Shi

Speak in the Scene: Diffusion-based Acoustic Scene Transfer toward Immersive Speech Generation

MISEUL KIM, Soo-Whan Chung, Youna Ji, Hong-Goo Kang, Min-Seok Choi

LiteFocus: Accelerated Diffusion Inference for Long Audio Synthesis

Zhenxiong Tan, Xinyin Ma, Gongfan Fang, Xinchao Wang

EARS: An Anechoic Fullband Speech Dataset Benchmarked for Speech Enhancement and Dereverberation

Julius Richter, YI-CHIAO WU, Steven Krenn, Simon Welker, Bunlong Lay

URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancement

Wangyou Zhang, Robin Scheibler, Kohei Saijo, Samuele Cornell, Chenda Li

Personalized Speech Enhancement Without a Separate Speaker Embedding Model

Tanel Parnamaa, Ando Saabas

VoiCor: A Residual Iterative Voice Correction Framework for Monaural Speech Enhancement

Rui Cao, Tianrui Wang, Meng Ge, Andong Li, Longbiao Wang

One-class learning with adaptive centroid shift for audio deepfake detection

Hyun Myung Kim, Kangwook Jang, Hoirin Kim

Genuine-Focused Learning using Mask AutoEncoder for Generalized Fake Audio Detection

Xiaopeng Wang, Ruibo Fu, Zhengqi Wen, Zhiyong Wang, Yuankun Xie

Singing Voice Graph Modeling for SingFake Detection

Xuanjun Chen, Haibin Wu, Roger Jang, Hung-yi Lee

Enhancing Partially Spoofed Audio Localization with Boundary-aware Attention Mechanism

Jiafeng Zhong, Bin Li, Jiangyan Yi

Generalized Source Tracing: Detecting Novel Audio Deepfake Algorithm with Real Emphasis and Fake Dispersion Strategy

Yuankun Xie, Ruibo Fu, Zhengqi Wen, Zhiyong Wang, Xiaopeng Wang

Towards generalisable and calibrated audio deepfake detection with self-supervised representations

Octavian Pascu, Adriana Stan, Dan Oneata, Elisabeta Oneata, Horia Cucu

Blind Zero-Shot Audio Restoration: A Variational Autoencoder Approach for Denoising and Inpainting

Veranika Boukun, Jakob Drefs, Jörg Lücke

DNSMOS Pro: A Reduced-Size DNN for Probabilistic MOS of Speech

Fredrik Cumlin, Xinyu Liang, Victor Ungureanu, Chandan K. A. Reddy, Christian Schüldt

LAFMA: A Latent Flow Matching Model for Text-to-Audio Generation

Wenhao Guan, Kaidi Wang, Wangjin Zhou, Yang Wang, Feng Deng

Enhancing Zero-shot Audio Classification using Sound Attribute Knowledge from Large Language Models

Xuenan Xu, Pingyue Zhang, Ming Yan, Ji Zhang, Mengyue Wu

Leveraging Language Model Capabilities for Sound Event Detection

Hualei Wang, Jianguo Mao, Zhifang Guo, Jiarui Wan, Hong Liu

DiveSound: LLM-Assisted Automatic Taxonomy Construction for Diverse Audio Generation

Baihan Li, Zeyu Xie, Xuenan Xu, Yiwei Guo, Ming Yan

Multi-label Bird Species Classification from Field Recordings using Mel_Graph-GCN Framework

Noumida A, Rajeev Rajan

Fully Few-shot Class-incremental Audio Classification Using Expandable Dual-embedding Extractor

Yongjie Si, Yanxiong Li, Jialong Li, Jiaxin Tan, Qianhua He

CtrSVDD: A Benchmark Dataset and Baseline Analysis for Controlled Singing Voice Deepfake Detection

Yongyi Zang, Jiatong Shi, You Zhang, Ryuichi Yamamoto, Jionghao Han

Multimodal Large Language Models with Fusion Low Rank Adaptation for Device Directed Speech Detection

Shruti Palaskar, Ognjen Rudovic, Sameer Dharur, Florian Pesce, Gautam Krishna

Generalized Fake Audio Detection via Deep Stable Learning

Zhiyong Wang, Ruibo Fu, Zhengqi Wen, Yuankun Xie, Yukun Liu

Comparative Analysis of Personalized Voice Activity Detection Systems: Assessing Real-World Effectiveness

Sai Srujana Buddi, Satyam Kumar, Utkarsh Sarawgi, Vineet Garg, Shivesh Ranjan

Measuring acoustic dissimilarity of hierarchical markers in task-oriented dialogue with MFCC-based dynamic time warping

Natalia Morozova, Guanghao You, Sabine Stoll, Adrian Bangerter

Predicting Heart Activity from Speech using Data-driven and Knowledge-based features

Gasser Elbanna, Zohreh Mostaani, Mathew Magimai.-Doss

Explainable by-design Audio Segmentation through Non-Negative Matrix Factorization and Probing

Martin Lebourdais, Théo Mariotte, Antonio Almudévar, Marie Tahon, Alfonso Ortega

Robust Laughter Segmentation with Automatic Diverse Data Synthesis

Taisei Omine, Kenta Akita, Reiji Tsuruno

ElasticAST: An Audio Spectrogram Transformer for All Length and Resolutions

Jiu Feng, Mehmet Hamza Erol, Joon Son Chung, Arda Senocak

LungAdapter: Efficient Adapting Audio Spectrogram Transformer for Lung Sound Classification

Li Xiao, Lucheng Fang, Yuhong Yang, Weiping Tu