Interspeech 2024

1,065 papers

A layer-wise analysis of Mandarin and English suprasegmentals in SSL speech models

Anton de la Fuente, Dan Jurafsky

A Comparative Analysis of Bilingual and Trilingual Wav2Vec Models for Automatic Speech Recognition in Multilingual Oral History Archives

Jan Lehečka, Josef V. Psutka, Luboš Šmídl, Pavel Ircing, Josef Psutka

DualPure: An Efficient Adversarial Purification Method for Speech Command Recognition

Hao Tan, Xiaochen Liu, Huan Zhang, Junjian Zhang, Yaguan Qian

Inclusive ASR for Disfluent Speech: Cascaded Large-Scale Self-Supervised Learning with Targeted Fine-Tuning and Data Augmentation

Dena Mujtaba, Nihar R. Mahapatra, Megan Arney, J. Scott Yaruss, Caryn Herring

On Disfluency and Non-lexical Sound Labeling for End-to-end Automatic Speech Recognition

Peter Mihajlik, Yan Meng, Mate S Kadar, Julian Linke, Barbara Schuppler

CrisperWhisper: Accurate Timestamps on Verbatim Speech Transcriptions

Mario Zusag, Laurin Wagner, Bernhad Thallinger

A Multitask Training Approach to Enhance Whisper with Open-Vocabulary Keyword Spotting

Yuang Li, Min Zhang, Chang Su, Yinglu Li, Xiaosong Qiao

Improving Domain-Specific ASR with LLM-Generated Contextual Descriptions

Jiwon Suh, Injae Na, Woohwan Jung

Improving Multilingual ASR Robustness to Errors in Language Input

Brady Houston, Omid Sadjadi, Zejiang Hou, Srikanth Vishnubhotla, Kyu J. Han

MSR-86K: An Evolving, Multilingual Corpus with 86,300 Hours of Transcribed Audio for Speech Recognition Research

Song Li, Yongbin You, Xuezhi Wang, Zhengkun Tian, Ke Ding

M2ASR: Multilingual Multi-task Automatic Speech Recognition via Multi-objective Optimization

A F M Saif, Lisha Chen, Xiaodong Cui, Songtao Lu, Brian Kingsbury

Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognition

Andres Pineiro-Martin, Carmen Garcia-Mateo, Laura Docio-Fernandez, Maria del Carmen Lopez-Perez, Georg Rehm

ML-SUPERB 2.0: Benchmarking Multilingual Speech Models Across Modeling Constraints, Languages, and Datasets

Jiatong Shi, Shih-Heng Wang, William Chen, Martijn Bartelds, Vanya Bannihatti Kumar

Continual Learning Optimizations for Auto-regressive Decoder of Multilingual ASR systems

Chin Yuen Kwok, Jia Qi Yip, Eng Siong Chng

Lifelong Learning MOS Prediction for Synthetic Speech Quality Evaluation

Félix Saget, Meysam Shamsi, Marie Tahon

Uncertainty-Aware Mean Opinion Score Prediction

Hui Wang, Shiwan Zhao, Jiaming Zhou, Xiguang Zheng, Haoqin Sun

What do people hear? Listeners’ Perception of Conversational Speech

Adaeze Adigwe, Sarenne Wallbridge, Simon King

Assessing the impact of contextual framing on subjective TTS quality

Jens Edlund, Christina Tånnander, Sébastien Le Maguer, Petra Wagner

Enhancing Out-of-Vocabulary Performance of Indian TTS Systems for Practical Applications through Low-Effort Data Strategies

Srija Anand, Praveen Srinivasa Varadhan, Ashwin Sankar, Giri Raju, Mitesh M. Khapra

SVSNet+: Enhancing Speaker Voice Similarity Assessment Models with Representations from Speech Foundation Models

Chun Yin, Tai-Shih Chi, Yu Tsao, Hsin-Min Wang

Guided conditioning with predictive network on score-based diffusion model for speech enhancement

Dail Kim, Da-Hee Yang, Donghyun Kim, Joon-Hyuk Chang, Jeonghwan Choi

Pre-training Feature Guided Diffusion Model for Speech Enhancement

Yiyuan Yang, Niki Trigoni, Andrew Markham

Thunder : Unified Regression-Diffusion Speech Enhancement with a Single Reverse Step using Brownian Bridge

Thanapat Trachu, Chawan Piansaddhayanon, Ekapol Chuangsuwanich

Schrödinger Bridge for Generative Speech Enhancement

Ante Jukić, Roman Korostik, Jagadeesh Balam, Boris Ginsburg

Genhancer: High-Fidelity Speech Enhancement via Generative Modeling on Discrete Codec Tokens

Haici Yang, Jiaqi Su, Minje Kim, Zeyu Jin

Universal Score-based Speech Enhancement with High Content Preservation

Robin Scheibler, Yusuke Fujita, Yuma Shirahata, Tatsuya Komatsu

Efficient Audio Captioning with Encoder-Level Knowledge Distillation

Xuenan Xu, Haohe Liu, Mengyue Wu, Wenwu Wang, Mark D. Plumbley

ParaCLAP – Towards a general language-audio model for computational paralinguistic tasks

Xin Jing, Andreas Triantafyllopoulos, Björn Schuller

Efficient CNNs with Quaternion Transformations and Pruning for Audio Tagging

Aryan Chaudhary, Arshdeep Singh, Vinayak Abrol, Mark D. Plumbley

Streaming Audio Transformers for Online Audio Tagging

Heinrich Dinkel, Zhiyong Yan, Yongqing Wang, Junbo Zhang, Yujun Wang

Audio-text Retrieval with Transformer-based Hierarchical Alignment and Disentangled Cross-modal Representation

Yifei Xin, Zhihong Zhu, Xuxin Cheng, Xusheng Yang, Yuexian Zou

Enhancing Automated Audio Captioning via Large Language Models with Optimized Audio Encoding

Jizhong Liu, Gang Li, Junbo Zhang, Heinrich Dinkel, Yongqing Wang

PFCA-Net: Pyramid Feature Fusion and Cross Content Attention Network for Automated Audio Captioning

Jianyuan Sun, Wenwu Wang, Mark D. Plumbley

DGPN: A Dual Graph Prototypical Network for Few-Shot Speech Spoofing Algorithm Recognition

Zirui Ge, Xinzhou Xu, Haiyan Guo, Tingting Wang, Zhen Yang

Interpretable Temporal Class Activation Representation for Audio Spoofing Detection

Menglu Li, Xiao-Ping Zhang

SecureSpectra: Safeguarding Digital Identity from Deep Fake Threats via Intelligent Signatures

Oguzhan Baser, Kaan Kale, Sandeep P. Chinchali

Revisiting and Improving Scoring Fusion for Spoofing-aware Speaker Verification Using Compositional Data Analysis

Xin Wang, Tomi Kinnunen, Kong Aik Lee, Paul-Gauthier Noé, Junichi Yamagishi

How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?

TIANCHI LIU, Lin Zhang, Rohan Kumar Das, Yi Ma, Ruijie Tao

Source Tracing of Audio Deepfake Systems

Nicholas Klein, Tianxiang Chen, Hemlata Tak, Ricardo Casal, Elie Khoury

Exploring Gender-Specific Speech Patterns in Automatic Suicide Risk Assessment

Maurice Gerczuk, Shahin Amiriparian, Justina Lutz, Wolfgang Strube, Irina Papazova

Modelling Lexical Characteristics of the Healthy Aging Population: A Corpus-Based Study

Han Kunmei

Learning Representation of Therapist Empathy in Counseling Conversation Using Siamese Hierarchical Attention Network

Dehua Tao, Tan Lee, Harold Chui, Sarah Luk

Detecting Empathy in Speech

Run Chen, Haozhe Chen, Anushka Kulkarni, Eleanor Lin, Linda Pang

Multimodal Belief Prediction

John Murzaku, Adil Soubki, Owen Rambow

A Functional Trade-off between Prosodic and Semantic Cues in Conveying Sarcasm

Zhu Li, Xiyuan Gao, Yuqing Zhang, Shekhar Nayak, Matt Coler

How rhythm metrics are linked to produced and perceived speaker charisma

Oliver Niebuhr, Nafiseh Taghva

Speech Emotion Recognition with Multi-level Acoustic and Semantic Information Extraction and Interaction

Yuan Gao, Hao Shi, Chenhui Chu, Tatsuya Kawahara

An Effective Local Prototypical Mapping Network for Speech Emotion Recognition

Yuxuan Xi, Yan Song, Lirong Dai, Haoyu Song, Ian McLoughlin

Confidence-aware Hypothesis Transfer Networks for Source-Free Cross-Corpus Speech Emotion Recognition

Jincen Wang, Yan Zhao, Cheng Lu, Hailun Lian, Hongli Chang

Unsupervised Domain Adaptation for Speech Emotion Recognition using K-Nearest Neighbors Voice Conversion

Pravin Mote, Berrak Sisman, Carlos Busso

Reinforcement Learning based Data Augmentation for Noise Robust Speech Emotion Recognition

Sumit Ranjan, Rupayan Chakraborty, Sunil Kumar Kopparapu

Mandarin T3 Production by Chinese and Japanese Native Speakers

Qi Wu

Exploring Impact of Pausing and Lexical Stress Patterns on L2 English Comprehensibility in Real Time

Sylvain Coulange, Tsuneo Kato, Solange Rossato, Monica Masperi

Bilingual Rhotic Production Patterns: A Generational Comparison of Spanish-English Bilingual Speakers in Canada

Ioana Colgiu, Laura Spinu, Rajiv Rao, Yasaman Rafat

Analysis of articulatory setting for L1 and L2 English speakers using MRI data

Kevin Huang, Jack Goldberg, Louis Goldstein, Shrikanth Narayanan

Automatic Speech Recognition with parallel L1 and L2 acoustic phone models to evaluate /l/ allophony in L2 English speech production

Anisia Popescu, Lori Lamel, Ioana Vasilescu, Laurence Devillers

Mmm whatcha say? Uncovering distal and proximal context effects in first and second-language word perception using psychophysical reverse correlation

Paige Tuttösí, H. Henny Yeung, Yue Wang, Fenqi Wang, Guillaume Denis

Frontier of Frontend for Conversational Speech Processing

Shoko Araki

EZTalking: English assessment platform for teachers and students

Yu-Sheng Tsao, Yung-Chang Hsu, Jiun-Ting Li, Siang-Hong Weng, Tien-Hong Lo

CreakVC: a voice conversion tool for modulating creaky voice

Harm Lameris, Joakim Gustafson, Eva Szekely

Reliable dialogue system for facilitating student-counselor communication

Mahdin Rohmatillah, Bryan Gautama Ngo, Willianto Sulaiman, Po-Chuan Chen, Jen-Tzung Chien

ConnecTone: a modular AAC system prototype with contextual generative text prediction and style-adaptive conversational TTS

Juliana Francis, Eva Szekely, Joakim Gustafson

Multi-speaker and multi-dialectal Catalan TTS models for video gaming

Alex Peiró-Lilja, José Giraldo, Martí Llopart-Font, Carme Armentano-Oller, Baybars Kulebi

Mobile PresenTra: NICT fast neural text-to-speech system on smartphones with incremental inference of MS-FC-HiFi-GAN for law-latency synthesis

Takuma Okamoto, Yamato Ohtani, Hisashi Kawai

Stress transfer in speech-to-speech machine translation

Sai Akarsh, Vamshiraghusimha Narasinga, Anil Kumar Vuppala

VoxFlow AI: wearable voice converter for atypical speech

Grzegorz P. Mika, Konrad Zieli´nski, Paweł Cyrta, Marek Grzelec

A powerful and modern AAC composition tool for impaired speakers

Aanchan Mohan, Monideep Chakraborti, Katelyn Eng, Nailia Kushaeva, Mirjana Prpa

Faster Vocoder: a multi threading approach to achieve low latency during TTS Inference

Vishal Gourav, Ankit Tyagi, Phanindra Mankale

Production of phrases by mechanical models of the human vocal tract

Takayuki Arai, Ryohei Suzuki, Chandler Earp, Shinya Tsuji, Keiko Ochi

Multilingual Speech and Language Analysis for the Assessment of Mild Cognitive Impairment: Outcomes from the Taukadial Challenge

Paula Andrea Pérez-Toro, Tomás Arias-Vergara, Philipp Klumpp, Tobias Weise, Maria Schuster

Translingual Language Markers for Cognitive Assessment from Spontaneous Speech

Bao Hoang, Yijiang Pang, Hiroko Dodge, Jiayu Zhou

Leveraging Universal Speech Representations for Detecting and Assessing the Severity of Mild Cognitive Impairment Across Languages

Anna Favaro, Tianyu Cao, Najim Dehak, Laureano Moro-Velazquez

The Interspeech 2024 TAUKADIAL Challenge: Multilingual Mild Cognitive Impairment Detection with Multimodal Approach

Benjamin Barrera-Altuna, Daeun Lee, Zaima Zarnaz, Jinyoung Han, Seungbae Kim

Pre-trained Feature Fusion and Matching for Mild Cognitive Impairment Detection

Junwen Duan, Fangyuan Wei, Hong-Dong Li, Jin Liu

Combining Acoustic Feature Sets for Detecting Mild Cognitive Impairment in the Interspeech'24 TAUKADIAL Challenge

Gábor Gosztolya, László Tóth

Cognitive Insights Across Languages: Enhancing Multimodal Interview Analysis

David Ortiz-Perez, José García-Rodríguez, David Tomás

Connected Speech-Based Cognitive Assessment in Chinese and English

Saturnino Luz, Sofia De La Fuente Garcia, Fasih Haider, Davida Fromm, Brian MacWhinney

Automatic Longitudinal Investigation of Multiple Sclerosis Subjects

Gábor Gosztolya, Veronika Svindt, Judit Bóna, Ildikó Hoffmann

YOLO-Stutter: End-to-end Region-Wise Speech Dysfluency Detection

Xuanru Zhou, Anshul Kashyap, Steve Li, Ayati Sharma, Brittany Morin

DysArinVox: DYSphonia & DYSarthria mandARIN speech corpus

Haojie Zhang, Tao Zhang, Ganjun Liu, Dehui Fu, Xiaohui Hou

Contrastive Learning Approach for Assessment of Phonological Precision in Patients with Tongue Cancer Using MRI Data

Tomás Arias-Vergara, Paula Andrea Pérez-Toro, Xiaofeng Liu, Fangxu Xing, Maureen Stone

Multimodal Continuous Fingerspelling Recognition via Visual Alignment Learning

Katerina Papadimitriou, Gerasimos Potamianos

Segmental and Suprasegmental Speech Foundation Models for Classifying Cognitive Risk Factors: Evaluating Out-of-the-Box Performance

Si-Ioi Ng, Lingfeng Xu, Kimberly D. Mueller, Julie Liss, Visar Berisha

A Cross-Attention Layer coupled with Multimodal Fusion Methods for Recognizing Depression from Spontaneous Speech

Loukas Ilias, Dimitris Askounis

Cascaded Transfer Learning Strategy for Cross-Domain Alzheimer's Disease Recognition through Spontaneous Speech

Guanlin Chen, Yun Jin

Improving Speech-Based Dysarthria Detection using Multi-task Learning with Gradient Projection

Yan Xiong, Visar Berisha, Julie Liss, Chaitali Chakrabarti

Whister: Using Whisper’s representations for Stuttering detection

Vrushank Changawala, Frank Rudzicz

Automatic Detection of Hearing Loss from Children's Speech using wav2vec 2.0 Features

Jessica Monaghan, Arun Sebastian, Nicky Chong-White, Vicky Zhang, Vijayalakshmi Easwar

Auditory Spatial Attention Detection Based on Feature Disentanglement and Brain Connectivity-Informed Graph Neural Networks

Yixiang Niu, Ning Chen, Hongqing Zhu, Zhiying Zhu, Guangqiang Li

Signal processing algorithm effective for sound quality of hearing loss simulators

Toshio Irino, Shintaro Doan, Minami Ishikawa

Production of fricative consonants in French-speaking children with cochlear implants and typical hearing: acoustic and phonological analyses.

Sophie Fagniart, Brigitte Charlier, Véronique Delvaux, Bernard Harmegnies, Anne Huberlant

Evaluating a 3-factor listener model for prediction of speech intelligibility to hearing-impaired listeners

Mark Huckvale, Gaston Hilkhuysen

SyncVSR: Data-Efficient Visual Speech Recognition with End-to-End Crossmodal Audio Token Synchronization

Young Jin Ahn, Jungwoo Park, Sangha Park, Jonghyun Choi, Kee-eung Kim

Automatic Assessment of Speech Production Skills for Children with Cochlear Implants Using Wav2Vec2.0 Acoustic Embeddings

Seonwoo Lee, Sunhee Kim, Minhwa Chung

Towards Speech-to-Pictograms Translation

Cécile Macaire, Chloé Dion, Didier Schwab, Benjamin Lecouteux, Emmanuelle Esperança-Rodier

Contrastive Feedback Mechanism for Simultaneous Speech Translation

Haotian Tan, Sakriani Sakti

Lightweight Audio Segmentation for Long-form Speech Translation

Jaesong Lee, Soyoon Kim, Hanbyul Kim, Joon Son Chung

Sign Value Constraint Decomposition for Efficient 1-Bit Quantization of Speech Translation Tasks

Nan Chen, Yonghe Wang, Feilong Bao

Diffusion Synthesizer for Efficient Multilingual Speech to Speech Translation

Nameer Hirschkind, Xiao Yu, Mahesh Kumar Nandwana, Joseph Liu, Eloi DuBois

Investigating Decoder-only Large Language Models for Speech-to-text Translation

Chao-Wei Huang, Hui Lu, Hongyu Gong, Hirofumi Inaguma, Ilia Kulikov