Interspeech 2024

1,065 papers

Bingliang Zhao, Jiangping Kong, Xiyu Wu

Effects of talker and playback rate of reverberation-induced speech on speech intelligibility of older adults

Nao Hodoshima

Listeners' F0 preferences in quiet and stationary noise

Olympia Simantiraki, Martin Cooke

Depression Enhances Internal Inconsistency between Spoken and Semantic Emotion: Evidence from the Analysis of Emotion Expression in Conversation

Xinyi Wu, Changqing Xu, Nan Li, Rongfeng Su, Lan Wang

Joint prediction of subjective listening effort and speech intelligibility based on end-to-end learning

Dirk Eike Hoffner, Jana Roßbach, Bernd T. Meyer

Ethnolinguistic Identification of Vietnamese-German Heritage Speech

Thanh Lan Truong, Andrea Weber

Effect of Complex Boundary Tones on Tone Identification: An Experimental Study with Mandarin-speaking Preschool Children

Aijun Li, Jun Gao, Zhiwei Wang

Towards Classifying Mother Tongue from Infant Cries - Findings Substantiating Prenatal Learning Theory

Tim Polzehl, Tim Herzig, Friedrich Wicke, Kathleen Wermke, Razieh Khamsehashari

On the relationship between speech production and vocabulary size in 3-5 year olds

Alexis DeMaere, Nicole van Rootselaar, Fangfang Li, Robbin Gibb, Claudia L. R. Gonzalez

Cross-Linguistic Intelligibility of Non-Compositional Expressions in Spoken Context

Iuliia Zaitova, Irina Stenger, Wei Xue, Tania Avgustinova, Bernd Möbius

The Production of Contrastive Focus by 7 to 13-year-olds Learning Mandarin Chinese

Zimeng Li, Zhongxuan Mao, Shengting Shen, Ivan Yuen, Ping Tang

Acquisition of high vowel devoicing in Japanese: A production experiment with three and four year olds

Hyun Kyung Hwang, Manami Hirayama

Perception of music and speech: Focus on rhythm processing

Barbara Tillmann

Low Bitrate High-Quality RVQGAN-based Discrete Speech Tokenizer

Slava Shechtman, Avihu Dekel

NAST: Noise Aware Speech Tokenization for Speech Language Models

Shoval Messica, Yossi Adi

COSMIC: Data Efficient Instruction-tuning For Speech In-Context Learning

Jing Pan, Jian Wu, Yashesh Gaur, Sunit Sivasankaran, Zhuo Chen

DeSTA: Enhancing Speech Language Models through Descriptive Speech-Text Alignment

Ke-Han Lu, Zhehuai Chen, Szu-Wei Fu, He Huang, Boris Ginsburg

DiscreteSLU: A Large Language Model with Self-Supervised Discrete Speech Units for Spoken Language Understanding

Suwon Shon, Kwangyoun Kim, Yi-Te Hsu, Prashant Sridhar, Shinji Watanabe

Can Large Language Models Understand Spatial Audio?

Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan

Understanding Sounds, Missing the Questions: The Challenge of Object Hallucination in Large Audio-Language Models

Chun-Yi Kuan, Wei-Ping Huang, Hung-yi Lee

Exploring In-Context Learning of Textless Speech Language Model for Speech Classification Tasks

Kai-Wei Chang, Ming-Hao Hsu, Shan-Wen Li, Hung-yi Lee

On the Effectiveness of Acoustic BPE in Decoder-Only TTS

Bohan Li, Feiyu Shen, Yiwei Guo, Shuai Wang, Xie Chen

CoLM-DSR: Leveraging Neural Codec Language Modeling for Multi-Modal Dysarthric Speech Reconstruction

Xueyuan Chen, Dongchao Yang, Dingdong Wang, Xixin Wu, Zhiyong Wu

Electroglottography for the assessment of dysphonia in Parkinson's disease and multiple system atrophy

Khalid Daoudi, Solange Milhé de Saint Victor, Alexandra Foubert-Samier, Margherita Fabbri, Anne Pavy-Le Traon

Beyond Binary: Multiclass Paraphasia Detection with Generative Pretrained Transformers and End-to-End Models

Matthew Perez, Aneesha Sampath, Minxue Niu, Emily Mower Provost

Exploring Syllable Discriminability during Diadochokinetic Task with Increasing Dysarthria Severity for Patients with Amyotrophic Lateral Sclerosis

Neelesh Samptur, Tanuka Bhattacharjee, Anirudh Chakravarty K, Seena Vengalil, Yamini Belur

CDSD: Chinese Dysarthria Speech Database

Yan Wan, Mengyi Sun, Xinchen Kang, Jingting Li, Pengfei Guo

Automatic Assessment of Dysarthria using Speech and synthetically generated Electroglottograph signal

Fathima Zaheera, Supritha Shetty, Gayadhar Pradhan, Deepak K T

Evaluating Speech Recognition Performance Towards Large Language Model Based Voice Assistants

Zhe Liu, Suyoun Kim, Ozlem Kalinli

Non-Linear Inference Time Intervention: Improving LLM Truthfulness

Jakub Hoscilowicz, Adam Wiacek, Jan Chojnacki, Adam Cieślak, Leszek Michon

Learning from Multiple Annotator Biased Labels in Multimodal Conversation

Kazutoshi Shinoda, Nobukatsu Hojo, Saki Mizuno, Keita Suzuki, Satoshi Kobashikawa

Well, what can you do with messy data? Exploring the prosody and pragmatic function of the discourse marker "well" with found data and speech synthesis

Johannah O'Mahony, Catherine Lai, Eva Szekely

Participant-Pair-Wise Bottleneck Transformer for Engagement Estimation from Video Conversation

Keita Suzuki, Nobukatsu Hojo, Kazutoshi Shinoda, Saki Mizuno, Ryo Masumura

Emotional Cues Extraction and Fusion for Multi-modal Emotion Prediction and Recognition in Conversation

Haoxiang Shi, Ziqi Liang, Jun Yu

MM-NodeFormer: Node Transformer Multimodal Fusion for Emotion Recognition in Conversation

Zilong Huang, Man-Wai Mak, Kong Aik Lee

Cross-Modal Denoising: A Novel Training Paradigm for Enhancing Speech-Image Retrieval

Lifeng Zhou, Yuke Li, Rui Deng, Yuting Yang, Haoqi Zhu

Rapport-Driven Virtual Agent: Rapport Building Dialogue Strategy for Improving User Experience at First Meeting

Muhammad Yeza Baihaqi, Angel Garcia Contreras, Seiya Kawano, Koichiro Yoshino

On the Use of Plausible Arguments in Explainable Conversational AI

Martina Di Bratto, Maria Di Maro, Antonio Origlia

Instruction Data Generation and Unsupervised Adaptation for Speech Language Models

Vahid Noroozi, Zhehuai Chen, Somshubra Majumdar, Steve Huang, Jagadeesh Balam

Reinforcement Learning from Answer Reranking Feedback for Retrieval-Augmented Answer Generation

Minh Nguyen, Toan Quoc Nguyen, Kishan KC, Zeyu Zhang, Thuy Vu

Towards Multilingual Audio-Visual Question Answering

Orchid Chetia Phukan, Priyabrata Mallick, Swarup Ranjan Behera, Aalekhya Satya Narayani, Arun Balaji Buduru

TM-PATHVQA: 90000+ Textless Multilingual Questions for Medical Visual Question Answering

Tonmoy Rajkhowa, Amartya Roy Chowdhury, Sankalp Nagaonkar, Achyut Mani Tripathi, Mahadeva Prasanna

Quantifying the Role of Textual Predictability in Automatic Speech Recognition

Sean Robertson, Gerald Penn, Ewan Dunbar

Analysis and Visualization of Directional Diversity in Listening Fluency of World Englishes Speakers in the Framework of Mutual Shadowing

Yu Tomita, Yingxiang Gao, Nobuaki Minematsu, Noriko Nakanishi, Daisuke Saito

Oversampling, Augmentation and Curriculum Learning for Speaking Assessment with Limited Training Data

Tin Mei Lun, Ekaterina Voskoboinik, Ragheb Al-Ghezi, Tamás Grósz, Mikko Kurimo

A Dataset and Two-pass System for Reading Miscue Detection

Raj Gothi, Rahul Kumar, Mildred Pereira, Nagesh Nayak, Preeti Rao

Context-Aware Speech Recognition Using Prompts for Language Learners

Jian Cheng

Optimizing Automatic Speech Assessment: W-RankSim Regularization and Hybrid Feature Fusion Strategies

Chung-Wen Wu, Berlin Chen

SC-MoE: Switch Conformer Mixture of Experts for Unified Streaming and Non-streaming Code-Switching ASR

Shuaishuai Ye, Shunfei Chen, Xinhui Hu, Xinkang Xu

Enhancing Neural Transducer for Multilingual ASR with Synchronized Language Diarization

Amir Hussein, Desh Raj, Matthew Wiesner, Daniel Povey, Paola García

EFFUSE: Efficient Self-Supervised Feature Fusion for E2E ASR in Low Resource and Multilingual Scenarios

Tejes Srivastava, Jiatong Shi, William Chen, Shinji Watanabe

Whispering in Norwegian: Navigating Orthographic and Dialectic Challenges

Per E Kummervold, Javier de la Rosa, Freddy Wetjen, Rolv-Arild Braaten, Per Erik Solberg

LUPET: Incorporating Hierarchical Information Path into Multilingual ASR

Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao, Tan Lee

Speech Recognition for Greek Dialects: A Challenging Benchmark

Socrates Vakirtzian, Chara Tsoukala, Stavros Bompolas, Katerina Mouzou, Vivian Stamou

The Greek podcast corpus: Competitive speech models for low-resourced languages with weakly supervised data

Georgios Paraskevopoulos, Chara Tsoukala, Athanasios Katsamanis, Vassilis Katsouros

Less is More: Accurate Speech Recognition & Translation without Web-Scale Data

Krishna C. Puvvada, Piotr Żelasko, He Huang, Oleksii Hrinchuk, Nithin Rao Koluguri

On the Effects of Heterogeneous Data Sources on Speech-to-Text Foundation Models

Jinchuan Tian, Yifan Peng, William Chen, Kwanghee Choi, Karen Livescu

Integrating Speech Self-Supervised Learning Models and Large Language Models for ASR

Ling Dong, Zhengtao Yu, Wenjun Wang, Yuxin Huang, Shengxiang Gao

A Unified Approach to Multilingual Automatic Speech Recognition with Improved Language Identification for Indic Languages

Nikhil Jakhar, Sudhanshu Srivastava, Arun Baby

All Ears: Building Self-Supervised Learning based ASR models for Indian Languages at scale

Vasista Sai Lodagala, Abhishek Biswas, Shoutrik Das, Jordan F, S Umesh

mHuBERT-147: A Compact Multilingual HuBERT Model

Marcely Zanon Boito, Vivek Iyer, Nikolaos Lagos, Laurent Besacier, Ioan Calapodescu

LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASR

Zheshu Song, Jianheng Zhuo, Yifan Yang, Ziyang Ma, Shixiong Zhang

A Parameter-efficient Language Extension Framework for Multilingual ASR

Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao, Tan Lee

Cross-Modality Diffusion Modeling and Sampling for Speech Recognition

Chia-Kai Yeh, Chih-Chun Chen, Ching-Hsien Hsu, Jen-Tzung Chien

Sequential Editing for Lifelong Training of Speech Recognition Models

Devang Kulshreshtha, Nikolaos Pappas, Brady Houston, Saket Dingliwal, Srikanth Ronanki

Learnable Layer Selection and Model Fusion for Speech Self-Supervised Learning Models

Sheng-Chieh Chiu, Chia-Hua Wu, Jih-Kang Hsieh, Yu Tsao, Hsin-Min Wang

Contextualized End-to-end Automatic Speech Recognition with Intermediate Biasing Loss

Muhammad Shakeel, Yui Sudo, Yifan Peng, Shinji Watanabe

ASTRA: Aligning Speech and Text Representations for Asr without Sampling

Neeraj Gaur, Rohan Agrawal, Gary Wang, Parisa Haghani, Andrew Rosenberg

Investigating the Effect of Label Topology and Training Criterion on ASR Performance and Alignment Quality

Tina Raissi, Christoph Lüscher, Simon Berger, Ralf Schlüter, Hermann Ney

BiVocoder: A Bidirectional Neural Vocoder Integrating Feature Extraction and Waveform Generation

Hui-Peng Du, Ye-Xin Lu, Yang Ai, Zhen-Hua Ling

QHM-GAN: Neural Vocoder based on Quasi-Harmonic Modeling

Shaowen Chen, Tomoki Toda

FA-GAN: Artifacts-free and Phase-aware High-fidelity GAN-based Vocoder

Rubing Shen, Yanzhen Ren, Zongkun Sun

JenGAN: Stacked Shifted Filters in GAN-Based Speech Synthesis

Hyunjae Cho, Junhyeok Lee, Wonbin Jung

QGAN: Low Footprint Quaternion Neural Vocoder for Speech Synthesis

Aryan Chaudhary, Vinayak Abrol

FreeV: Free Lunch For Vocoders Through Pseudo Inversed Mel Filter

Yuanjun Lv, Hai Li, Ying Yan, Junhui Liu, Danming Xie

Exploring Sentence Type Effects on the Lombard Effect and Intelligibility Enhancement: A Comparative Study of Natural and Grid Sentences

Hongyang Chen, Yuhong Yang, Zhongyuan Wang, Weiping Tu, Haojun Ai

Enhancing Non-Matching Reference Speech Quality Assessment through Dynamic Weight Adaptation

Bao Thang Ta, Van Hai Do, Huynh Thi Thanh Binh

The PESQetarian: On the Relevance of Goodhart's Law for Speech Enhancement

Danilo de Oliveira, Simon Welker, Julius Richter, Timo Gerkmann

No-Reference Speech Intelligibility Prediction Leveraging a Noisy-Speech ASR Pre-Trained Model

Haolan Wang, Amin Edraki, Wai-Yip Chan, Iván López-Espejo, Jesper Jensen

Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata

Ryandhimas E. Zezario, Fei Chen, Chiou-shann Fuh, Hsin-Min Wang, Yu Tsao

Transfer Learning from Whisper for Microscopic Intelligibility Prediction

Paul Best, Santiago Cuervo, Ricard Marxer

Motion Based Audio-Visual Segmentation

Jiahao Li, Miao Liu, Shu Yang, Jing Wang, Xiang Xie

A comparative analysis of sequential models that integrate syllable dependency for automatic syllable stress detection

Jhansi Mallela, Sai Harshitha Aluru, Chiranjeevi Yarra

XANE: eXplainable Acoustic Neural Embeddings

Sri Harsha Dumpala, Dushyant Sharma, Chandramouli Shama Sastry, Stanislav Kruchinin, James Fosburgh

A Transformer-Based Voice Activity Detector

Biswajit Karan, Joshua Jansen van Vüren, Febe de Wet, Thomas Niesler

VAE-based Phoneme Alignment Using Gradient Annealing and SSL Acoustic Features

Tomoki Koriyama

Predefined Prototypes for Intra-Class Separation and Disentanglement

Antonio Almudévar, Théo Mariotte, Alfonso Ortega, Marie Tahon, Luis Vicente

Attention-augmented X-vectors for the Evaluation of Mimicked Speech Using Sparse Autoencoder-LSTM framework

Bhasi K. C., Rajeev Rajan, Noumida A

Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models

Minh Nguyen, Franck Dernoncourt, Seunghyun Yoon, Hanieh Deilamsalehy, Hao Tan

Contrastive Learning and Inter-Speaker Distribution Alignment Based Unsupervised Domain Adaptation for Robust Speaker Verification

Zuoliang Li, Wu Guo, Bin Gu, Shengyu Peng, Jie Zhang

Disentangling Age and Identity with a Mutual Information Minimization for Cross-Age Speaker Verification

Fengrun Zhang, Wangjin Zhou, Yiming Liu, Wang Geng, Yahui Shan

CEC: A Noisy Label Detection Method for Speaker Recognition

Yao Shen, Yingying Gao, Yaqian Hao, Chenguang Hu, Fulin Zhang

W-GVKT: Within-Global-View Knowledge Transfer for Speaker Verification

Zezhong Jin, Youzhi Tu, Man-Wai Mak

On the Usefulness of Speaker Embeddings for Speaker Retrieval in the Wild: A Comparative Study of x-vector and ECAPA-TDNN Models

Erfan Loweimi, Mengjie Qian, Kate Knill, Mark Gales

Specializing Self-Supervised Speech Representations for Speaker Segmentation

Séverin Baroudi, Thomas Pellegrini, Hervé Bredin

On the calibration of powerset speaker diarization models

Alexis Plaquet, Hervé Bredin

Joint vs Sequential Speaker-Role Detection and Automatic Speech Recognition for Air-traffic Control

Alexander Blatt, Aravind Krishnan, Dietrich Klakow

DiarizationLM: Speaker Diarization Post-Processing with Large Language Models

Quan Wang, Yiling Huang, Guanlong Zhao, Evan Clark, Wei Xia

Efficient Speaker Embedding Extraction Using a Twofold Sliding Window Algorithm for Speaker Diarization

Jeong-Hwan Choi, Ye-Rin Jeoung, Ilseok Kim, Joon-Hyuk Chang

Variable Segment Length and Domain-Adapted Feature Optimization for Speaker Diarization

Chenyuan Zhang, Linkai Luo, Hong Peng, Wei Wen