Interspeech 2024

1,065 papers

FastAST: Accelerating Audio Spectrogram Transformer via Token Merging and Cross-Model Knowledge Distillation

Swarup Ranjan Behera, Abhishek Dhiman, Karthik Gowda, Aalekhya Satya Narayani

Getting More for Less: Using Weak Labels and AV-Mixup for Robust Audio-Visual Speaker Verification

Anith Selvakumar, Homa Fashandi

Self-Supervised Learning with Multi-Head Multi-Mode Knowledge Distillation for Speaker Verification

Zezhong Jin, Youzhi Tu, Man-Wai Mak

SCDNet: Self-supervised Learning Feature based Speaker Change Detection

Yue Li, Xinsheng Wang, Li Zhang, Lei Xie

Self-Supervised Speaker Verification with Mini-Batch Prediction Correction

Junxu Wang, Zhihua Fang, Liang He

Exploring Self-Supervised Multi-view Contrastive Learning for Speech Emotion Recognition with Limited Annotations

Bulat Khaertdinov, Pedro Jeruis, Annanda Sousa, Enrique Hortal

MFSN: Multi-perspective Fusion Search Network For Pre-training Knowledge in Speech Emotion Recognition

Haiyang Sun, Fulin Zhang, Yingying Gao, Shilei Zhang, Zheng Lian

Are Paralinguistic Representations all that is needed for Speech Emotion Recognition?

Orchid Chetia Phukan, Gautam Siddharth Kashyap, Arun Balaji Buduru, Rajesh Sharma

A Layer-Anchoring Strategy for Enhancing Cross-Lingual Speech Emotion Recognition

Shreya G. Upadhyay, Carlos Busso, Chi-Chun Lee

Bridging Emotions Across Languages: Low Rank Adaptation for Multilingual Speech Emotion Recognition

Lucas Goncalves, Donita Robinson, Elizabeth Richerson, Carlos Busso

Enhancing Multimodal Emotion Recognition through ASR Error Compensation and LLM Fine-Tuning

Jehyun Kyung, Serin Heo, Joon-Hyuk Chang

Tackling Missing Modalities in Audio-Visual Representation Learning Using Masked Autoencoders

Georgios Chochlakis, Chandrashekhar Lavania, Prashant Mathur, Kyu J. Han

A multimodal analysis of different types of laughter expression in conversational dialogues

Kexin Wang, Carlos Ishi, Ryoko Hayashi

Prompt Link Multimodal Fusion in Multimodal Sentiment Analysis

Kang Zhu, Cunhang Fan, Jianhua Tao, Zhao Lv

Enhancing Modal Fusion by Alignment and Label Matching for Multimodal Emotion Recognition

Qifei Li, Yingming Gao, Yuhua Wen, Cong Wang, Ya Li

LoRA-MER: Low-Rank Adaptation of Pre-Trained Speech Models for Multimodal Emotion Recognition Using Mutual Information

Yunrui Cai, Zhiyong Wu, Jia Jia, Helen Meng

Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition System

Lingwei Meng, Jiawen Kang, Yuejiao Wang, Zengrui Jin, Xixin Wu

Can you Remove the Downstream Model for Speaker Recognition with Self-Supervised Speech Features?

Zakaria Aldeneh, Takuya Higuchi, Jee-weon Jung, Skyler Seto, Tatiana Likhomanenko

Self-supervised Speech Representations Still Struggle with African American Vernacular English

Kalvin Chang, Yi-Hui Chou, Jiatong Shi, Hsuan-Ming Chen, Nicole Holliday

On the social bias of speech self-supervised models

Yi-Cheng Lin, Tzu-Quan Lin, Hsi-Che Lin, Andy T. Liu, Hung-yi Lee

Emo-bias: A Large Scale Evaluation of Social Bias on Speech Emotion Recognition

Yi-Cheng Lin, Haibin Wu, Huang-Cheng Chou, Chi-Chun Lee, Hung-yi Lee

Unveiling Biases while Embracing Sustainability: Assessing the Dual Challenges of Automatic Speech Recognition Systems

Ajinkya Kulkarni, Atharva Kulkarni, Miguel Couceiro, Isabel Trancoso

Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models

Dominik Wagner, Ilja Baumann, Korbinian Riedhammer, Tobias Bocklet

Speech foundation models in healthcare: Effect of layer selection on pathological speech feature prediction

Daniela A. Wiepert, Rene L. Utianski, Joseph R. Duffy, John L. Stricker, Leland R. Barnard

Searching for Structure: Appraising the Organisation of Speech Features in wav2vec 2.0 Embeddings

Patrick Cormac English, John D. Kelleher, Julie Carson-berndsen

Are Articulatory Feature Overlaps Shrouded in Speech Embeddings?

Erfan A. Shams, Iona Gessinger, Patrick Cormac English, Julie Carson-berndsen

Exploring Self-Supervised Speech Representations for Cross-lingual Acoustic-to-Articulatory Inversion

Yun Hao, Reihaneh Amooie, Wietse de Vries, Thomas Tienkamp, Rik van Noord

Exploring Pre-trained Speech Model for Articulatory Feature Extraction in Dysarthric Speech Using ASR

Yuqin Lin, Longbiao Wang, Jianwu Dang, Nobuaki Minematsu

Human-like Linguistic Biases in Neural Speech Models: Phonetic Categorization and Phonotactic Constraints in Wav2Vec2.0

Marianne de Heer Kloots, Willem Zuidema

Guiding Frame-Level CTC Alignments Using Self-knowledge Distillation

Eungbeom Kim, Hantae Kim, Kyogu Lee

Enhancing CTC-based speech recognition with diverse modeling units

Shiyi Han, MingBin Xu, Zhihong Lei, Zhen Huang, Xingyu Na

Self-Supervised Speech Representations are More Phonetic than Semantic

Kwanghee Choi, Ankita Pasad, Tomohiko Nakamura, Satoru Fukayama, Karen Livescu

Optimizing Large-Scale Context Retrieval for End-to-End ASR

Zhiqi Huang, Diamantino Caseiro, Kandarp Joshi, Christopher Li, Pat Rondon

Revisiting Convolution-free Transformer for Speech Recognition

Zejiang Hou, Goeric Huybrechts, Anshu Bhatia, Daniel Garcia-Romero, Kyu J. Han

Dynamic Encoder Size Based on Data-Driven Layer-wise Pruning for Speech Recognition

Jingjing Xu, Wei Zhou, Zijian Yang, Eugen Beck, Ralf Schlüter

SeMaScore: A new evaluation metric for automatic speech recognition tasks

Zitha Sasindran, Harsha Yelchuri, T. V. Prabhakar

Deep Prosodic Features in Tandem with Perceptual Judgments of Word Reduction for Tone Recognition in Conversed Speech

Xiang-Li Lu, Yi-Fen Liu

Comparing ASR Systems in the Context of Speech Disfluencies

Maria Teleki, Xiangjue Dong, Soohwan Kim, James Caverlee

Beyond Levenshtein: Leveraging Multiple Algorithms for Robust Word Error Rate Computations And Granular Error Classifications

Korbinian Kuhn, Verena Kersken, Gottfried Zimmermann

Quantification of stylistic differences in human- and ASR-produced transcripts of African American English

Annika Heuser, Tyler Kendall, Miguel Del Rio, Quinn McNamara, Nishchal Bhandari

A Small and Fast BERT for Chinese Medical Punctuation Restoration

Tongtao Ling, Yutao Lai, Lei Chen, Shilei Huang, Yi Liu

ED-sKWS: Early-Decision Spiking Neural Networks for Rapid, and Energy-Efficient Keyword Spotting

Zeyang Song, Qianhui Liu, Qu Yang, Yizhou Peng, Haizhou Li

Global-Local Convolution with Spiking Neural Networks for Energy-efficient Keyword Spotting

Shuai Wang, Dehao Zhang, Kexin Shi, Yuchen Wang, Wenjie Wei

RepTor: Re-parameterizable Temporal Convolution for Keyword Spotting via Differentiable Kernel Search

Eunik Park, Daehyun Ahn, Hyungjun Kim

DAISY: Data Adaptive Self-Supervised Early Exit for Speech Representation Models

Tzu-Quan Lin, Hung-yi Lee, Hao Tang

USM RNN-T model weights binarization

Oleg Rybakov, Dmitriy Serdyuk, Chengjian Zheng

One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model

Zhaoqing Li, Haoning Xu, Tianzi Wang, Shoukang Hu, Zengrui Jin

SparseWAV: Fast and Accurate One-Shot Unstructured Pruning for Large Speech Foundation Models

Tianteng Gu, Bei Liu, Hang Shao, Yanmin Qian

Mitigating Overfitting in Structured Pruning of ASR Models with Gradient-Guided Parameter Regularization

Dong-Hyun Kim, Joon-Hyuk Chang

Dynamic Data Pruning for Automatic Speech Recognition

Qiao Xiao, Pingchuan Ma, Adriana Fernandez-Lopez, Boqian Wu, Lu Yin

Simul-Whisper: Attention-Guided Streaming Whisper with Truncation Detection

Haoyu Wang, Guoqiang Hu, Guodong Lin, Wei-Qiang Zhang, Jian Li

Improving Streaming Speech Recognition With Time-Shifted Contextual Attention And Dynamic Right Context Masking

Khanh Le, Duc Chau

TfCleanformer: A streaming, array-agnostic, full- and sub-band modeling front-end for robust ASR

Jens Heitkaemper, Joe Caroselli, Arun Narayanan, Nathan Howard

Streaming Decoder-Only Automatic Speech Recognition with Discrete Speech Units: A Pilot Study

Peikun Chen, Sining Sun, Changhao Shan, Qing Yang, Lei Xie

Decoder-only Architecture for Streaming End-to-end Speech Recognition

Emiru Tsunoo, Hayato Futami, Yosuke Kashiwagi, Siddhant Arora, Shinji Watanabe

Learning from Back Chunks: Acquiring More Future Knowledge for Streaming ASR Models via Self Distillation

Yuting Yang, Guodong Ma, Yuke Li, Binbin Du, Haoqi Zhu

DiffVC+: Improving Diffusion-based Voice Conversion for Speaker Anonymization

Fan Huang, Kun Zeng, Wei Zhu

Probing the Feasibility of Multilingual Speaker Anonymization

Sarina Meyer, Florian Lux, Ngoc Thang Vu

Asynchronous Voice Anonymization Using Adversarial Perturbation On Speaker Embedding

Rui Wang, Liping Chen, Kong Aik Lee, Zhen-Hua Ling

Anonymising Elderly and Pathological Speech: Voice Conversion Using DDSP and Query-by-Example

Suhita Ghosh, Melanie Jouaiti, Arnab Das, Yamini Sinha, Tim Polzehl

TacoLM: GaTed Attention Equipped Codec Language Model are Efficient Zero-Shot Text to Speech Synthesizers

Yakun Song, Zhuo Chen, Xiaofei Wang, Ziyang Ma, Guanrou Yang

Generating Speakers by Prompting Listener Impressions for Pre-trained Multi-Speaker Text-to-Speech Systems

Zhengyang Chen, Xuechen Liu, Erica Cooper, Junichi Yamagishi, Yanmin Qian

DualSpeech: Enhancing Speaker-Fidelity and Text-Intelligibility Through Dual Classifier-Free Guidance

Jinhyeok Yang, Junhyeok Lee, Hyeong-Seok Choi, Seunghoon Ji, Hyeongju Kim

PitchFlow: adding pitch control to a Flow-matching based TTS model

Tasnima Sadekova, Mikhail Kudinov, Vadim Popov, Assel Yermekova, Artem Khrapov

VoiceTailor: Lightweight Plug-In Adapter for Diffusion-Based Personalized Text-to-Speech

Heeseung Kim, Sang-gil Lee, Jiheum Yeom, Che Hyun Lee, Sungwon Kim

Exploring the Robustness of Text-to-Speech Synthesis Based on Diffusion Probabilistic Models to Heavily Noisy Transcriptions

Jingyi Feng, Yusuke Yasuda, Tomoki Toda

Sample-Efficient Diffusion for Text-To-Speech Synthesis

Justin Lovelace, Soham Ray, Kwangyoun Kim, Kilian Q. Weinberger, Felix Wu

SimpleSpeech: Towards Simple and Efficient Text-to-Speech with Scalar Latent Transformer Diffusion Models

Dongchao Yang, Dingdong Wang, Haohan Guo, Xueyuan Chen, Xixin Wu

Knowledge Distillation from Self-Supervised Representation Learning Model with Discrete Speech Units for Any-to-Any Streaming Voice Conversion

Hiroki Kanagawa, Yusuke Ijima

USD-AC: Unsupervised Speech Disentanglement for Accent Conversion

Jen-Hung Huang, Wei-Tsung Lee, Chung-Hsien Wu

Accent Conversion with Articulatory Representations

Yashish M. Siriwardena, Nathan Swedlow, Audrey Howard, Evan Gitterman, Dan Darcy

Hear Your Face: Face-based voice conversion with F0 estimation

Jaejun Lee, Yoori Oh, Injune Hwang, Kyogu Lee

DreamVoice: Text-Guided Voice Conversion

Jiarui Hai, Karan Thakkar, Helin Wang, Zengyi Qin, Mounya Elhilali

HybridVC: Efficient Voice Style Conversion with Text and Audio Prompts

Xinlei Niu, Jing Zhang, Charles Patrick Martin

PRVAE-VC2: Non-Parallel Voice Conversion by Distillation of Speech Representations

Kou Tanaka, Hirokazu Kameoka, Takuhiro Kaneko, Yuto Kondo

Towards Naturalistic Voice Conversion: NaturalVoices Dataset with an Automatic Processing Pipeline

Ali N. Salman, Zongyang Du, Shreeram Suresh Chandra, İsmail Rasim Ülgen, Carlos Busso

SPA-SVC: Self-supervised Pitch Augmentation for Singing Voice Conversion

Bingsong Bai, Fengping Wang, Yingming Gao, Ya Li

Target Speaker Extraction with Curriculum Learning

Yun Liu, Xuechen Liu, Xiaoxiao Miao, Junichi Yamagishi

Unified Audio Visual Cues for Target Speaker Extraction

Tianci Wu, Shulin He, Jiahui Pan, Haifeng Huang, Zhijian Mo

Knowledge boosting during low-latency inference

Vidya Srinivas, Malek Itani, Tuochao Chen, Emre Sefik Eskimez, Takuya Yoshioka

Centroid Estimation with Transformer-Based Speaker Embedder for Robust Target Speaker Extraction

Woon-Haeng Heo, Joongyu Maeng, Yoseb Kang, Namhyun Cho

All Neural Low-latency Directional Speech Extraction

Ashutosh Pandey, Sanha Lee, Juan Azcarreta, Daniel Wong, Buye Xu

Binaural Selective Attention Model for Target Speaker Extraction

Hanyu Meng, Qiquan Zhang, Xiangyu Zhang, Vidhyasaharan Sethu, Eliathamby Ambikairajah

Enhancing Speech and Music Discrimination Through the Integration of Static and Dynamic Features

Liangwei Chen, Xiren Zhou, Qiang Tu, Huanhuan Chen

Speech Topic Classification Based on Multi-Scale and Graph Attention Networks

Fangjing Niu, Xiaozhe Qi, Xinya Chen, Liang He

CogniVoice: Multimodal and Multilingual Fusion Networks for Mild Cognitive Impairment Assessment from Spontaneous Speech

Jiali Cheng, Mohamed Elgaar, Nidhi Vakil, Hadi Amiri

AraOffence: Detecting Offensive Speech Across Dialects in Arabic Media

Youssef Nafea, Shady Shehata, Zeerak Talat, Ahmed Aboeitta, Ahmed Sharshar

Enhancing Multilingual Voice Toxicity Detection with Speech-Text Alignment

Joseph Liu, Mahesh Kumar Nandwana, Janne Pylkkönen, Hannes Heikinheimo, Morgan McGuire

E-ODN: An Emotion Open Deep Network for Generalised and Adaptive Speech Emotion Recognition

Liuxian Ma, Lin Shen, Ruobing Li, Haojie Zhang, Kun Qian

VSASV: a Vietnamese Dataset for Spoofing-Aware Speaker Verification

Vu Hoang, Viet Thanh Pham, Hoa Nguyen Xuan, Pham Nhi, Phuong Dat

Active Speaker Detection in Fisheye Meeting Scenes with Scene Spatial Spectrums

Xinghao Huang, Weiwei Jiang, Long Rao, Wei Xu, Wenqing Cheng

ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models

Jee-weon Jung, Wangyou Zhang, Jiatong Shi, Zakaria Aldeneh, Takuya Higuchi

WeSep: A Scalable and Flexible Toolkit Towards Generalizable Target Speaker Extraction

Shuai Wang, Ke ZHANG, Shaoxiong Lin, Junjie Li, Xuefei Wang

As Biased as You Measure: Methodological Pitfalls of Bias Evaluations in Speaker Verification Research

Wiebke Hutiri, Tanvina Patel, Aaron Yi Ding, Odette Scharenborg

VoxBlink2: A 100K+ Speaker Recognition Corpus and the Open-Set Speaker-Identification Benchmark

Yuke Lin, Ming Cheng, Fulin Zhang, Yingying Gao, Shilei Zhang

Do we EXPECT TO find phonetic traces for syntactic traces?

Jonathan Him Nok Lee, Mark Liberman, Martin Salzmann

Modeling probabilistic reduction across domains with Naive Discriminative Learning

Anna Stein, Kevin Tang

Perceptual Learning in Lexical Tone: Phonetic Similarity vs. Phonological Categories

Ariëlle Reitsema, Chenxin Li, Leanne van Lambalgen, Laura Preining, Saskia Galindo Jong

Phonological Symmetry Does Not Predict Generalization of Perceptual Adaptation to Vowels

Zuheyra Tokac, Jennifer Cole

Quantity-sensitivity affects recall performance of word stress

Constantijn Kaland, Maria Lialiou