NAACL 2025

1,927 papers

Adapting Sentence-level Automatic Metrics for Document-level Simplification Evaluation

Mounica Maddela, Fernando Alva-Manchego

On The Origin of Cultural Biases in Language Models: From Pre-training Data to Linguistic Phenomena

Tarek Naous, Wei Xu

CoME: An Unlearning-based Approach to Conflict-free Model Editing

DaHyun Jung, Jaehyung Seo, Jaewook Lee, Chanjun Park, Heuiseok Lim

A Cognitive Evaluation Benchmark of Image Reasoning and Description for Large Vision-Language Models

Xiujie Song, Mengyue Wu, Kenny Q. Zhu, Chunhao Zhang, Yanyi Chen

CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worlds

Lei Wang, Jianxun Lian, Yi Huang, Yanqi Dai, Haoxuan Li

Effective Skill Unlearning through Intervention and Abstention

Yongce Li, Chung-En Sun, Tsui-Wei Weng

Model Surgery: Modulating LLM’s Behavior Via Simple Parameter Editing

Huanqian Wang, Yang Yue, Rui Lu, Jingxin Shi, Andrew Zhao

Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?

Qisheng Hu, Quanyu Long, Wenya Wang

Rethinking the Role of LLMs for Document-level Relation Extraction: a Refiner with Task Distribution and Probability Fusion

Fu Zhang, Xinlong Jin, Jingwei Cheng, Hongsen Yu, Huangming Xu

FinEval: A Chinese Financial Domain Knowledge Evaluation Benchmark for Large Language Models

Xin Guo, Haotian Xia, Zhaowei Liu, Hanyang Cao, Zhi Yang

Language Models can Categorize System Inputs for Performance Analysis

Dominic Sobhani, Ruiqi Zhong, Edison Marrese-Taylor, Keisuke Sakaguchi, Yutaka Matsuo

EmoCharacter: Evaluating the Emotional Fidelity of Role-Playing Agents in Dialogues

Qiming Feng, Qiujie Xie, Xiaolong Wang, Qingqiu Li, Yuejie Zhang

EvoAgent: Towards Automatic Multi-Agent Generation via Evolutionary Algorithms

Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Dongsheng Li

FLEURS-ASL: Including American Sign Language in Massively Multilingual Multitask Evaluation

Garrett Tanzer

Examining and Adapting Time for Multilingual Classification via Mixture of Temporal Experts

Weisi Liu, Guangzeng Han, Xiaolei Huang

Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages

Jannik Brinkmann, Chris Wendler, Christian Bartelt, Aaron Mueller

VTechAGP: An Academic-to-General-Audience Text Paraphrase Dataset and Benchmark Models

Ming Cheng, Jiaying Gong, Chenhan Yuan, William A Ingram, Edward Fox

VisDoM: Multi-Document QA with Visually Rich Elements Using Multimodal Retrieval-Augmented Generation

Manan Suri, Puneet Mathur, Franck Dernoncourt, Kanika Goswami, Ryan A. Rossi

TinyThinker: Distilling Reasoning through Coarse-to-Fine Knowledge Internalization with Self-Reflection

Shengmin Piao, Sanghyun Park

Hephaestus: Improving Fundamental Agent Capabilities of Large Language Models through Continual Pre-Training

Yuchen Zhuang, Jingfeng Yang, Haoming Jiang, Xin Liu, Kewei Cheng

UOREX: Towards Uncertainty-Aware Open Relation Extraction

Rebii Jamal, Mounir Ourekouch, Mohammed Erradi

AEGIS2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrails

Shaona Ghosh, Prasoon Varshney, Makesh Narsimhan Sreedhar, Aishwarya Padmakumar, Traian Rebedea

Uncovering Bias in Large Vision-Language Models at Scale with Counterfactuals

Phillip Howard, Kathleen C. Fraser, Anahita Bhiwandiwalla, Svetlana Kiritchenko

DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback

Jiao Sun, Deqing Fu, Yushi Hu, Su Wang, Royi Rassin

The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models

Seungone Kim, Juyoung Suk, Ji Yong Cho, Shayne Longpre, Chaeeun Kim

Stronger Universal and Transferable Attacks by Suppressing Refusals

David Huang, Avidan Shah, Alexandre Araujo, David Wagner, Chawin Sitawarin

Do RAG Systems Cover What Matters? Evaluating and Optimizing Responses with Sub-Question Coverage

Kaige Xie, Philippe Laban, Prafulla Kumar Choubey, Caiming Xiong, Chien-Sheng Wu

SUNAR: Semantic Uncertainty based Neighborhood Aware Retrieval for Complex QA

Venktesh V, Mandeep Rathee, Avishek Anand

Rethinking Word Similarity: Semantic Similarity through Classification Confusion

Kaitlyn Zhou, Haishan Gao, Sarah Li Chen, Dan Edelstein, Dan Jurafsky

VoiceTextBlender: Augmenting Large Language Models with Speech Capabilities via Single-Stage Joint Speech-Text Supervised Fine-Tuning

Yifan Peng, Krishna C Puvvada, Zhehuai Chen, Piotr Żelasko, He Huang

Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities

Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng

uDistil-Whisper: Label-Free Data Filtering for Knowledge Distillation in Low-Data Regimes

Abdul Waheed, Karima Kadaoui, Bhiksha Raj, Muhammad Abdul-Mageed

BEMEAE: Moving Beyond Exact Span Match for Event Argument Extraction

Enfa Fane, Md Nayem Uddin, Oghenevovwe Ikumariegbe, Daniyal Kashif, Eduardo Blanco

SynthDetoxM: Modern LLMs are Few-Shot Parallel Detoxification Data Annotators

Daniil Moskovskiy, Nikita Sushko, Sergey Pletenev, Elena Tutubalina, Alexander Panchenko

ConQRet: A New Benchmark for Fine-Grained Automatic Evaluation of Retrieval Augmented Computational Argumentation

Kaustubh Dhole, Kai Shu, Eugene Agichtein

Waste Not, Want Not; Recycled Gumbel Noise Improves Consistency in Natural Language Generation

Damien de Mijolla, Hannan Saddiq, Kim Moore

Efficient One-shot Compression via Low-Rank Local Feature Distillation

Yaya Sy, Christophe Cerisara, Irina Illina

Elevating Legal LLM Responses: Harnessing Trainable Logical Structures and Semantic Knowledge with Legal Reasoning

Rujing Yao, Yang Wu, Chenghao Wang, Jingwei Xiong, Fang Wang

Diverse In-Context Example Selection After Decomposing Programs and Aligned Utterances Improves Semantic Parsing

Mayank Kothyari, Sunita Sarawagi, Soumen Chakrabarti, Gaurav Arora, Srujana Merugu

Fine-Tuning Large Language Models with Sequential Instructions

Hanxu Hu, Simon Yu, Pinzhen Chen, Edoardo Ponti

GuideLLM: Exploring LLM-Guided Conversation with Applications in Autobiography Interviewing

Jinhao Duan, Xinyu Zhao, Zhuoxuan Zhang, Eunhye Grace Ko, Lily Boddy

Anticipating Future with Large Language Model for Simultaneous Machine Translation

Siqi Ouyang, Oleksii Hrinchuk, Zhehuai Chen, Vitaly Lavrukhin, Jagadeesh Balam

Arabic Dataset for LLM Safeguard Evaluation

Yasser Ashraf, Yuxia Wang, Bin Gu, Preslav Nakov, Timothy Baldwin

Aggregation Artifacts in Subjective Tasks Collapse Large Language Models’ Posteriors

Georgios Chochlakis, Alexandros Potamianos, Kristina Lerman, Shrikanth Narayanan

ETHIC: Evaluating Large Language Models on Long-Context Tasks with High Information Coverage

Taewhoo Lee, Chanwoong Yoon, Kyochul Jang, Donghyeon Lee, Minju Song

Evaluating Evidence Attribution in Generated Fact Checking Explanations

Rui Xing, Timothy Baldwin, Jey Han Lau

RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models

Bang An, Shiyue Zhang, Mark Dredze

Multilingual Machine Translation with Open Large Language Models at Practical Scale: An Empirical Study

Menglong Cui, Pengzhi Gao, Wei Liu, Jian Luan, Bin Wang

CSEval: Towards Automated, Multi-Dimensional, and Reference-Free Counterspeech Evaluation using Auto-Calibrated LLMs

Amey Hengle, Aswini Kumar Padhi, Anil Bandhakavi, Tanmoy Chakraborty

Token-based Decision Criteria Are Suboptimal in In-context Learning

Hakaze Cho, Yoshihiro Sakai, Mariko Kato, Kenshiro Tanaka, Akira Ishii

A Novel Computational Modeling Foundation for Automatic Coherence Assessment

Aviya Maimon

Query-focused Referentiability Learning for Zero-shot Retrieval

Jaeyoung Kim, Dohyeon Lee, Seung-won Hwang

Not all Hallucinations are Good to Throw Away When it Comes to Legal Abstractive Summarization

Nihed Bendahman, Karen Pinel-Sauvagnat, Gilles Hubert, Mokhtar Boumedyen Billami

High-Dimension Human Value Representation in Large Language Models

Samuel Cahyawijaya, Delong Chen, Yejin Bang, Leila Khalatbari, Bryan Wilie

My LLM might Mimic AAE - But When Should It?

Sandra Camille Sandoval, Christabel Acquaye, Kwesi Adu Cobbina, Mohammad Nayeem Teli, Hal Daume III

Hello Again! LLM-powered Personalized Agent for Long-term Dialogue

Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang

DIRAS: Efficient LLM Annotation of Document Relevance for Retrieval Augmented Generation

Jingwei Ni, Tobias Schimanski, Meihong Lin, Mrinmaya Sachan, Elliott Ash

Exploiting Edited Large Language Models as General Scientific Optimizers

Qitan Lv, Tianyu Liu, Hong Wang

Exploring the Potential of Large Language Models for Heterophilic Graphs

Yuxia Wu, Shujie Li, Yuan Fang, Chuan Shi

Verify-in-the-Graph: Entity Disambiguation Enhancement for Complex Claim Verification with Interactive Graph Representation

Hoang Pham, Thanh-Do Nguyen, Khac-Hoai Nam Bui

Multilingual Needle in a Haystack: Investigating Long-Context Behavior of Multilingual Large Language Models

Amey Hengle, Prasoon Bajpai, Soham Dan, Tanmoy Chakraborty

On the Vulnerability of Text Sanitization

Meng Tong, Kejiang Chen, Xiaojian Yuan, Jiayang Liu, Weiming Zhang

MoDification: Mixture of Depths Made Easy

Chen Zhang, Meizhi Zhong, Qimeng Wang, Xuantao Lu, Zheyu Ye

Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering

Yu Zhao, Alessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema

MeNTi: Bridging Medical Calculator and LLM Agent with Nested Tool Calling

Yakun Zhu, Shaohang Wei, Xu Wang, Kui Xue, Shaoting Zhang

Are We Done with MMLU?

Aryo Pradipta Gema, Joshua Ong Jun Leang, Giwon Hong, Alessio Devoto, Alberto Carlo Maria Mancino

VisCGEC: Benchmarking the Visual Chinese Grammatical Error Correction

Xiaoman Wang, Dan Yuan, Xin Liu, Yike Zhao, Xiaoxiao Zhang

Test-Time Code-Switching for Cross-lingual Aspect Sentiment Triplet Extraction

Dongming Sheng, Kexin Han, Hao Li, Yan Zhang, Yucheng Huang

Label Drop for Multi-Aspect Relation Modeling in Universal Information Extraction

Lu Yang, Jiajia Li, En Ci, Lefei Zhang, Zuchao Li

FactCG: Enhancing Fact Checkers with Graph-Based Multi-Hop Data

Deren Lei, Yaxi Li, Siyao Li, Mengya Hu, Rui Xu

AgentSense: Benchmarking Social Intelligence of Language Agents through Interactive Scenarios

Xinyi Mou, Jingcong Liang, Jiayu Lin, Xinnong Zhang, Xiawei Liu

MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria

Wentao Ge, Shunian Chen, Hardy Chen, Nuo Chen, Junying Chen

AI-Assisted Human Evaluation of Machine Translation

Vilém Zouhar, Tom Kocmi, Mrinmaya Sachan

Dynamic Fisher-weighted Model Merging via Bayesian Optimization

Sanwoo Lee, Jiahao Liu, Qifan Wang, Jingang Wang, Xunliang Cai

Evaluating Small Language Models for News Summarization: Implications and Factors Influencing Performance

Borui Xu, Yao Chen, Zeyi Wen, Weiguo Liu, Bingsheng He

A Unified Supervised and Unsupervised Dialogue Topic Segmentation Framework Based on Utterance Pair Modeling

Shihao Yang, Ziyi Zhang, Yue Jiang, Chunsheng Qin, Shuhua Liu

Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction

Wei Li, Wen Luo, Guangyue Peng, Houfeng Wang

Multimodal Cognitive Reframing Therapy via Multi-hop Psychotherapeutic Reasoning

Subin Kim, Hoonrae Kim, Heejin Do, Gary Lee

Analyzing (In)Abilities of SAEs via Formal Languages

Abhinav Menon, Manish Shrivastava, David Krueger, Ekdeep Singh Lubana

MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning

Hanqing Wang, Yixia Li, Shuo Wang, Guanhua Chen, Yun Chen

MMEvalPro: Calibrating Multimodal Benchmarks Towards Trustworthy and Efficient Evaluation

Jinsheng Huang, Liang Chen, Taian Guo, Fu Zeng, Yusheng Zhao

SLIM: Let LLM Learn More and Forget Less with Soft LoRA and Identity Mixture

Jiayi Han, Liang Du, Hongwei Du, Xiangguo Zhou, Yiwen Wu

Learning to Solve Domain-Specific Calculation Problems with Knowledge-Intensive Programs Generator

Chengyuan Liu, Shihang Wang, Lizhi Qing, Jun Lin, Ji Zhang

ReachAgent: Enhancing Mobile Agent via Page Reaching and Operation

Qinzhuo Wu, Wei Liu, Jian Luan, Bin Wang

Fact, Fetch, and Reason: A Unified Evaluation of Retrieval-Augmented Generation

Satyapriya Krishna, Kalpesh Krishna, Anhad Mohananey, Steven Schwarcz, Adam Stambler

StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion

Yinghao Aaron Li, Xilin Jiang, Cong Han, Nima Mesgarani

Revisiting Early Detection of Sexual Predators via Turn-level Optimization

JinMyeong An, Sangwon Ryu, Heejin Do, Yunsu Kim, Jungseul Ok

DyPCL: Dynamic Phoneme-level Contrastive Learning for Dysarthric Speech Recognition

WonJun Lee, Solee Im, Heejin Do, Yunsu Kim, Jungseul Ok

Constrained Decoding with Speculative Lookaheads

Nishanth Sridhar Nakshatri, Shamik Roy, Rajarshi Das, Suthee Chaidaroon, Leonid Boytsov

Diversity Helps Jailbreak Large Language Models

Weiliang Zhao, Daniel Ben-Levi, Wei Hao, Junfeng Yang, Chengzhi Mao

What We Talk About When We Talk About LMs: Implicit Paradigm Shifts and the Ship of Language Models

Shengqi Zhu, Jeffrey Rzeszotarski

Entropy-Based Decoding for Retrieval-Augmented Large Language Models

Zexuan Qiu, Zijing Ou, Bin Wu, Jingjing Li, Aiwei Liu

Improving Model Evaluation using SMART Filtering of Benchmark Datasets

Vipul Gupta, Candace Ross, David Pantoja, Rebecca J. Passonneau, Megan Ung

UniHGKR: Unified Instruction-aware Heterogeneous Knowledge Retrievers

Dehai Min, Zhiyang Xu, Guilin Qi, Lifu Huang, Chenyu You

A Top-down Graph-based Tool for Modeling Classical Semantic Maps: A Case Study of Supplementary Adverbs

Zhu Liu, Cunliang Kong, Ying Liu, Maosong Sun

C2: Scalable Auto-Feedback for LLM-based Chart Generation

Woosung Koh, Janghan Yoon, MinHyung Lee, Youngjin Song, Jaegwan Cho

Sharpness-Aware Minimization for Topic Models with High-Quality Document Representations

Tung Nguyen, Tue Le, Hoang Tran Vuong, Quang Duc Nguyen, Duc Anh Nguyen

SSMLoRA: Enhancing Low-Rank Adaptation with State Space Model

Jiayang Yu, Yihang Zhang, Bin Wang, Peiqin Lin, Yongkang Liu

AID: Adaptive Integration of Detectors for Safe AI with Language Models

Xinran Wang, Enmao Diao, Qi Le, Jie Ding, Ali Anwar

FLEX: Expert-level False-Less EXecution Metric for Text-to-SQL Benchmark

Heegyu Kim, Jeon Taeyang, Seunghwan Choi, Seungtaek Choi, Hyunsouk Cho