Papers
QUENCH: Measuring the gap between Indic and Non-Indic Contextual General Reasoning in LLMs
Mohammad Aflah Khan, Neemesh Yadav, Sarah Masud et al.
What’s the most important value? INVP: INvestigating the Value Priorities of LLMs through Decision-making in Social Scenarios
Xuelin Liu, Pengyuan Liu, Dong Yu
BasqBBQ: A QA Benchmark for Assessing Social Biases in LLMs for Basque, a Low-Resource Language
Muitze Zulaika, Xabier Saralegi
Interactive Evaluation for Medical LLMs via Task-oriented Dialogue System
Ruoyu Liu, Kui Xue, Xiaofan Zhang et al.
Extracting structure from an LLM - how to improve on surprisal-based models of Human Language Processing
Daphne P. Wang, Mehrnoosh Sadrzadeh, Miloš Stanojević et al.
What Makes Cryptic Crosswords Challenging for LLMs?
Abdelrahman Sadallah, Daria Kotova, Ekaterina Kochmar
Unlike “Likely”, “Unlike” is Unlikely: BPE-based Segmentation hurts Morphological Derivations in LLMs
Paul Lerner, François Yvon
LLMs meet Bloom’s Taxonomy: A Cognitive View on Large Language Model Evaluations
Thomas Huber, Christina Niklaus
TOOL-ED: Enhancing Empathetic Response Generation with the Tool Calling Capability of LLM
Huiying Cao, Yiqun Zhang, Shi Feng et al.
Do LLMs Play Dice? Exploring Probability Distribution Sampling in Large Language Models for Behavioral Simulation
Jia Gu, Liang Pang, Huawei Shen et al.
MQM-APE: Toward High-Quality Error Annotation Predictors with Automatic Post-Editing in LLM Translation Evaluators
Qingyu Lu, Liang Ding, Kanjian Zhang et al.
On Evaluating LLMs’ Capabilities as Functional Approximators: A Bayesian Evaluation Framework
Shoaib Ahmed Siddiqui, Yanzhi Chen, Juyeon Heo et al.
LLMs May Perform MCQA by Selecting the Least Incorrect Option
Haochun Wang, Sendong Zhao, Zewen Qiang et al.
Empirical Study on Data Attributes Insufficiency of Evaluation Benchmarks for LLMs
Chuang Liu, Renren Jin, Zheng Yao et al.
Multi-Layered Evaluation Using a Fusion of Metrics and LLMs as Judges in Open-Domain Question Answering
Rashin Rahnamoun, Mehrnoush Shamsfard
Leveraging Taxonomy and LLMs for Improved Multimodal Hierarchical Classification
Shijing Chen, Mohamed Reda Bouadjenek, Usman Naseem et al.
Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs
Guillermo Marco, Luz Rello, Julio Gonzalo
Beyond Surprisal: A Dual Metric Framework for Lexical Skill Acquisition in LLMs
Nazanin Shafiabadi, Guillaume Wisniewski
LLM4RE: A Data-centric Feasibility Study for Relation Extraction
Anushka Swarup, Tianyu Pan, Ronald Wilson et al.
SelfPrompt: Autonomously Evaluating LLM Robustness via Domain-Constrained Knowledge Guidelines and Refined Adversarial Prompts
Aihua Pei, Zehua Yang, Shunan Zhu et al.
A Framework for Effective Invocation Methods of Various LLM Services
Can Wang, Dianbo Sui, Bolin Zhang et al.
Enhancing Event Causality Identification with LLM Knowledge and Concept-Level Event Relations
Ya Su, Hu Zhang, Guangjun Zhang et al.
Propulsion: Steering LLM with Tiny Fine-Tuning
Md Kowsher, Nusrat Jahan Prottasha, Prakash Bhat
Less is More: A Simple yet Effective Token Reduction Method for Efficient Multi-modal LLMs
Dingjie Song, Wenjun Wang, Shunian Chen et al.