model evaluation

LLäMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratch ACL 2025 LLM-as-a-Judge Failures at Automating the Identification of Poor Quality Outputs in Free-Form Texts EMNLP 2025 LLM Evaluate: An Industry-Focused Evaluation Tool for Large Language Models COLING 2025 LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks ACL 2025 LLMs May Perform MCQA by Selecting the Least Incorrect Option COLING 2025 LLMsPark: A Benchmark for Evaluating Large Language Models in Strategic Gaming Contexts EMNLP 2025 LMUNIT: Fine-grained Evaluation with Natural Language Unit Tests EMNLP 2025 Lost in Inference: Rediscovering the Role of Natural Language Inference for Large Language Models NAACL 2025 Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models ACL 2025 Maximizing Signal in Human-Model Preference Alignment AAAI 2025 Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric ACL 2025 MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks ICLR 2025 MM-CRITIC: A Holistic Evaluation of Large Multimodal Models as Multimodal Critique EMNLP 2025 MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models ICLR 2025 Model Consistency as a Cheap yet Predictive Proxy for LLM Elo Scores EMNLP 2025 M-RewardBench: Evaluating Reward Models in Multilingual Settings ACL 2025 MR. Judge: Multimodal Reasoner as a Judge EMNLP 2025 Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events ACL 2025 NOVA-63: Native Omni-lingual Versatile Assessments of 63 Disciplines EMNLP 2025 ODP-Bench: Benchmarking Out-of-Distribution Performance Prediction ICCV 2025 OpenNER 1.0: Standardized Open-Access Named Entity Recognition Datasets in 50+ Languages EMNLP 2025 Out of the Box, into the Clinic? Evaluating State-of-the-Art ASR for Clinical Applications for Older Adults EMNLP 2025 PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications EMNLP 2025 posteriordb: Testing, Benchmarking and Developing Bayesian Inference Algorithms AISTATS 2025 Promptception: How Sensitive Are Large Multimodal Models to Prompts? EMNLP 2025