reasoning benchmark

PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning ACL 2025 RCScore: Quantifying Response Consistency in Large Language Models EMNLP 2025 Reasoning is All You Need for Video Generalization: A Counterfactual Benchmark with Sub-question Evaluation ACL 2025 Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? IJCNLP 2025 SCITAT: A Question Answering Benchmark for Scientific Tables and Text Covering Diverse Reasoning Types ACL 2025 ThinkSLM: Towards Reasoning in Small Language Models EMNLP 2025 ThinkSwitcher: When to Think Hard, When to Think Fast EMNLP 2025 TurnaboutLLM: A Deductive Reasoning Benchmark from Detective Games EMNLP 2025 DyVal: Dynamic Evaluation of Large Language Models for Reasoning Tasks ICLR 2024 FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs EMNLP 2024 Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language Models EMNLP 2024 MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI CVPR 2024 Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution ICML 2024 Rationale-Aware Answer Verification by Pairwise Self-Evaluation EMNLP 2024 ReMI: A Dataset for Reasoning with Multiple Images NeurIPS 2024 Retrieved In-Context Principles from Previous Mistakes EMNLP 2024 Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? EMNLP 2024 TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models ACL 2024 Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate EMNLP 2023 Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models EMNLP 2023 Making Language Models Better Reasoners with Step-Aware Verifier ACL 2023 MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic EMNLP 2023