conftrace_

reasoning benchmark

44 papers

Explore in graph

Co-occurring keywords

large language model (13587) benchmark evaluation (1556) reasoning evaluation (63) logical reasoning (310) multimodal reasoning (197) theory of mind (122) multimodal large language model (966) visual reasoning (493) language model evaluation (227) model evaluation (445)

Papers

MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset ACL 2025
PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning ACL 2025
CipherBank: Exploring the Boundary of LLM Reasoning Capabilities through Cryptography Challenge ACL 2025
Mind Your Theory: Theory of Mind Goes Deeper Than Reasoning ACL 2025
Demystifying Multilingual Reasoning in Process Reward Modeling EMNLP 2025
TurnaboutLLM: A Deductive Reasoning Benchmark from Detective Games EMNLP 2025
ReMI: A Dataset for Reasoning with Multiple Images NIPS 2024
Rationale-Aware Answer Verification by Pairwise Self-Evaluation EMNLP 2024
MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI CVPR 2024
FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs EMNLP 2024
TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models ACL 2024
Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language Models EMNLP 2024
Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? EMNLP 2024
Retrieved In-Context Principles from Previous Mistakes EMNLP 2024
Making Language Models Better Reasoners with Step-Aware Verifier ACL 2023
MindGames: Targeting Theory of Mind in Large Language Models with Dynamic Epistemic Modal Logic EMNLP 2023
Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Models EMNLP 2023
Can ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate EMNLP 2023
Commonsense Knowledge in Word Associations and ConceptNet EMNLP 2021