conftrace_

reasoning evaluation

63 papers

Explore in graph

Co-occurring keywords

large language model (13587) benchmark evaluation (1556) question answering (2963) reasoning benchmark (44) evaluation benchmark (257) logical reasoning (310) chain-of-thought prompting (313) chain-of-thought reasoning (551) multilingual nlp (1426) lateral thinking (35)

Papers

ARC ‘Challenge’ Is Not That Challenging ACL 2025
Evaluating Mathematical Reasoning Beyond Accuracy AAAI 2025
Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events ACL 2025
Temporalizing Confidence: Evaluation of Chain-of-Thought Reasoning with Signal Temporal Logic ACL 2025
GAMEBoT: Transparent Assessment of LLM Reasoning in Games ACL 2025
PhysReason: A Comprehensive Benchmark towards Physics-Based Reasoning ACL 2025
Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? IJCNLP 2025
TuringQ: Benchmarking AI Comprehension in Theory of Computation EMNLP 2024
GeminiPro at SemEval-2024 Task 9: BrainTeaser on Gemini SEMEVAL 2024
SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense SEMEVAL 2024
When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives EMNLP 2024
LogicAsker: Evaluating and Improving the Logical Reasoning Ability of Large Language Models EMNLP 2024
Belief Revision: The Adaptability of Large Language Models Reasoning EMNLP 2024
Rationales for Answers to Simple Math Word Problems Confuse Large Language Models ACL 2024
Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks ACL 2024
Competition-Level Problems are Effective LLM Evaluators ACL 2024
A Peek into Token Bias: Large Language Models Are Not Yet Genuine Reasoners EMNLP 2024
Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language Models EMNLP 2024
ReMI: A Dataset for Reasoning with Multiple Images NIPS 2024
S3Eval: A Synthetic, Scalable, Systematic Evaluation Suite for Large Language Model NAACL 2024
SocREval: Large Language Models with the Socratic Method for Reference-free Reasoning Evaluation NAACL 2024
Discursive Socratic Questioning: Evaluating the Faithfulness of Language Models’ Understanding of Discourse Relations ACL 2024
From Generation to Selection: Findings of Converting Analogical Problem-Solving into Multiple-Choice Questions EMNLP 2024
Understanding Faithfulness and Reasoning of Large Language Models on Plain Biomedical Summaries EMNLP 2024
Weak-eval-Strong: Evaluating and Eliciting Lateral Thinking of LLMs with Situation Puzzles NIPS 2024