benchmark evaluation

ProbTS: Benchmarking Point and Distributional Forecasting across Diverse Prediction Horizons NeurIPS 2024 ProLex: A Benchmark for Language Proficiency-oriented Lexical Substitution ACL 2024 PromISe: Releasing the Capabilities of LLMs with Prompt Introspective Search COLING 2024 PromptBench: A Unified Library for Evaluation of Large Language Models JMLR 2024 Prompting open-source and commercial language models for grammatical error correction of English learner text ACL 2024 ProxyQA: An Alternative Framework for Evaluating Long-Form Text Generation with Large Language Models ACL 2024 PUB: A Pragmatics Understanding Benchmark for Assessing LLMs’ Pragmatics Capabilities ACL 2024 Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision ICLR 2024 QGEval: Benchmarking Multi-dimensional Evaluation for Question Generation EMNLP 2024 Question Answering over Tabular Data with DataBench: A Large-Scale Empirical Evaluation of LLMs COLING 2024 RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation EMNLP 2024 Rainbow - A Benchmark for Systematic Testing of How Sensitive Visio-Linguistic Models are to Color Naming EACL 2024 Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs EMNLP 2024 Reasoning Paths with Reference Objects Elicit Quantitative Spatial Reasoning in Large Vision-Language Models EMNLP 2024 Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning IJCAI 2024 Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models EMNLP 2024 RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content NeurIPS 2024 RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems ICLR 2024 Rethinking the Effectiveness of Graph Classification Datasets in Benchmarks for Assessing GNNs IJCAI 2024 Revisiting, Benchmarking and Understanding Unsupervised Graph Domain Adaptation NeurIPS 2024 Revisiting the Self-Consistency Challenges in Multi-Choice Question Formats for Large Language Model Evaluation COLING 2024 ReXTime: A Benchmark Suite for Reasoning-Across-Time in Videos NeurIPS 2024 RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning EMNLP 2024 Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? NeurIPS 2024 Schema-Driven Information Extraction from Heterogeneous Tables EMNLP 2024