benchmark creation

Benchmarking LLM’s Capability in Reasoning over Conflicting Web References ACL 2026 DomainCQA: Crafting Knowledge-Intensive QA from Domain-Specific Charts AAAI 2026 PodBench: A Comprehensive Benchmark for Instruction-Aware Audio-Oriented Podcast Script Generation ACL 2026 Pro-QuEST: A Prompt-chain based Quiz Engine for testing Specialized Technical Product Knowledge EACL 2026 Adaptively profiling models with task elicitation EMNLP 2025 PerCoR: Evaluating Commonsense Reasoning in Persian via Multiple-Choice Sentence Completion IJCNLP 2025 What Has Been Lost with Synthetic Evaluation? EMNLP 2025 Fit for our purpose, not yours: Benchmark for a low-resource, Indigenous language NeurIPS 2024 TAIL: A Toolkit for Automatic and Realistic Long-Context Large Language Model Evaluation EMNLP 2024 Towards a new Benchmark for Emotion Detection in NLP: A Unifying Framework of Recent Corpora EMNLP 2024 NusaCrowd: Open Source Initiative for Indonesian NLP Resources ACL 2023 Real-Time Visual Feedback to Guide Benchmark Creation: A Human-and-Metric-in-the-Loop Workflow EACL 2023 SummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization EMNLP 2023 AnnIE: An Annotation Platform for Constructing Complete Open Information Extraction Benchmark ACL 2022 Building RDF Content for Data-to-Text Generation COLING 2016