conftrace_

evaluation benchmark

257 papers

Explore in graph

Co-occurring keywords

large language model (13587) benchmark evaluation (1556) question answering (2963) language model (4599) model evaluation (445) language model evaluation (227) instruction following (396) machine translation (2499) multimodal learning (4645) multilingual nlp (1426)

Papers

BiasBed - Rigorous Texture Bias Evaluation CVPR 2023
ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind EMNLP 2023
Can we trust the evaluation on ChatGPT? ACL 2023
A Better Way to Do Masked Language Model Scoring ACL 2023
Blackbird language matrices (BLM), a new task for rule-like generalization in neural networks: Can Large Language Models pass the test? EMNLP 2023
Separating form and meaning: Using self-consistency to quantify task understanding across multiple senses EMNLP 2023
Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarks EMNLP 2023
How Well Do Large Language Models Perform on Faux Pas Tests? ACL 2023
GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective ACL 2023
WYWEB: A NLP Evaluation Benchmark For Classical Chinese ACL 2023
GUMSum: Multi-Genre Data and Evaluation for English Abstractive Summarization ACL 2023
INVITE: a Testbed of Automatically Generated Invalid Questions to Evaluate Large Language Models for Hallucinations EMNLP 2023
Analyzing Multilingual Competency of LLMs in Multi-Turn Instruction Following: A Case Study of Arabic EMNLP 2023
DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models ICCV 2023
An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models ACL 2022
Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair NAACL 2022
TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Media COLING 2022
MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation EMNLP 2022
On the Effectiveness of Automated Metrics for Text Generation Systems EMNLP 2022
Of Human Criteria and Automatic Metrics: A Benchmark of the Evaluation of Story Generation COLING 2022
What Makes Data-to-Text Generation Hard for Pretrained Language Models? EMNLP 2022
Is NLP Ready for Standardization? EMNLP 2022
Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Dataset EMNLP 2022
BECEL: Benchmark for Consistency Evaluation of Language Models COLING 2022
Measure and Improve Robustness in NLP Models: A Survey NAACL 2022