Evaluation

GenAI Arena: An Open Evaluation Platform for Generative Models NeurIPS 2024 Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models ACL 2024 GeoEval: Benchmark for Evaluating LLMs and Multi-Modal Models on Geometry Problem-Solving ACL 2024 GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation CVPR 2024 Greed is All You Need: An Evaluation of Tokenizer Inference Methods ACL 2024 GuardBench: A Large-Scale Benchmark for Guardrail Models EMNLP 2024 HEMM: Holistic Evaluation of Multimodal Foundation Models NeurIPS 2024 IDGen: Item Discrimination Induced Prompt Generation for LLM Evaluation NeurIPS 2024 ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object CVPR 2024 Impact of Decoding Methods on Human Alignment of Conversational LLMs ACL 2024 Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluation EMNLP 2024 Information Parity: Measuring and Predicting the Multilingual Capabilities of Language Models EMNLP 2024 Integrated Decision Gradients: Compute Your Attributions Where the Model Makes Its Decision AAAI 2024 Knowledge Acquisition through Continued Pretraining is Difficult: A Case Study on r/AskHistorians ACL 2024 LaRE^2: Latent Reconstruction Error Based Method for Diffusion-Generated Image Detection CVPR 2024 LawBench: Benchmarking Legal Knowledge of Large Language Models EMNLP 2024 L-Eval: Instituting Standardized Evaluation for Long Context Language Models ACL 2024 LLMBox: A Comprehensive Library for Large Language Models ACL 2024 LLM-Check: Investigating Detection of Hallucinations in Large Language Models NeurIPS 2024 LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scores ACL 2024 LooGLE: Can Long-Context Language Models Understand Long Contexts? ACL 2024 M5 – A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Tasks EMNLP 2024 MARiA at SemEval 2024 Task-6: Hallucination Detection Through LLMs, MNLI, and Cosine similarity SemEval 2024 Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models EMNLP 2024 Measuring Psychological Depth in Language Models EMNLP 2024