benchmark evaluation

SciEval: A Multi-Level Large Language Model Evaluation Benchmark for Scientific Research AAAI 2024 SciEx: Benchmarking Large Language Models on Scientific Exams with Human Expert Grading and Automatic Grading EMNLP 2024 SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation NeurIPS 2024 SeaEval for Multilingual Foundation Models: From Cross-Lingual Alignment to Cultural Reasoning NAACL 2024 SEED-Bench: Benchmarking Multimodal Large Language Models CVPR 2024 Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? EMNLP 2024 SETLEXSEM CHALLENGE: Using Set Operations to Evaluate the Lexical and Semantic Robustness of Language Models NeurIPS 2024 Shopping MMLU: A Massive Multi-Task Online Shopping Benchmark for Large Language Models NeurIPS 2024 SocialBench: Sociality Evaluation of Role-Playing Conversational Agents ACL 2024 Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future ACL 2024 Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets EMNLP 2024 SpaRC and SpaRP: Spatial Reasoning Characterization and Path Generation for Understanding Spatial Reasoning Capability of Large Language Models ACL 2024 SpeciaLex: A Benchmark for In-Context Specialized Lexicon Learning EMNLP 2024 Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows? NeurIPS 2024 SpreadsheetBench: Towards Challenging Real World Spreadsheet Manipulation NeurIPS 2024 StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models ACL 2024 StackEval: Benchmarking LLMs in Coding Assistance NeurIPS 2024 Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning NeurIPS 2024 Still Not Quite There! Evaluating Large Language Models for Comorbid Mental Health Diagnosis EMNLP 2024 STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions EMNLP 2024 STORYSUMM: Evaluating Faithfulness in Story Summarization EMNLP 2024 StructEval: Deepen and Broaden Large Language Model Assessment via Structured Evaluation ACL 2024 StudentEval: A Benchmark of Student-Written Prompts for Large Language Models of Code ACL 2024 SuperGLEBer: German Language Understanding Evaluation Benchmark NAACL 2024 SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering NeurIPS 2024