conftrace_

pretraining corpus

20 papers

Explore in graph

Co-occurring keywords

language model (4599) large language model (13587) data contamination (69) language model pretraining (105) low-resource language (2273) in-context learning (1405) few-shot learning (3398) data filtering (99) pretraining datum (38) pretrained corpus (2)

Papers

On the Path to Make Ukrainian a High-Resource Language ACL 2025
Bias Analysis and Mitigation through Protected Attribute Detection and Regard Classification EMNLP 2025
Diagnosing Memorization in Chain-of-Thought Reasoning, One Token at a Time EMNLP 2025
Data Caricatures: On the Representation of African American Language in Pretraining Corpora ACL 2025
Paloma: A Benchmark for Evaluating Language Model Fit NIPS 2024
Oasis: Data Curation and Assessment System for Pretraining of Large Language Models IJCAI 2024
DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models EMNLP 2024
On the In-context Generation of Language Models EMNLP 2024
A CURATEd CATalog: Rethinking the Extraction of Pretraining Corpora for Mid-Resourced Languages COLING 2024
A Taxonomy for Data Contamination in Large Language Models ACL 2024
MathPile: A Billion-Token-Scale Pretraining Corpus for Math NIPS 2024
BanglaTLit: A Benchmark Dataset for Back-Transliteration of Romanized Bangla EMNLP 2024
Impact of Co-occurrence on Factual Knowledge of Large Language Models EMNLP 2023
Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model ACL 2023
Data Contamination: From Memorization to Exploitation ACL 2022
What Language Model to Train if You Have One Million GPU Hours? EMNLP 2022
On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model NAACL 2022
Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance EMNLP 2022
IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation EMNLP 2021
An Analysis of Dataset Overlap on Winograd-Style Tasks COLING 2020