conftrace_

pretraining corpus

34 papers

Explore in graph

Co-occurring keywords · papers mentioning both

large language model (12) language model (9) data contamination (4) language model pretraining (4) in-context learning (4) pretraining data (3) low-resource language (3) domain adaptation (3) few-shot learning (3) representation learning (3)

Papers

Is a Document Educational or Just Wikipedia-Style? — Pitfalls of Classifier-Based Quality Filtering ACL 2026 Attributing Culture-Conditioned Generations to Pretraining Corpora ICLR 2025 Bias Analysis and Mitigation through Protected Attribute Detection and Regard Classification EMNLP 2025 Data Caricatures: On the Representation of African American Language in Pretraining Corpora ACL 2025 Diagnosing Memorization in Chain-of-Thought Reasoning, One Token at a Time EMNLP 2025 Domain Pre-training Impact on Representations EMNLP 2025 Investigating and Scaling up Code-Switching for Multilingual Language Model Pre-Training ACL 2025 MetaSynth: Meta-Prompting-Driven Agentic Scaffolds for Diverse Synthetic Data Generation ACL 2025 On the Path to Make Ukrainian a High-Resource Language ACL 2025 On the Reliability of Large Language Models for Causal Discovery ACL 2025 Scaling Long Context Training Data by Long-Distance Referrals ICLR 2025 UnitCoder: Scalable Code Synthesis from Pre-training Corpora EMNLP 2025 A CURATEd CATalog: Rethinking the Extraction of Pretraining Corpora for Mid-Resourced Languages COLING 2024 A Taxonomy for Data Contamination in Large Language Models ACL 2024 BanglaTLit: A Benchmark Dataset for Back-Transliteration of Romanized Bangla EMNLP 2024 Data Contamination Report from the 2024 CONDA Shared Task ACL 2024 DecorateLM: Data Engineering through Corpus Rating, Tagging, and Editing with Language Models EMNLP 2024 MathPile: A Billion-Token-Scale Pretraining Corpus for Math NeurIPS 2024 Oasis: Data Curation and Assessment System for Pretraining of Large Language Models IJCAI 2024 On the In-context Generation of Language Models EMNLP 2024 Paloma: A Benchmark for Evaluating Language Model Fit NeurIPS 2024 The Reasonableness Behind Unreasonable Translation Capability of Large Language Model ICLR 2024 Understanding In-Context Learning from Repetitions ICLR 2024 Farewell to Aimless Large-scale Pretraining: Influential Subset Selection for Language Model ACL 2023 GLADIS: A General and Large Acronym Disambiguation Benchmark EACL 2023