pretraining data

Beyond a Single Extractor: Re-thinking HTML-to-Text Extraction for LLM Pre-training EACL 2026 BhashaKritika: Building Synthetic Pretraining Data at Scale for Indic Languages AAAI 2026 Personal Information Parroting in Language Models EACL 2026 SELECting over Tokens: Curating Pre-training Data at Scale via Token Classification ACL 2026 Uncovering Pretraining Code in LLMs: A Syntax-Aware Attribution Approach AAAI 2026 Automated Detection of Pre-training Text in Black-box LLMs IJCAI 2025 Bias Analysis and Mitigation through Protected Attribute Detection and Regard Classification EMNLP 2025 Combatting Dimensional Collapse in LLM Pre-Training Data via Submodular File Selection ICLR 2025 Con-ReCall: Detecting Pre-training Data in LLMs via Contrastive Decoding COLING 2025 DataDecide: How to Predict Best Pretraining Data with Small Experiments ICML 2025 DataMan: Data Manager for Pre-training Large Language Models ICLR 2025 Data Mixing Laws: Optimizing Data Mixtures by Predicting Language Modeling Performance ICLR 2025 Data Selection via Optimal Control for Language Models ICLR 2025 Enhancing LLMs via High-Knowledge Data Selection AAAI 2025 Fine-tuning can Help Detect Pretraining Data from Large Language Models ICLR 2025 From Data to Knowledge: Evaluating How Efficiently Language Models Learn Facts ACL 2025 Generalization v.s. Memorization: Tracing Language Models’ Capabilities Back to Pretraining Data ICLR 2025 Improving Pretraining Data Using Perplexity Correlations ICLR 2025 Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoders NAACL 2025 Investigating How Pre-training Data Leakage Affects Models’ Reproduction and Detection Capabilities EMNLP 2025 Judging Quality Across Languages: A Multilingual Approach to Pretraining Data Filtering with Language Models EMNLP 2025 LLMs on the Line: Data Determines Loss-to-Loss Scaling Laws ICML 2025 ModernBERT or DeBERTaV3? Examining Architecture and Data Influence on Transformer Encoder Models Performance IJCNLP 2025 Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Models CVPR 2025 On Linear Representations and Pretraining Data Frequency in Language Models ICLR 2025