conftrace_

data mixture

9 papers

Explore in graph

Co-occurring keywords

large language model (13587) scaling law (120) language model training (44) knowledge transfer (648) token frequency (7) linear program (26) training data composition (2) membership inference (114) reasoning benchmark (44) submodular function (121)

Papers

TReX: Tokenizer Regression for Optimal Data Mixture EACL 2026
Rethinking Data Mixture for Large Language Models: A Comprehensive Survey and New Perspectives EACL 2026
Optimizing Pre-Training Data Mixtures with Mixtures of Data Expert Models ACL 2025
Demystifying Synthetic Data in LLM Pre-training: A Systematic Study of Scaling Laws, Benefits, and Pitfalls EMNLP 2025
Towards Effective and Efficient Continual Pre-training of Large Language Models ACL 2025
AutoMixer: Checkpoint Artifacts as Automatic Data Mixers ACL 2025
SMART: Submodular Data Mixture Strategy for Instruction Tuning ACL 2024
Data Mixture Inference Attack: BPE Tokenizers Reveal Training Data Compositions NIPS 2024
CMR Scaling Law: Predicting Critical Mixture Ratios for Continual Pre-training of Language Models EMNLP 2024