language model training

CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages COLING 2024 Do Language Models Care about Text Quality? Evaluating Web-Crawled Corpora across 11 Languages COLING 2024 Improving Large Language Models via Fine-grained Reinforcement Learning with Minimum Editing Constraint ACL 2024 LiMe: A Latin Corpus of Late Medieval Criminal Sentences COLING 2024 Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models ACL 2024 LongWanjuan: Towards Systematic Measurement for Long Text Quality EMNLP 2024 Teaching Language Models to Self-Improve through Interactive Demonstrations NAACL 2024 CAME: Confidence-guided Adaptive Memory Efficient Optimization ACL 2023 On the Impact of Cross-Domain Data on German Language Models EMNLP 2023 Skill-it! A data-driven skills framework for understanding and training language models NeurIPS 2023 Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training EMNLP 2023 Text Injection for Capitalization and Turn-Taking Prediction in Speech Models Interspeech 2023 The ROOTS Search Tool: Data Transparency for LLMs ACL 2023 Do Data-based Curricula Work? ACL 2022 Great Power, Great Responsibility: Recommendations for Reducing Energy for Training Language Models NAACL 2022 The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset NeurIPS 2022 Whose Language Counts as High Quality? Measuring Language Ideologies in Text Data Selection EMNLP 2022 Hyperparameter Power Impact in Transformer Language Model Training EMNLP 2021 An experimental analysis of Noise-Contrastive Estimation: the noise distribution matters EACL 2017