continued pretraining

SaulLM-54B & SaulLM-141B: Scaling Up Domain Adaptation for the Legal Domain NeurIPS 2024 SeaLLMs - Large Language Models for Southeast Asia ACL 2024 SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling NAACL 2024 Super donors and super recipients: Studying cross-lingual transfer between high-resource and low-resource languages ACL 2024 wav2vec-S: Adapting Pre-trained Speech Models for Streaming ACL 2024 What happens in continued pre-training? Analysis of self-supervised speech models with continued pre-training for colloquial Finnish ASR Interspeech 2024 Acoustic Word Embeddings for Untranscribed Target Languages with Continued Pretraining and Learned Pooling Interspeech 2023 BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting ACL 2023 Difference-Masking: Choosing What to Mask in Continued Pretraining EMNLP 2023 KILM: Knowledge Injection into Encoder-Decoder Language Models ACL 2023 Mixture-of-Domain-Adapters: Decoupling and Injecting Domain Knowledge to Pre-trained Language Models’ Memories ACL 2023 Oolong: Investigating What Makes Transfer Learning Hard with Controlled Studies EMNLP 2023 Teaching the Pre-trained Model to Generate Simple Texts for Text Simplification ACL 2023 Combining Static and Contextualised Multilingual Embeddings ACL 2022 Continued Pretraining for Better Zero- and Few-Shot Promptability EMNLP 2022 Domain- and Task-Adaptation for VaccinChatNL, a Dutch COVID-19 FAQ Answering Corpus and Classification Model COLING 2022 Evaluating Gender Bias Transfer from Film Data NAACL 2022 Time Waits for No One! Analysis and Challenges of Temporal Misalignment NAACL 2022 EntityBERT: Entity-centric Masking Strategy for Model Pretraining for the Clinical Domain NAACL 2021 How to Adapt Your Pretrained Multilingual Model to 1600 Languages ACL 2021 Revisiting Pretraining with Adapters ACL 2021 Taming Pre-trained Language Models with N-gram Representations for Low-Resource Domain Adaptation ACL 2021 Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media EMNLP 2021