language model pretraining

Towards A Unified View of Sparse Feed-Forward Network in Pretraining Large Language Model EMNLP 2023 Training for Grammatical Error Correction Without Human-Annotated L2 Learners’ Corpora ACL 2023 Transfer Learning for Code-Mixed Data: Do Pretraining Languages Matter? ACL 2023 Unsupervised Improvement of Factual Knowledge in Language Models EACL 2023 Varta: A Large-Scale Headline-Generation Dataset for Indic Languages ACL 2023 Automatic Document Selection for Efficient Encoder Pretraining EMNLP 2022 Automatic Generation of Large-scale Multi-turn Dialogues from Reddit COLING 2022 BanglaBERT: Language Model Pretraining and Benchmarks for Low-Resource Language Understanding Evaluation in Bangla NAACL 2022 BRCC and SentiBahasaRojak: The First Bahasa Rojak Corpus for Pretraining and Sentiment Analysis Dataset COLING 2022 Cooperative Self-training of Machine Reading Comprehension NAACL 2022 DictBERT: Dictionary Description Knowledge Enhanced Language Model Pre-training via Contrastive Learning IJCAI 2022 Dict-BERT: Enhancing Language Model Pre-training with Dictionary ACL 2022 DocQueryNet: Value Retrieval with Arbitrary Queries for Form-like Documents COLING 2022 Entity Extraction in Low Resource Domains with Selective Pre-training of Large Language Models EMNLP 2022 Formulating Few-shot Fine-tuning Towards Language Model Pre-training: A Pilot Study on Named Entity Recognition EMNLP 2022 GLM: General Language Model Pretraining with Autoregressive Blank Infilling ACL 2022 Improving negation detection with negation-focused pre-training NAACL 2022 Improving Temporal Generalization of Pre-trained Language Models with Lexical Semantic Change EMNLP 2022 Instance Regularization for Discriminative Language Model Pre-training EMNLP 2022 Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora ACL 2022 Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora NAACL 2022 LinkBERT: Pretraining Language Models with Document Links ACL 2022 MonoByte: A Pool of Monolingual Byte-level Language Models COLING 2022 On the Role of Bidirectionality in Language Model Pre-Training EMNLP 2022 Phone-ing it in: Towards Flexible Multi-Modal Language Model Training by Phonetic Representations of Data ACL 2022