Skip to content
conftrace_

pretraining corpus

34 papers

Explore in graph

Co-occurring keywords · papers mentioning both

large language model (12) language model (9) data contamination (4) language model pretraining (4) in-context learning (4) pretraining data (3) low-resource language (3) domain adaptation (3) few-shot learning (3) representation learning (3)

Papers

Impact of Co-occurrence on Factual Knowledge of Large Language Models EMNLP 2023 Koala: An Index for Quantifying Overlaps with Pre-training Corpora EMNLP 2023 Trained on 100 million words and still in shape: BERT meets British National Corpus EACL 2023 Data Contamination: From Memorization to Exploitation ACL 2022 On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Model NAACL 2022 Snoopy: An Online Interface for Exploring the Effect of Pretraining Term Frequencies on Few-Shot LM Performance EMNLP 2022 What Language Model to Train if You Have One Million GPU Hours? EMNLP 2022 IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation EMNLP 2021 An Analysis of Dataset Overlap on Winograd-Style Tasks COLING 2020