conftrace_
← Techniques
Deep Learning › Techniques ›

Pretraining

2,483 papers

Papers per year

1
5
219
374
'10
'15
'20
'25

Papers

BERTologiCoMix: How does Code-Mixing interact with Multilingual BERT? EACL 2021
Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data EACL 2021
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora EMNLP 2021
Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining EMNLP 2021
A Thorough Evaluation of Task-Specific Pretraining for Summarization EMNLP 2021
Condenser: a Pre-training Architecture for Dense Retrieval EMNLP 2021
How much pretraining data do language models need to learn syntax? EMNLP 2021
mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs EMNLP 2021
AttentionRank: Unsupervised Keyphrase Extraction using Self and Cross Attentions EMNLP 2021
DyLex: Incorporating Dynamic Lexicons into BERT for Sequence Labeling EMNLP 2021
The Power of Scale for Parameter-Efficient Prompt Tuning EMNLP 2021
Improving Neural Machine Translation by Bidirectional Training EMNLP 2021
Improving Math Word Problems with Pre-trained Knowledge and Hierarchical Reasoning EMNLP 2021
Topic Transferable Table Question Answering EMNLP 2021
Revisiting Pivot-Based Paraphrase Generation: Language Is Not the Only Optional Pivot EMNLP 2021
Filling the Gaps in Ancient Akkadian Texts: A Masked Language Modelling Approach EMNLP 2021
LayoutReader: Pre-training of Text and Layout for Reading Order Detection EMNLP 2021
ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning EMNLP 2021
TADPOLE: Task ADapted Pre-Training via AnOmaLy DEtection EMNLP 2021
Contrastive Code Representation Learning EMNLP 2021
Implicit Premise Generation with Discourse-aware Commonsense Knowledge Models EMNLP 2021
Effective Sequence-to-Sequence Dialogue State Tracking EMNLP 2021
FAME: Feature-Based Adversarial Meta-Embeddings for Robust Input Representations EMNLP 2021
CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation EMNLP 2021
ARMAN: Pre-training with Semantically Selecting and Reordering of Sentences for Persian Abstractive Summarization EMNLP 2021