← Techniques
Deep Learning › Techniques ›

Pretraining

2471 directly classified papers

Papers per year

Papers

BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies? ACL 2021
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models ACL 2021
Are Pretrained Convolutions Better than Pretrained Transformers? ACL 2021
SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine Translation ACL 2021
Pre-training Universal Language Representation ACL 2021
Syntax-Enhanced Pre-trained Model ACL 2021
Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherence ACL 2021
Scientific Discourse Tagging for Evidence Extraction EACL 2021
Annealing Knowledge Distillation EACL 2021
Long Document Summarization in a Low Resource Setting using Pretrained Language Models IJCNLP 2021
SemiHand: Semi-Supervised Hand Pose Estimation With Consistency ICCV 2021
Team “NoConflict” at CASE 2021 Task 1: Pretraining for Sentence-Level Protest Event Detection IJCNLP 2021
KoDF: A Large-Scale Korean DeepFake Detection Dataset ICCV 2021
FINDINGS OF THE IWSLT 2021 EVALUATION CAMPAIGN IJCNLP 2021
Compressed Video Contrastive Learning NIPS 2021
ERNIE-Gram: Pre-Training with Explicitly N-Gram Masked Language Modeling for Natural Language Understanding NAACL 2021
Improving Neural RST Parsing Model with Silver Agreement Subtrees NAACL 2021
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding IJCNLP 2021
LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval NAACL 2021
mT5: A Massively Multilingual Pre-trained Text-to-Text Transformer NAACL 2021
Transfer Learning for Sequence Generation: from Single-source to Multi-source IJCNLP 2021
A Million Tweets Are Worth a Few Points: Tuning Transformers for Customer Service Tasks NAACL 2021
LZ1904 at SemEval-2021 Task 5: Bi-LSTM-CRF for Toxic Span Detection using Pretrained Word Embedding IJCNLP 2021
Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer IJCNLP 2021
With a Little Help From My Friends: Nearest-Neighbor Contrastive Learning of Visual Representations ICCV 2021