← Techniques
Deep Learning › Techniques ›

Pretraining

2471 directly classified papers

Papers per year

Papers

Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages EMNLP 2021
Can Transformer Models Measure Coherence In Text: Re-Thinking the Shuffle Test ACL 2021
Attending Self-Attention: A Case Study of Visually Grounded Supervision in Vision-and-Language Transformers ACL 2021
SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining ACL 2021
Is Domain Adaptation Worth Your Investment? Comparing BERT and FinBERT on Financial Tasks EMNLP 2021
Enriching Language Models with Visually-grounded Word Vectors and the Lancaster Sensorimotor Norms EMNLP 2021
ArabicTransformer: Efficient Large Arabic Language Model with Funnel Transformer and ELECTRA Objective EMNLP 2021
JUST-BLUE at SemEval-2021 Task 1: Predicting Lexical Complexity using BERT and RoBERTa Pre-trained Language Models IJCNLP 2021
Voice2Series: Reprogramming Acoustic Models for Time Series Classification ICML 2021
Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks ICML 2021
[RETRACTED] Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training ACL 2021
PLOME: Pre-training with Misspelled Knowledge for Chinese Spelling Correction ACL 2021
VECO: Variable and Flexible Cross-lingual Pre-training for Language Understanding and Generation ACL 2021
A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations ACL 2021
Structural Pre-training for Dialogue Comprehension ACL 2021
NICT Kyoto Submission for the WMT’21 Quality Estimation Task: Multimetric Multilingual Pretraining for Critical Error Detection EMNLP 2021
Papago’s Submission for the WMT21 Quality Estimation Shared Task EMNLP 2021
Tencent AI Lab Machine Translation Systems for the WMT21 Biomedical Translation Task EMNLP 2021
Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task EMNLP 2021
TenTrans Large-Scale Multilingual Machine Translation System for WMT21 EMNLP 2021
JuriBERT: A Masked-Language Model Adaptation for French Legal Text EMNLP 2021
Bilingual Alignment Pre-Training for Zero-Shot Cross-Lingual Transfer EMNLP 2021
Batavia asked for advice. Pretrained language models for Named Entity Recognition in historical texts. EMNLP 2021
Discovering Representation Sprachbund For Multilingual Pre-Training EMNLP 2021
Improving Abstractive Dialogue Summarization with Hierarchical Pretraining and Topic Segment EMNLP 2021