← Techniques
Deep Learning › Techniques ›

Pretraining

2471 directly classified papers

Papers per year

Papers

SkoltechNLP at SemEval-2021 Task 5: Leveraging Sentence-level Pre-training for Toxic Span Detection IJCNLP 2021
BERTTune: Fine-Tuning Neural Machine Translation with BERTScore IJCNLP 2021
Syntax-Enhanced Pre-trained Model IJCNLP 2021
Pre-training Universal Language Representation IJCNLP 2021
SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine Translation IJCNLP 2021
FII FUNNY at SemEval-2021 Task 7: HaHackathon: Detecting and rating Humor and Offense SEMEVAL 2021
HiddenCut: Simple Data Augmentation for Natural Language Understanding with Better Generalizability IJCNLP 2021
BERT & Family Eat Word Salad: Experiments with Text Understanding AAAI 2021
A Study into Pre-Training Strategies for Spoken Language Understanding on Dysarthric Speech INTERSPEECH 2021
Pretrained Language Model for Text Generation: A Survey IJCAI 2021
Are Pretrained Convolutions Better than Pretrained Transformers? IJCNLP 2021
Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models IJCNLP 2021
Few-Shot Question Answering by Pretraining Span Selection IJCNLP 2021
A Hands-On Comparison of DNNs for Dialog Separation Using Transfer Learning from Music Source Separation INTERSPEECH 2021
Transformer Based End-to-End Mispronunciation Detection and Diagnosis INTERSPEECH 2021
Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion INTERSPEECH 2021
AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization NAACL 2021
Efficiently Summarizing Text and Graph Encodings of Multi-Document Clusters NAACL 2021
Explicit Alignment Objectives for Multilingual Bidirectional Encoders NAACL 2021
UIBert: Learning Generic Multimodal Representations for UI Understanding IJCAI 2021
BERTologiCoMix: How does Code-Mixing interact with Multilingual BERT? EACL 2021
EmpathBERT: A BERT-based Framework for Demographic-aware Empathy Prediction EACL 2021
Few-Shot Learning of an Interleaved Text Summarization Model by Pretraining with Synthetic Data EACL 2021
Optimizing Deeper Transformers on Small Datasets ACL 2021
LayoutLMv2: Multi-modal Pre-training for Visually-rich Document Understanding ACL 2021