← Techniques
Deep Learning › Techniques ›

Pretraining

2471 directly classified papers

Papers per year

Papers

UNIMO: Towards Unified-Modal Understanding and Generation via Cross-Modal Contrastive Learning IJCNLP 2021
BennettNLP at SemEval-2021 Task 5: Toxic Spans Detection using Stacked Embedding Powered Toxic Entity Recognizer IJCNLP 2021
Meta-learning to Improve Pre-training NIPS 2021
Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning NIPS 2021
Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models IJCNLP 2021
Exploring Metaphoric Paraphrase Generation CONLL 2021
Diverse Pretrained Context Encodings Improve Document Translation ACL 2021
StructuralLM: Structural Pre-training for Form Understanding ACL 2021
Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup IJCNLP 2021
Tuning Large Neural Networks via Zero-Shot Hyperparameter Transfer NIPS 2021
Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks IJCNLP 2021
Exploring the Limits of Out-of-Distribution Detection NIPS 2021
Probing Inter-modality: Visual Parsing with Self-Attention for Vision-and-Language Pre-training NIPS 2021
When Do You Need Billions of Words of Pretraining Data? IJCNLP 2021
ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information IJCNLP 2021
Structural Encoding and Pre-training Matter: Adapting BERT for Table-Based Fact Verification EACL 2021
Few-Shot Question Answering by Pretraining Span Selection ACL 2021
Domain-Adaptive Pretraining Methods for Dialogue Understanding ACL 2021
MoRe: A Large-Scale Motorcycle Re-Identification Dataset WACV 2021
Countering Language Drift with Seeded Iterated Learning ICML 2020
EViLBERT: Learning Task-Agnostic Multimodal Sense Embeddings IJCAI 2020
High-order Refining for End-to-end Chinese Semantic Role Labeling AACL 2020
Training with Noise Adversarial Network: A Generalization Method for Object Detection on Sonar Image WACV 2020
ERNIE-GEN: An Enhanced Multi-Flow Pre-training and Fine-tuning Framework for Natural Language Generation IJCAI 2020
FinBERT: A Pre-trained Financial Language Representation Model for Financial Text Mining IJCAI 2020