← Techniques
Deep Learning › Techniques ›

Pretraining

2471 directly classified papers

Papers per year

Papers

ECONET: Effective Continual Pretraining of Language Models for Event Temporal Reasoning EMNLP 2021
Improving Math Word Problems with Pre-trained Knowledge and Hierarchical Reasoning EMNLP 2021
mT6: Multilingual Pretrained Text-to-Text Transformer with Translation Pairs EMNLP 2021
A Thorough Evaluation of Task-Specific Pretraining for Summarization EMNLP 2021
Low-Resource Dialogue Summarization with Domain-Agnostic Multi-Source Pretraining EMNLP 2021
ERNIE-M: Enhanced Multilingual Representation by Aligning Cross-lingual Semantics with Monolingual Corpora EMNLP 2021
Using Large Self-Supervised Models for Low-Resource Speech Recognition INTERSPEECH 2021
Three-Module Modeling For End-to-End Spoken Language Understanding Using Pre-Trained DNN-HMM-Based Acoustic-Phonetic Model INTERSPEECH 2021
Speak or Chat with Me: End-to-End Spoken Language Understanding System with Flexible Inputs INTERSPEECH 2021
TA-MAMC at SemEval-2021 Task 4: Task-adaptive Pretraining and Multi-head Attention for Abstract Meaning Reading Comprehension SEMEVAL 2021
IIITH at SemEval-2021 Task 7: Leveraging transformer-based humourous and offensive text detection architectures using lexical and hurtlex features and task adaptive pretraining SEMEVAL 2021
BennettNLP at SemEval-2021 Task 5: Toxic Spans Detection using Stacked Embedding Powered Toxic Entity Recognizer SEMEVAL 2021
A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages EACL 2021
The Lottery Tickets Hypothesis for Supervised and Self-Supervised Pre-Training in Computer Vision Models CVPR 2021
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts CVPR 2021
Partial transfusion: on the expressive influence of trainable batch norm parameters for transfer learning MIDL 2021
Multilingual Speech Translation KIT @ IWSLT2021 ACL 2021
Data Augmentation for Voice-Assistant NLU using BERT-based Interchangeable Rephrase EACL 2021
GTA: Graph Truncated Attention for Retrosynthesis AAAI 2021
Learning to Pre-train Graph Neural Networks AAAI 2021
Anytime Inference with Distilled Hierarchical Neural Ensembles AAAI 2021
Token-Aware Virtual Adversarial Training in Natural Language Understanding AAAI 2021
Extracting Zero-shot Structured Information from Form-like Documents: Pretraining with Keys and Triggers AAAI 2021
ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces AAAI 2021
TA-MAMC at SemEval-2021 Task 4: Task-adaptive Pretraining and Multi-head Attention for Abstract Meaning Reading Comprehension IJCNLP 2021