Pretraining

Structural Encoding and Pre-training Matter: Adapting BERT for Table-Based Fact Verification EACL 2021 StructuralLM: Structural Pre-training for Form Understanding ACL 2021 Structural Pre-training for Dialogue Comprehension ACL 2021 SU-NLP at CASE 2021 Task 1: Protest News Detection for English ACL 2021 SUPERB: Speech Processing Universal PERformance Benchmark Interspeech 2021 Support-set bottlenecks for video-text representation learning ICLR 2021 Syntax-BERT: Improving Pre-trained Transformers with Syntax Trees EACL 2021 Syntax-Enhanced Pre-trained Model ACL 2021 TADPOLE: Task ADapted Pre-Training via AnOmaLy DEtection EMNLP 2021 Taking Notes on the Fly Helps Language Pre-Training ICLR 2021 TA-MAMC at SemEval-2021 Task 4: Task-adaptive Pretraining and Multi-head Attention for Abstract Meaning Reading Comprehension SemEval 2021 TAP: Text-Aware Pre-Training for Text-VQA and Text-Caption CVPR 2021 Team JARS: DialDoc Subtask 1 - Improved Knowledge Identification with Supervised Out-of-Domain Pretraining ACL 2021 Team “NoConflict” at CASE 2021 Task 1: Pretraining for Sentence-Level Protest Event Detection ACL 2021 Techniques for Symbol Grounding with SATNet NeurIPS 2021 Tencent AI Lab Machine Translation Systems for the WMT21 Biomedical Translation Task EMNLP 2021 Tensor Programs IV: Feature Learning in Infinite-Width Neural Networks ICML 2021 TenTrans Large-Scale Multilingual Machine Translation System for WMT21 EMNLP 2021 The Effect of Pretraining on Extractive Summarization for Scientific Documents NAACL 2021 The Lottery Tickets Hypothesis for Supervised and Self-Supervised Pre-Training in Computer Vision Models CVPR 2021 The Power of Scale for Parameter-Efficient Prompt Tuning EMNLP 2021 Three-Module Modeling For End-to-End Spoken Language Understanding Using Pre-Trained DNN-HMM-Based Acoustic-Phonetic Model Interspeech 2021 Token-Aware Virtual Adversarial Training in Natural Language Understanding AAAI 2021 Topic Scene Graph Generation by Attention Distillation From Caption ICCV 2021 Topic Transferable Table Question Answering EMNLP 2021