← Techniques
Deep Learning › Techniques ›

Pretraining

2471 directly classified papers

Papers per year

Papers

Deep Visual Forced Alignment: Learning to Align Transcription with Talking Face Video AAAI 2023
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models ACL 2023
TrOCR: Transformer-Based Optical Character Recognition with Pre-trained Models AAAI 2023
Neural Data Transformer 2: Multi-context Pretraining for Neural Spiking Activity NIPS 2023
Personality-aware Training based Speaker Adaptation for End-to-end Speech Recognition INTERSPEECH 2023
Learning Sample Difficulty from Pre-trained Models for Reliable Prediction NIPS 2023
Embedding Articulatory Constraints for Low-resource Speech Recognition Based on Large Pre-trained Model INTERSPEECH 2023
Rethinking Masked Language Modeling for Chinese Spelling Correction ACL 2023
Multitask Pretraining with Structured Knowledge for Text-to-SQL Generation ACL 2023
Downstream Datasets Make Surprisingly Good Pretraining Corpora ACL 2023
ETHICIST: Targeted Training Data Extraction Through Loss Smoothed Soft Prompting and Calibrated Confidence Estimation ACL 2023
Socratic Pretraining: Question-Driven Pretraining for Controllable Summarization ACL 2023
ManagerTower: Aggregating the Insights of Uni-Modal Experts for Vision-Language Representation Learning ACL 2023
Revisiting Commonsense Reasoning in Machine Translation: Training, Evaluation and Challenge ACL 2023
Plug-and-Play Document Modules for Pre-trained Models ACL 2023
NarrowBERT: Accelerating Masked Language Model Pretraining and Inference ACL 2023
How to Distill your BERT: An Empirical Study on the Impact of Weight Initialisation and Distillation Objectives ACL 2023
OpenRT: An Open-source Framework for Reasoning Over Tabular Data ACL 2023
Prompt Tuning for Unified Multimodal Pretrained Models ACL 2023
Encoder and Decoder, Not One Less for Pre-trained Language Model Sponsored NMT ACL 2023
RHO: Reducing Hallucination in Open-domain Dialogues with Knowledge Grounding ACL 2023
PTCSpell: Pre-trained Corrector Based on Character Shape and Pinyin for Chinese Spelling Correction ACL 2023
Pre-trained Language Model with Prompts for Temporal Knowledge Graph Completion ACL 2023
CoMix: Guide Transformers to Code-Mix using POS structure and Phonetics ACL 2023
MVP: Multi-task Supervised Pre-training for Natural Language Generation ACL 2023