conftrace_
← Learning Types
Deep Learning › Learning Types ›

Pre-Training

23 papers

Papers per year

2
6
3
1
11

Papers

MoCa: Modality-aware Continual Pre-training Makes Better Bidirectional Multimodal Embeddings ACL 2026
TiKMiX: Efficient Semi-Dynamic Data Mixture via Data Influence for LLM Pre-training ACL 2026
Language Acquisition Device in Large Language Models ACL 2026
Perplexity-Aware Data Scaling Law: Perplexity Landscapes Predict Performance for Continual Pre-training ACL 2026
KoCo: Conditioning Language Model Pre-training on Knowledge Coordinates ACL 2026
Beyond Scaling: Measuring and Predicting the Upper Bound of Knowledge Retention in Language Model Pre-Training ACL 2026
Demystifying Data Organization for Enhanced LLM Training ACL 2026
The Role of Mixed-Language Documents for Multilingual Large Language Model Pretraining ACL 2026
SciPedia: Unlocking the Value of Scientific Data for Pre-training ACL 2026
Is a Document Educational or Just Wikipedia-Style? — Pitfalls of Classifier-Based Quality Filtering ACL 2026
Practical Guidelines for Model Merging in LLMs Pre-Training ACL 2026
Integrating Structural Semantic Knowledge for Enhanced Information Extraction Pre-training EMNLP 2024
Boosting Low-Data Instance Segmentation by Unsupervised Pre-Training With Saliency Prompt CVPR 2023
Exploring Graph Pre-training for Aspect-based Sentiment Analysis EMNLP 2023
CONTRASTE: Supervised Contrastive Pre-training With Aspect-based Prompts For Aspect Sentiment Triplet Extraction EMNLP 2023
GALAXY: A Generative Pre-trained Model for Task-Oriented Dialog with Semi-supervised Learning and Explicit Policy Injection AAAI 2022
DEEP: DEnoising Entity Pre-training for Neural Machine Translation ACL 2022
Scheduled Multi-task Learning for Neural Chat Translation ACL 2022
MarkupLM: Pre-training of Text and Markup Language for Visually Rich Document Understanding ACL 2022
ReasTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples EMNLP 2022
STAR: SQL Guided Pre-Training for Context-dependent Text-to-SQL Parsing EMNLP 2022
Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models ACL 2021
ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation ACL 2021