conftrace_
← Resources & Methods
Natural Language Processing › Resources & Methods ›

Pretraining

77 papers

Papers per year

1
6
9
10
15
18
3
10
5

Papers

AfriqueLLM: How Data Mixing and Model Architecture Impact Continued Pre-training for African Languages ACL 2026
How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models ACL 2026
Corpus-Dependent Subcharacter Encoding via HMM-Guided Code Assignment ACL 2026
From Where Words Come: Efficient Regularization of Code Tokenizers Through Source Attribution ACL 2026
Enhancing Linguistic Competence of Language Models through Pre-training with Language Learning Tasks ACL 2026
DELTA: Pre-Train a Discriminative Encoder for Legal Case Retrieval via Structural Word Alignment AAAI 2025
Unveiling the Potential of BERT-family: A New Recipe for Building Scalable, General and Competitive Large Language Models ACL 2025
Continued Pretraining and Interpretability-Based Evaluation for Low-Resource Languages: A Galician Case Study ACL 2025
Instruction-Tuning Data Synthesis from Scratch via Web Reconstruction ACL 2025
Designing and Contextualising Probes for African Languages ACL 2025
On the Path to Make Ukrainian a High-Resource Language ACL 2025
VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search EMNLP 2025
Data-Efficient Selection via Grammatical Complexity in Continual Pre-training of Domain-Specific LLMs EMNLP 2025
Training compute-optimal transformer encoder models EMNLP 2025
LP Data Pipeline: Lightweight, Purpose-driven Data Pipeline for Large Language Models EMNLP 2025
MathPile: A Billion-Token-Scale Pretraining Corpus for Math NIPS 2024
REFeREE: A REference-FREE Model-Based Metric for Text Simplification COLING 2024
OtoBERT: Identifying Suffixed Verbal Forms in Modern Hebrew Literature EMNLP 2024
A Survey on Model Compression and Acceleration for Pretrained Language Models AAAI 2023
PairSpanBERT: An Enhanced Language Model for Bridging Resolution ACL 2023
DarkBERT: A Language Model for the Dark Side of the Internet ACL 2023
Pre-training Multi-party Dialogue Models with Latent Discourse Inference ACL 2023
ESCOXLM-R: Multilingual Taxonomy-driven Pre-training for the Job Market Domain ACL 2023
DrBERT: A Robust Pre-trained Model in French for Biomedical and Clinical domains ACL 2023
Improving Syntactic Probing Correctness and Robustness with Control Tasks ACL 2023