Skip to content
conftrace_

sparse pre-training

4 papers

Explore in graph

Co-occurring keywords · papers mentioning both

model compression (4) weight sparsity (2) dense fine-tuning (1) large language model (1) low-rank adapter (1) biomedical language model (1) domain-specific model (1) model efficiency (1) parameter efficiency (1) inference optimization (1)

Papers

SLoPe: Double-Pruned Sparse Plus Lazy Low-Rank Adapter Pretraining of LLMs ICLR 2025 The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws ICLR 2025 MediSwift: Efficient Sparse Pre-trained Biomedical Language Models ACL 2024 SPDF: Sparse Pre-training and Dense Fine-tuning for Large Language Models UAI 2023