Skip to content
conftrace_

optimizer state

10 papers

Explore in graph

Co-occurring keywords · papers mentioning both

model compression (4) memory efficiency (3) large language model (2) activation quantization (2) model training (2) dimensionality (1) randomized subspace optimization (1) fp8 training (1) gradient statistics (1) gradient (1)

Papers

SYMI: Efficient Mixture-of-Experts Training via Model and Optimizer State Decoupling NSDI 2026 A Memory Efficient Randomized Subspace Optimization Method for Training Large Language Models ICML 2025 COAP: Memory-Efficient Training with Correlation-Aware Gradient Projection CVPR 2025 COAT: Compressing Optimizer states and Activations for Memory-Efficient FP8 Training ICLR 2025 Scaling FP8 training to trillion-token LLMs ICLR 2025 Exploring Quantization for Efficient Pre-Training of Transformer Language Models EMNLP 2024 GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection ICML 2024 Memory-Efficient LLM Training with Online Subspace Descent NeurIPS 2024 Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization NeurIPS 2023 8-bit Optimizers via Block-wise Quantization ICLR 2022