conftrace_

activation memory

6 papers

Explore in graph

Co-occurring keywords

memory optimization (114) large language model (13587) memory efficiency (261) training efficiency (60) model compression (3302) throughput optimization (46) backward propagation (5) transformer training (13) gradient checkpointing (8) computational graph (12)

Papers

Pipeline Parallelism with Controllable Memory NIPS 2024

DropBP: Accelerating Fine-Tuning of Large Language Models by Dropping Backward Propagation NIPS 2024

FedMef: Towards Memory-efficient Federated Dynamic Pruning CVPR 2024

Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch ICML 2023

Softmax Output Approximation for Activation Memory-Efficient Training of Attention-based Networks NIPS 2023

Efficient Combination of Rematerialization and Offloading for Training DNNs NIPS 2021