conftrace_

block-wise quantization

2 papers

Explore in graph

Co-occurring keywords · papers mentioning both

optimizer state compression (1) low-bitwidth quantization (1) memory-efficient training (1) optimizer state (1) gradient statistics (1) dynamic quantization (1) moment estimation (1) memory footprint (1) language modeling (1)

Papers

Memory Efficient Optimizers with 4-bit States NeurIPS 2023 8-bit Optimizers via Block-wise Quantization ICLR 2022