conftrace_

gradient quantization

27 papers

Explore in graph

Co-occurring keywords

communication efficiency (297) distributed training (188) model compression (3302) distributed learning (563) gradient compression (78) distributed optimization (275) neural network optimization (1293) stochastic gradient descent (1091) convolutional neural network (4226) efficient computing (781)

Papers

BitDP: Ultra-low-bit Communication for Data Parallelism in LLM Training AAAI 2026
GradQ-ViT: Robust and Efficient Gradient Quantization for Vision Transformers AAAI 2025
Few-bit Backward: Quantized Gradients of Activation Functions for Memory Footprint Reduction ICML 2023
Quantized Distributed Training of Large Models with Convergence Guarantees ICML 2023
Training Transformers with 4-bit Integers NIPS 2023
Prompt Certified Machine Unlearning with Randomized Gradient Smoothing and Quantization NIPS 2022
Information-constrained optimization: can adaptive processing of gradients help? NIPS 2021
Revealing and Protecting Labels in Distributed Training NIPS 2021
Distribution Adaptive INT8 Quantization for Training CNNs AAAI 2021
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization JMLR 2021
RATQ: A Universal Fixed-Length Quantizer for Stochastic Optimization AISTATS 2020
Adaptive Gradient Quantization for Data-Parallel SGD NIPS 2020
Don’t Waste Your Bits! Squeeze Activations and Gradients for Deep Neural Networks via TinyScript ICML 2020
On Efficient Constructions of Checkpoints ICML 2020
Towards Unified INT8 Training for Convolutional Neural Network CVPR 2020
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks NIPS 2020
Indirect Stochastic Gradient Quantization and Its Application in Distributed Deep Learning AAAI 2020
FracTrain: Fractionally Squeezing Bit Savings Both Temporally and Spatially for Efficient DNN Training NIPS 2020
Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification and Local Computations NIPS 2019
Exploring Fast and Communication-Efficient Algorithms in Large-Scale Distributed Networks AISTATS 2019
Double Quantization for Communication-Efficient Distributed Optimization NIPS 2019
Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients NIPS 2019
A Linear Speedup Analysis of Distributed Deep Learning with Sparse and Quantized Communication NIPS 2018
Scalable methods for 8-bit training of neural networks NIPS 2018
GradiVeQ: Vector Quantization for Bandwidth-Efficient Gradient Aggregation in Distributed CNN Training NIPS 2018