Dan Alistarh

Speculative Decoding Speed-of-Light: Optimal Lower Bounds via Branching Random Walks EACL 2026 Cache Me If You Must: Adaptive Key-Value Quantization for Large Language Models ICML 2025 EvoPress: Accurate Dynamic Model Compression via Evolutionary Search ICML 2025 “Give Me BF16 or Give Me Death”? Accuracy-Performance Trade-Offs in LLM Quantization ACL 2025 HIGGS: Pushing the Limits of Large Language Model Quantization via the Linearity Theorem NAACL 2025 Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence AAAI 2025 Layer-wise Quantization for Quantized Optimistic Dual Averaging ICML 2025 LDAdam: Adaptive Optimization from Low-Dimensional Gradient Statistics ICLR 2025 QuEST: Stable Training of LLMs with 1-Bit Weights and Activations ICML 2025 Scalable Mechanistic Neural Networks ICLR 2025 The Journey Matters: Average Parameter Count over Pre-training Unifies Sparse and Dense Scaling Laws ICLR 2025 Wasserstein Distances, Neuronal Entanglement, and Sparsity ICLR 2025 AsGrad: A Sharp Unified Analysis of Asynchronous-SGD Algorithms AISTATS 2024 Communication-Efficient Federated Learning With Data and Client Heterogeneity AISTATS 2024 Error Feedback Can Accurately Compress Preconditioners ICML 2024 Extreme Compression of Large Language Models via Additive Quantization ICML 2024 Mathador-LM: A Dynamic Benchmark for Mathematical Reasoning on Large Language Models EMNLP 2024 MicroAdam: Accurate Adaptive Optimization with Low Space Overhead and Provable Convergence NeurIPS 2024 PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression NeurIPS 2024 QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs NeurIPS 2024 QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models EMNLP 2024 RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptation ICML 2024 Scaling Laws for Sparsely-Connected Foundation Models ICLR 2024 SPADE: Sparsity-Guided Debugging for Deep Neural Networks ICML 2024 SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression ICLR 2024 The Iterative Optimal Brain Surgeon: Faster Sparse Recovery by Leveraging Second-Order Information NeurIPS 2024 Bias in Pruned Vision Models: In-Depth Analysis and Countermeasures CVPR 2023 CAP: Correlation-Aware Pruning for Highly-Accurate Sparse Vision Models NeurIPS 2023 CrAM: A Compression-Aware Minimizer ICLR 2023 Knowledge Distillation Performs Partial Variance Reduction NeurIPS 2023 OPTQ: Accurate Quantization for Generative Pre-trained Transformers ICLR 2023 Quantized Distributed Training of Large Models with Convergence Guarantees ICML 2023 SparseGPT: Massive Language Models Can be Accurately Pruned in One-Shot ICML 2023 SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks at the Edge ICML 2023 ZipLM: Inference-Aware Structured Pruning of Language Models NeurIPS 2023 How Well Do Sparse ImageNet Models Transfer? CVPR 2022 Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning NeurIPS 2022 SPDY: Accurate Pruning with Speedup Guarantees ICML 2022 The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022 AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networks NeurIPS 2021 Asynchronous Decentralized SGD with Quantized and Local Updates NeurIPS 2021 Asynchronous Optimization Methods for Efficient Training of Deep Neural Networks with Guarantees AAAI 2021 Byzantine-Resilient Non-Convex Stochastic Gradient Descent ICLR 2021 Communication-Efficient Distributed Optimization with Quantized Preconditioners ICML 2021 Distributed Principal Component Analysis with Limited Communication NeurIPS 2021 Elastic Consistency: A Practical Consistency Model for Distributed Stochastic Gradient Descent AAAI 2021 M-FAC: Efficient Matrix-Free Approximations of Second-Order Information NeurIPS 2021 New Bounds For Distributed Mean Estimation and Variance Reduction ICLR 2021 NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization JMLR 2021 Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks JMLR 2021 Towards Tight Communication Lower Bounds for Distributed Optimisation NeurIPS 2021 Adaptive Gradient Quantization for Data-Parallel SGD NeurIPS 2020 Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks ICML 2020 On the Sample Complexity of Adversarial Multi-Source PAC Learning ICML 2020 Scalable Belief Propagation via Relaxed Scheduling NeurIPS 2020 WoodFisher: Efficient Second-Order Approximation for Neural Network Compression NeurIPS 2020 Distributed Learning over Unreliable Networks ICML 2019 Powerset Convolutional Neural Networks NeurIPS 2019 Byzantine Stochastic Gradient Descent NeurIPS 2018 Model compression via distillation and quantization ICLR 2018 The Convergence of Sparsified Gradient Methods NeurIPS 2018 QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding NeurIPS 2017 ZipML: Training Linear Models with End-to-End Low Precision, and a Little Bit of Deep Learning ICML 2017 Streaming Min-max Hypergraph Partitioning NeurIPS 2015