Zeyuan Allen-Zhu

Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process ICLR 2025 Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems ICLR 2025 Physics of Language Models: Part 3.2, Knowledge Manipulation ICLR 2025 Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws ICLR 2025 Physics of Language Models: Part 3.1, Knowledge Storage and Extraction ICML 2024 Backward Feature Correction: How Deep Learning Performs Deep (Hierarchical) Learning COLT 2023 Forward Super-Resolution: How Can GANs Learn Hierarchical Generative Models for Real-World Distributions ICLR 2023 SALSA VERDE: a machine learning attack on LWE with sparse small secrets NeurIPS 2023 Towards Understanding Ensemble, Knowledge Distillation and Self-Distillation in Deep Learning ICLR 2023 LoRA: Low-Rank Adaptation of Large Language Models ICLR 2022 Byzantine-Resilient Non-Convex Stochastic Gradient Descent ICLR 2021 A Convergence Theory for Deep Learning via Over-Parameterization ICML 2019 Can SGD Learn Recurrent Neural Networks with Provable Generalization? NeurIPS 2019 Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers NeurIPS 2019 On the Convergence Rate of Training Recurrent Neural Networks NeurIPS 2019 What Can ResNet Learn Efficiently, Going Beyond Kernels? NeurIPS 2019 Byzantine Stochastic Gradient Descent NeurIPS 2018 How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD NeurIPS 2018 Is Q-Learning Provably Efficient? NeurIPS 2018 Katyusha: The First Direct Acceleration of Stochastic Gradient Methods JMLR 2018 Katyusha X: Simple Momentum Method for Stochastic Sum-of-Nonconvex Optimization ICML 2018 Make the Minority Great Again: First-Order Regret Bound for Contextual Bandits ICML 2018 Natasha 2: Faster Non-Convex Optimization Than SGD NeurIPS 2018 NEON2: Finding Local Minima via First-Order Oracles NeurIPS 2018 The Lingering of Gradients: How to Reuse Gradients Over Time NeurIPS 2018 Doubly Accelerated Methods for Faster CCA and Generalized Eigendecomposition ICML 2017 Faster Principal Component Regression and Stable Matrix Chebyshev Approximation ICML 2017 Follow the Compressed Leader: Faster Online Learning of Eigenvectors and Faster MMWU ICML 2017 Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls NeurIPS 2017 Natasha: Faster Non-Convex Stochastic Optimization via Strongly Non-Convex Parameter ICML 2017 Near-Optimal Design of Experiments via Regret Minimization ICML 2017 Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling ICML 2016 Exploiting the Structure: Stochastic Gradient Methods Using Raw Clusters NeurIPS 2016 Improved SVRG for Non-Strongly-Convex or Sum-of-Non-Convex Objectives ICML 2016 LazySVD: Even Faster SVD Decomposition Yet Without Agonizing Pain NeurIPS 2016 Optimal Black-Box Reductions Between Optimization Objectives NeurIPS 2016 Variance Reduction for Faster Non-Convex Optimization ICML 2016