Daniel Soudry

Optimal L2 Regularization in High-dimensional Continual Linear Regression ALT 2026 From Continual Learning to SGD and Back: Better Rates for Continual Linear Models ALT 2026 When Diffusion Models Memorize: Inductive Biases in Probability Flow of Minimum-Norm Shallow Neural Nets ICML 2025 Scaling FP8 training to trillion-token LLMs ICLR 2025 Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes NIPS 2024 Provable Tempered Overfitting of Minimal Nets and Typical Nets NIPS 2024 The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting — An Analytical Model ICLR 2024 Exponential Quantum Communication Advantage in Distributed Inference and Learning NIPS 2024 The Implicit Bias of Gradient Descent on Separable Multiclass Data NIPS 2024 How Uniform Random Weights Induce Non-uniform Bias: Typical Interpolating Neural Networks Generalize with Narrow Teachers ICML 2024 Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators ICLR 2024 Minimum Variance Unbiased N:M Sparsity for the Neural Gradients ICLR 2023 Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond ICML 2023 The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks ICLR 2023 Continual Learning in Linear Classification on Separable Data ICML 2023 DropCompute: simple and more robust distributed synchronous training via compute variance reduction NIPS 2023 How do Minimum-Norm Shallow Denoisers Look in Function Space? NIPS 2023 Explore to Generalize in Zero-Shot RL NIPS 2023 The Role of Codeword-to-Class Assignments in Error-Correcting Codes: An Empirical Study AISTATS 2023 Alias-Free Convnets: Fractional Shift Invariance via Polynomial Activations CVPR 2023 Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats ICLR 2023 Regularization Guarantees Generalization in Bayesian Reinforcement Learning through Algorithmic Stability AAAI 2022 Implicit Bias of the Step Size in Linear Diagonal Neural Networks ICML 2022 How catastrophic can catastrophic forgetting be in linear regression? COLT 2022 A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Networks ICLR 2022 Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks NIPS 2021 The Implicit Bias of Minima Stability: A View from Function Space NIPS 2021 On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent ICML 2021 Physics-Aware Downsampling with Deep Learning for Scalable Flood Modeling NIPS 2021 Neural gradients are near-lognormal: improved quantized and sparse training ICLR 2021 Accurate Post Training Quantization With Small Calibration Sets ICML 2021 The Knowledge Within: Methods for Data-Free Model Compression CVPR 2020 Implicit Bias in Deep Linear Classification: Initialization Scale vs Training Accuracy NIPS 2020 Kernel and Rich Regimes in Overparametrized Models COLT 2020 Augment Your Batch: Improving Generalization Through Instance Repetition CVPR 2020 A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case ICLR 2020 At Stability's Edge: How to Adjust Hyperparameters to Preserve Minima Selection in Asynchronous Training of Neural Networks? ICLR 2020 Beyond Signal Propagation: Is Feature Diversity Necessary in Deep Neural Network Initialization? ICML 2020 Post training 4-bit quantization of convolutional networks for rapid-deployment NIPS 2019 Stochastic Gradient Descent on Separable Data: Exact Convergence with a Fixed Learning Rate AISTATS 2019 How do infinite width bounded norm networks look in function space? COLT 2019 Convergence of Gradient Descent on Separable Data AISTATS 2019 A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off NIPS 2019 Lexicographic and Depth-Sensitive Margins in Homogeneous and Non-Homogeneous Deep Models ICML 2019 Fix your classifier: the marginal value of training the last weight layer ICLR 2018 Characterizing Implicit Bias in Terms of Optimization Geometry ICML 2018 Implicit Bias of Gradient Descent on Linear Convolutional Networks NIPS 2018 Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations JMLR 2018 The Implicit Bias of Gradient Descent on Separable Data JMLR 2018 The Implicit Bias of Gradient Descent on Separable Data ICLR 2018 Scalable methods for 8-bit training of neural networks NIPS 2018 Norm matters: efficient and accurate normalization schemes in deep networks NIPS 2018 Train longer, generalize better: closing the generalization gap in large batch training of neural networks NIPS 2017 Binarized Neural Networks NIPS 2016 Expectation Backpropagation: Parameter-Free Training of Multilayer Neural Networks with Continuous or Discrete Weights NIPS 2014 Neuronal Spike Generation Mechanism as an Oversampling, Noise-shaping A-to-D converter NIPS 2012