gradient clipping

Federated Learning with Client Subsampling, Data Heterogeneity, and Unbounded Smoothness: A New Algorithm and Lower Bounds NeurIPS 2023 FedNAR: Federated Optimization with Normalized Annealing Regularization NeurIPS 2023 Generalized Teacher Forcing for Learning Chaotic Dynamics ICML 2023 Improved Convergence in High Probability of Clipped Gradient Methods with Heavy Tailed Noise NeurIPS 2023 Improved Convergence of Differential Private SGD with Gradient Clipping ICLR 2023 Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be ICLR 2023 Provably Robust Temporal Difference Learning for Heavy-Tailed Rewards NeurIPS 2023 Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees ICML 2023 A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks NeurIPS 2022 A Loss Curvature Perspective on Training Instabilities of Deep Learning Models ICLR 2022 Detecting Unintended Memorization in Language-Model-Fused ASR Interspeech 2022 Differentially Private Coordinate Descent for Composite Empirical Risk Minimization ICML 2022 Differentially Private Normalizing Flows for Synthetic Tabular Data Generation AAAI 2022 High Probability Guarantees for Nonconvex Stochastic Gradient Descent with Heavy Tails ICML 2022 Private Stochastic Convex Optimization and Sparse Learning with Heavy-tailed Data Revisited IJCAI 2022 Robustness to Unbounded Smoothness of Generalized SignSGD NeurIPS 2022 Scalable and Efficient Training of Large Convolutional Neural Networks with Differential Privacy NeurIPS 2022 SparseFed: Mitigating Model Poisoning Attacks in Federated Learning with Sparsification AISTATS 2022 Understanding Clipping for Federated Learning: Convergence and Client-Level Differential Privacy ICML 2022 High-probability Bounds for Non-Convex Stochastic Optimization with Heavy Tails NeurIPS 2021 On Proximal Policy Optimization’s Heavy-tailed Gradients ICML 2021 Reverse engineering learned optimizers reveals known and novel mechanisms NeurIPS 2021 Stability and Convergence of Stochastic Gradient Clipping: Beyond Lipschitz Continuity and Smoothness ICML 2021 Tempered Sigmoid Activations for Deep Learning with Differential Privacy AAAI 2021 Understanding Gradient Clipping In Incremental Gradient Methods AISTATS 2021