weight decay

FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Models AAAI 2026 Optimizer Choice and Calibration for QARiB on Arabic-Script Social Media Offensive Language Detection EACL 2026 Segmentation Fault at SemEval-2026 Task 13: A Regularization-First Approach with Generator-Based Out-of-Distribution Splits for Detecting AI-Generated Code SemEval 2026 Adaptive kernel predictors from feature-learning infinite limits of neural networks ICML 2025 An Investigative Study on Recent Sharpness- and Flatness-Based Optimizers for Enhanced Self-Supervised Speaker Verification Interspeech 2025 Fine-tuning LLMs with Cross-Attention-based Weight Decay for Bias Mitigation EMNLP 2025 FORGETTER with forgetful hyperparameters and recurring sleeps can continue to learn beyond normal overtfitting limits EMNLP 2025 How Much Can We Forget about Data Contamination? ICML 2025 How to set AdamW’s weight decay as you scale model and dataset size ICML 2025 Investigating the Role of Weight Decay in Enhancing Nonconvex SGD CVPR 2025 LoRA Training Provably Converges to a Low-Rank Global Minimum Or It Fails Loudly (But it Probably Won’t Fail) ICML 2025 Mirror, Mirror of the Flow: How Does Regularization Shape Implicit Bias? ICML 2025 The Directionality of Optimization Trajectories in Neural Networks ICLR 2025 Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse ICLR 2025 Calibration Bottleneck: Over-compressed Representations are Less Calibratable ICML 2024 Convex Relaxations of ReLU Neural Networks Approximate Global Optima in Polynomial Time ICML 2024 Fast Equilibrium of SGD in Generic Situations ICLR 2024 Global Convergence in Training Large-Scale Transformers NeurIPS 2024 Implicit Bias of AdamW: ℓ_∞-Norm Constrained Optimization ICML 2024 Improving Deep Learning Optimization through Constrained Parameter Regularization NeurIPS 2024 Lion Secretly Solves a Constrained Optimization: As Lyapunov Predicts ICLR 2024 Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks NeurIPS 2024 Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models NeurIPS 2024 Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks ICML 2024 Symmetry Induces Structure and Constraint of Learning ICML 2024