conftrace_

weight decay

50 papers

Explore in graph

Also known as

WD

Co-occurring keywords

neural network (6616) neural network optimization (1293) stochastic gradient descent (1091) convolutional neural network (4226) learning rate (192) deep network (152) gradient descent (1144) batch normalization (222) adversarial robustness (1341) optimization dynamics (10)

Papers

FedAdamW: A Communication-Efficient Optimizer with Convergence and Generalization Guarantees for Federated Large Models AAAI 2026
Optimizer Choice and Calibration for QARiB on Arabic-Script Social Media Offensive Language Detection EACL 2026
Investigating the Role of Weight Decay in Enhancing Nonconvex SGD CVPR 2025
FORGETTER with forgetful hyperparameters and recurring sleeps can continue to learn beyond normal overtfitting limits EMNLP 2025
Fine-tuning LLMs with Cross-Attention-based Weight Decay for Bias Mitigation EMNLP 2025
Nonparametric Classification on Low Dimensional Manifolds using Overparameterized Convolutional Residual Networks NIPS 2024
Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models NIPS 2024
Improving Deep Learning Optimization through Constrained Parameter Regularization NIPS 2024
Global Convergence in Training Large-Scale Transformers NIPS 2024
Why Do We Need Weight Decay in Modern Deep Learning? NIPS 2024
Weight decay induces low-rank attention layers NIPS 2024
On Minimizing the Impact of Dataset Shifts on Actionable Explanations UAI 2023
On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective NIPS 2023
FedNAR: Federated Optimization with Normalized Annealing Regularization NIPS 2023
Improving Robustness with Adaptive Weight Decay NIPS 2023
Deconstructing Data Reconstruction: Multiclass, Weight Decay and General Losses NIPS 2023
Fast Mixing of Stochastic Gradient Descent with Normalization and Weight Decay NIPS 2022
Robust Training of Neural Networks Using Scale Invariant Architectures ICML 2022
Exact Solutions of a Deep Linear Network NIPS 2022
Surprising Instabilities in Training Deep Networks and a Theoretical Analysis NIPS 2022
The Effects of Regularization and Data Augmentation are Class Dependent NIPS 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction NIPS 2022
Learning by Turning: Neural Architecture Aware Optimisation ICML 2021
Banach Space Representer Theorems for Neural Networks and Ridge Splines JMLR 2021
Understanding Decoupled and Early Weight Decay AAAI 2021