Neural Network Optimization

How to set AdamW’s weight decay as you scale model and dataset size ICML 2025 HVAdam: A Full-Dimension Adaptive Optimizer AAAI 2025 Hyper-Connections ICLR 2025 HyperNet Fields: Efficiently Training Hypernetworks without Ground Truth by Learning Weight Trajectories CVPR 2025 IAM: Efficient Inference through Attention Mapping between Different-scale LLMs ACL 2025 IDInit: A Universal and Stable Initialization Method for Neural Network Training ICLR 2025 Ignore the KL Penalty! Boosting Exploration on Critical Tokens to Enhance RL Fine-Tuning NAACL 2025 ILIF: Temporal Inhibitory Leaky Integrate-and-Fire Neuron for Overactivation in Spiking Neural Networks IJCAI 2025 Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks ICML 2025 Implicit Bias of Mirror Flow for Shallow Neural Networks in Univariate Regression ICLR 2025 Implicit Reasoning in Transformers is Reasoning through Shortcuts ACL 2025 Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent ICML 2025 Improve Safety Training of Large Language Models with Safety-Critical Singular Vectors Localization ACL 2025 Improving Deep Learning Speed and Performance Through Synaptic Neural Balance AAAI 2025 Improving Transformer Based Line Segment Detection with Matched Predicting and Re-ranking AAAI 2025 Inductive Gradient Adjustment for Spectral Bias in Implicit Neural Representations ICML 2025 Inference Optimal VLMs Need Fewer Visual Tokens and More Parameters ICLR 2025 Injecting Imbalance Sensitivity for Multi-Task Learning IJCAI 2025 Inner Thinking Transformer: Leveraging Dynamic Depth Scaling to Foster Adaptive Internal Thinking ACL 2025 Input Space Mode Connectivity in Deep Neural Networks ICLR 2025 Instability, Computational Efficiency and Statistical Accuracy JMLR 2025 Instability in Downstream Task Performance During LLM Pretraining EMNLP 2025 Investigating ReLoRA: Effects on the Learning Dynamics of Small Language Models EMNLP 2025 Investigating the Role of Weight Decay in Enhancing Nonconvex SGD CVPR 2025 IRASim: A Fine-Grained World Model for Robot Manipulation ICCV 2025