Neural Network Optimization

On the Implicit Bias of Adam ICML 2024 On the Inductive Bias of Stacking Towards Improving Reasoning NeurIPS 2024 On the Parameterization of Second-Order Optimization Effective towards the Infinite Width ICLR 2024 On the Power of Small-size Graph Neural Networks for Linear Programming NeurIPS 2024 On the Relationship between Skill Neurons and Robustness in Prompt Tuning COLING 2024 On the Sparsity of the Strong Lottery Ticket Hypothesis NeurIPS 2024 On the token distance modeling ability of higher RoPE attention dimension EMNLP 2024 On the Unstable Convergence Regime of Gradient Descent AAAI 2024 Open Problem: Black-Box Reductions and Adaptive Gradient Methods for Nonconvex Optimization COLT 2024 Operator-Learning-Inspired Modeling of Neural Ordinary Differential Equations AAAI 2024 OptEx: Expediting First-Order Optimization with Approximately Parallelized Iterations NeurIPS 2024 Optimal Bump Functions for Shallow ReLU networks: Weight Decay, Depth Separation, Curse of Dimensionality JMLR 2024 Optimal First-Order Algorithms as a Function of Inequalities JMLR 2024 Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMs EMNLP 2024 Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling NeurIPS 2024 Ordered Momentum for Asynchronous SGD NeurIPS 2024 OrthCaps: An Orthogonal CapsNet with Sparse Attention Routing and Pruning CVPR 2024 Outliers with Opposing Signals Have an Outsized Effect on Neural Network Optimization ICLR 2024 Overcoming the Optimizer’s Curse: Obtaining Realistic Prescriptions from Neural Networks ICML 2024 Overparametrized Multi-layer Neural Networks: Uniform Concentration of Neural Tangent Kernel and Convergence of Stochastic Gradient Descent JMLR 2024 Over-Reasoning and Redundant Calculation of Large Language Models EACL 2024 Parallelizing Linear Transformers with the Delta Rule over Sequence Length NeurIPS 2024 Parameter-Agnostic Optimization under Relaxed Smoothness AISTATS 2024 Parameter-Efficient Sparsity Crafting from Dense to Mixture-of-Experts for Instruction Tuning on General Tasks EMNLP 2024 Parameter-free Clipped Gradient Descent Meets Polyak NeurIPS 2024