Neural Network Optimization

Stable Language Model Pre-training by Reducing Embedding Variability EMNLP 2024 Stable Minima Cannot Overfit in Univariate ReLU Networks: Generalization by Large Step Sizes NeurIPS 2024 Stepping on the Edge: Curvature Aware Learning Rate Tuners NeurIPS 2024 Stochastic Newton Proximal Extragradient Method NeurIPS 2024 StreamFlow: Streamlined Multi-Frame Optical Flow Estimation for Video Sequences NeurIPS 2024 Swift Sampler: Efficient Learning of Sampler by 10 Parameters NeurIPS 2024 Symmetries in Overparametrized Neural Networks: A Mean Field View NeurIPS 2024 Take A Shortcut Back: Mitigating the Gradient Vanishing for Training Spiking Neural Networks NeurIPS 2024 Tending Towards Stability: Convergence Challenges in Small Language Models EMNLP 2024 The Challenges of the Nonlinear Regime for Physics-Informed Neural Networks NeurIPS 2024 The Empirical Impact of Neural Parameter Symmetries, or Lack Thereof NeurIPS 2024 The Feature Speed Formula: a flexible approach to scale hyper-parameters of deep neural networks NeurIPS 2024 The Impact of Geometric Complexity on Neural Collapse in Transfer Learning NeurIPS 2024 The Implicit Bias of Adam on Separable Data NeurIPS 2024 The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions NeurIPS 2024 The Ladder in Chaos: Improving Policy Learning by Harnessing the Parameter Evolving Path in A Low-dimensional Space NeurIPS 2024 The Road Less Scheduled NeurIPS 2024 The Role of Over-Parameterization in Machine Learning – the Good, the Bad, the Ugly AAAI 2024 Tight Convergence Rate Bounds for Optimization Under Power Law Spectral Conditions JMLR 2024 Topological Generalization Bounds for Discrete-Time Stochastic Optimization Algorithms NeurIPS 2024 Towards Accurate and Robust Architectures via Neural Architecture Search CVPR 2024 Towards More Accurate Diffusion Model Acceleration with A Timestep Tuner CVPR 2024 Towards the Dynamics of a DNN Learning Symbolic Interactions NeurIPS 2024 Towards training digitally-tied analog blocks via hybrid gradient computation NeurIPS 2024 Training Dynamics of Transformers to Recognize Word Co-occurrence via Gradient Flow Analysis NeurIPS 2024