Neural Network Optimization

Sequence-Based Plan Feasibility Prediction for Efficient Task and Motion Planning RSS 2023 SERF: Towards Better Training of Deep Neural Networks Using Log-Softplus ERror Activation Function WACV 2023 (S)GD over Diagonal Linear Networks: Implicit bias, Large Stepsizes and Edge of Stability NeurIPS 2023 SGD with AdaGrad Stepsizes: Full Adaptivity with High Probability to Unknown Parameters, Unbounded Gradients and Affine Variance ICML 2023 SGD with Large Step Sizes Learns Sparse Features ICML 2023 Sharpness-Aware Minimization Leads to Low-Rank Features NeurIPS 2023 Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization NeurIPS 2023 SharPT: Shared Latent Space Prompt Tuning EACL 2023 SHOT: Suppressing the Hessian along the Optimization Trajectory for Gradient-Based Meta-Learning NeurIPS 2023 Show Me How It’s Done: The Role of Explanations in Fine-Tuning Language Models ACML 2023 Simple Hardware-Efficient Long Convolutions for Sequence Modeling ICML 2023 Simple initialization and parametrization of sinusoidal networks via their kernel bandwidth ICLR 2023 Simulated Annealing in Early Layers Leads to Better Generalization CVPR 2023 Sketchy: Memory-efficient Adaptive Regularization with Frequent Directions NeurIPS 2023 Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Models NeurIPS 2023 Softmax Output Approximation for Activation Memory-Efficient Training of Attention-based Networks NeurIPS 2023 SOL: Sampling-based Optimal Linear bounding of arbitrary scalar functions NeurIPS 2023 SparseProp: Efficient Sparse Backpropagation for Faster Training of Neural Networks at the Edge ICML 2023 Sparse Training with Lipschitz Continuous Loss Functions and a Weighted Group L0-norm Constraint JMLR 2023 Sparse tree-based Initialization for Neural Networks ICLR 2023 Speaker-independent Speech Inversion for Estimation of Nasalance Interspeech 2023 Special Properties of Gradient Descent with Large Learning Rates ICML 2023 Spectral Evolution and Invariance in Linear-width Neural Networks NeurIPS 2023 Spherical Fourier Neural Operators: Learning Stable Dynamics on the Sphere ICML 2023 Spike-Based Anytime Perception WACV 2023