Neural Network Optimization

Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions NeurIPS 2022 Transformer-based quality assessment model for generalized user-generated multimedia audio content Interspeech 2022 Transformer Networks for Non-Intrusive Speech Quality Prediction Interspeech 2022 Transformer Quality in Linear Time ICML 2022 Transition to Linearity of General Neural Networks with Directed Acyclic Graph Architecture NeurIPS 2022 Transition to Linearity of Wide Neural Networks is an Emerging Property of Assembling Weak Models ICLR 2022 Understanding and Preventing Capacity Loss in Reinforcement Learning ICLR 2022 Understanding Deep Contrastive Learning via Coordinate-wise Optimization NeurIPS 2022 Understanding Deep Neural Function Approximation in Reinforcement Learning via ε-Greedy Exploration NeurIPS 2022 Understanding Gradient Descent on the Edge of Stability in Deep Learning ICML 2022 Understanding Stochastic Optimization Behavior at the Layer Update Level (Student Abstract) AAAI 2022 Understanding the Failure of Batch Normalization for Transformers in NLP NeurIPS 2022 Understanding the unstable convergence of gradient descent ICML 2022 Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search NeurIPS 2022 Unravelling the Performance of Physics-informed Graph Neural Networks for Dynamical Systems NeurIPS 2022 Using Deep Mixture-of-Experts to Detect Word Meaning Shift for TempoWiC EMNLP 2022 Vanishing Curvature in Randomly Initialized Deep ReLU Networks AISTATS 2022 VectorAdam for Rotation Equivariant Geometry Optimization NeurIPS 2022 What Can Linear Interpolation of Neural Network Loss Landscapes Tell Us? ICML 2022 When Do Flat Minima Optimizers Work? NeurIPS 2022 XPrompt: Exploring the Extreme of Prompt Tuning EMNLP 2022 1-bit Adam: Communication Efficient Large-Scale Training with Adam’s Convergence Speed ICML 2021 A case where a spindly two-layer linear network decisively outperforms any neural network with a fully connected input layer ALT 2021 Accelerating Feedforward Computation via Parallel Nonlinear Equation Solving ICML 2021 Accelerating Gossip SGD with Periodic Global Averaging ICML 2021