Neural Network Optimization

The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models NeurIPS 2022 Three-stage Evolution and Fast Equilibrium for SGD with Non-degerate Critical Points ICML 2022 Thrifty Neural Architecture Search for Medical Image Segmentation (Student Abstract) AAAI 2022 TO-FLOW: Efficient Continuous Normalizing Flows With Temporal Optimization Adjoint With Moving Speed CVPR 2022 Topology-aware Generalization of Decentralized SGD ICML 2022 Topology-Preserving Shape Reconstruction and Registration via Neural Diffeomorphic Flow CVPR 2022 Total Energy Shaping with Neural Interconnection and Damping Assignment - Passivity Based Control L4DC 2022 To update or not to update? Neurons at equilibrium in deep models NeurIPS 2022 Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis NeurIPS 2022 Towards Deepening Graph Neural Networks: A GNTK-based Optimization Perspective ICLR 2022 Towards Fully Sparse Training: Information Restoration with Spatial Similarity AAAI 2022 Towards Joint Intent Detection and Slot Filling via Higher-order Attention IJCAI 2022 Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration JMLR 2022 Towards Robust and Reproducible Active Learning Using Neural Networks CVPR 2022 Towards Scaling Difference Target Propagation by Learning Backprop Targets ICML 2022 Towards Statistical and Computational Complexities of Polyak Step Size Gradient Descent AISTATS 2022 Towards Theoretically Inspired Neural Initialization Optimization NeurIPS 2022 Towards understanding how momentum improves generalization in deep learning ICML 2022 Towards Understanding Sharpness-Aware Minimization ICML 2022 Training invariances and the low-rank phenomenon: beyond linear networks ICLR 2022 Training Quantised Neural Networks With STE Variants: The Additive Noise Annealing Algorithm CVPR 2022 Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes NeurIPS 2022 Training stochastic stabilized supralinear networks by dynamics-neutral growth NeurIPS 2022 Training Two-Layer ReLU Networks with Gradient Descent is Inconsistent JMLR 2022 Training Your Sparse Neural Network Better with Any Mask ICML 2022