Neural Network Optimization

On the Accelerated Noise-Tolerant Power Method AISTATS 2023 On the Convergence and Sample Complexity Analysis of Deep Q-Networks with ε-Greedy Exploration NeurIPS 2023 On the Convergence of Continual Learning with Adaptive Methods UAI 2023 On the Convergence of Encoder-only Shallow Transformers NeurIPS 2023 On the Convergence of Gradient Flow on Multi-layer Linear Models ICML 2023 On the Convergence of Stochastic Gradient Descent with Bandwidth-based Step Size JMLR 2023 On the Convergence to a Global Solution of Shuffling-Type Gradient Algorithms NeurIPS 2023 On the Correctness of Automatic Differentiation for Neural Networks with Machine-Representable Parameters ICML 2023 On the Dynamics Under the Unhinged Loss and Beyond JMLR 2023 On the Global Convergence of Fitted Q-Iteration with Two-layer Neural Network Parametrization ICML 2023 On the Implicit Geometry of Cross-Entropy Parameterizations for Label-Imbalanced Data AISTATS 2023 On the Initialization of Graph Neural Networks ICML 2023 On the Neural Tangent Kernel Analysis of Randomly Pruned Neural Networks AISTATS 2023 On the Overlooked Pitfalls of Weight Decay and How to Mitigate Them: A Gradient-Norm Perspective NeurIPS 2023 On the Overlooked Structure of Stochastic Gradients NeurIPS 2023 On the Performance of Temporal Difference Learning With Neural Networks ICLR 2023 On the spectral bias of two-layer linear networks NeurIPS 2023 On the Stability and Generalization of Triplet Learning AAAI 2023 On the Training Instability of Shuffling SGD with Batch Normalization ICML 2023 Operation-Level Early Stopping for Robustifying Differentiable NAS NeurIPS 2023 Optimistic Meta-Gradients NeurIPS 2023 Optimizing Mode Connectivity for Class Incremental Learning ICML 2023 Overcoming Catastrophic Forgetting in Massively Multilingual Continual Learning ACL 2023 Over-Parameterization Exponentially Slows Down Gradient Descent for Learning a Single Neuron COLT 2023 Over-parameterized Deep Nonparametric Regression for Dependent Data with Its Applications to Reinforcement Learning JMLR 2023