Neural Network Optimization

Two-layer neural network on infinite dimensional data: global optimization guarantee in the mean-field regime NeurIPS 2022 Understanding Stochastic Optimization Behavior at the Layer Update Level (Student Abstract) AAAI 2022 Unifying and Boosting Gradient-Based Training-Free Neural Architecture Search NeurIPS 2022 VectorAdam for Rotation Equivariant Geometry Optimization NeurIPS 2022 What Works and Doesn’t Work, A Deep Decoder for Neural Machine Translation ACL 2022 ZARTS: On Zero-order Optimization for Neural Architecture Search NeurIPS 2022 Zero Stability Well Predicts Performance of Convolutional Neural Networks AAAI 2022 Accelerating Ill-Conditioned Low-Rank Matrix Estimation via Scaled Gradient Descent JMLR 2021 Activate or Not: Learning Customized Activation CVPR 2021 Adaptive Knowledge Driven Regularization for Deep Neural Networks AAAI 2021 Adversarial Robustness Guarantees for Random Deep Neural Networks ICML 2021 A Modular Analysis of Provable Acceleration via Polyak’s Momentum: Training a Wide ReLU Network and a Deep Linear Network ICML 2021 Analytic Insights into Structure and Rank of Neural Network Hessian Maps NeurIPS 2021 ARCH: Efficient Adversarial Regularized Training with Caching EMNLP 2021 A Recipe for Global Convergence Guarantee in Deep Neural Networks AAAI 2021 ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networks ICML 2021 Attention is not all you need: pure attention loses rank doubly exponentially with depth ICML 2021 AttentiveNAS: Improving Neural Architecture Search via Attentive Sampling CVPR 2021 AutoDropout: Learning Dropout Patterns to Regularize Deep Networks AAAI 2021 AutoLR: Layer-wise Pruning and Auto-tuning of Learning Rates in Fine-tuning of Deep Networks AAAI 2021 Automated Model Design and Benchmarking of Deep Learning Models for COVID-19 Detection with Chest CT Scans AAAI 2021 Bayesian Deep Learning via Subnetwork Inference ICML 2021 Benign Overfitting of Constant-Stepsize SGD for Linear Regression COLT 2021 Better Training using Weight-Constrained Stochastic Dynamics ICML 2021 Certifying Incremental Quadratic Constraints for Neural Networks via Convex Optimization L4DC 2021