Neural Network Optimization

On the inductive bias of neural networks for learning read-once DNFs UAI 2022 On the Last Iterate Convergence of Momentum Methods ALT 2022 On the non-universality of deep learning: quantifying the cost of symmetry NeurIPS 2022 On the Parameterization and Initialization of Diagonal State Space Models NeurIPS 2022 On the Practical Robustness of the Nesterov’s Accelerated Quasi-Newton Method AAAI 2022 On the Prediction Network Architecture in RNN-T for ASR Interspeech 2022 On the SDEs and Scaling Rules for Adaptive Gradient Algorithms NeurIPS 2022 On the Stability and Scalability of Node Perturbation Learning NeurIPS 2022 OpReg-Boost: Learning to Accelerate Online Algorithms with Operator Regression L4DC 2022 Optimal and Adaptive Monteiro-Svaiter Acceleration NeurIPS 2022 Optimized Potential Initialization for Low-Latency Spiking Neural Networks AAAI 2022 Oracle Complexity in Nonsmooth Nonconvex Optimization JMLR 2022 Ortho-Shot: Low Displacement Rank Regularization With Data Augmentation for Few-Shot Learning WACV 2022 Overcoming a Theoretical Limitation of Self-Attention ACL 2022 Overcoming The Spectral Bias of Neural Value Approximation ICLR 2022 Overparameterization of Deep ResNet: Zero Loss and Mean-field Analysis JMLR 2022 Parameters or Privacy: A Provable Tradeoff Between Overparameterization and Membership Inference NeurIPS 2022 Partial Class Activation Attention for Semantic Segmentation CVPR 2022 Patch Slimming for Efficient Vision Transformers CVPR 2022 PCEE-BERT: Accelerating BERT Inference via Patient and Confident Early Exiting NAACL 2022 Peek-a-Boo: What (More) is Disguised in a Randomly Weighted Neural Network, and How to Find It Efficiently ICLR 2022 Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning ICML 2022 Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks NeurIPS 2022 Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models ICLR 2022 Playing Lottery Tickets with Vision and Language AAAI 2022