Neural Network Optimization

Scaling and Scalability: Provable Nonconvex Low-Rank Tensor Completion AISTATS 2022 Scaling Laws Under the Microscope: Predicting Transformer Performance from Small Scale Experiments EMNLP 2022 Searching for BurgerFormer with Micro-Meso-Macro Space Design ICML 2022 Search to Pass Messages for Temporal Knowledge Graph Completion EMNLP 2022 Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networks NeurIPS 2022 Sharpness-Aware Minimization with Dynamic Reweighting EMNLP 2022 Sharpness-Aware Training for Free NeurIPS 2022 Signed Neuron with Memory: Towards Simple, Accurate and High-Efficient ANN-SNN Conversion IJCAI 2022 Simple and Optimal Stochastic Gradient Methods for Nonsmooth Nonconvex Optimization JMLR 2022 Simple Local Attentions Remain Competitive for Long-Context Tasks NAACL 2022 SInGE: Sparsity via Integrated Gradients Estimation of Neuron Relevance NeurIPS 2022 Smoothing Advantage Learning AAAI 2022 Smooth Maximum Unit: Smooth Activation Function for Deep Networks Using Smoothing Maximum Technique CVPR 2022 Sobolev Acceleration and Statistical Optimality for Learning Elliptic Equations via Gradient Descent NeurIPS 2022 Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation ICML 2022 Solving PDE-Constrained Control Problems Using Operator Learning AAAI 2022 Sparse Structure Search for Delta Tuning NeurIPS 2022 Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime NeurIPS 2022 Speeding up Transformer Decoding via an Attention Refinement Network COLING 2022 Squashed Weight Distribution for Low Bit Quantization of Deep Models Interspeech 2022 Stability and Generalization Analysis of Gradient Methods for Shallow Neural Networks NeurIPS 2022 Stability of SGD: Tightness analysis and improved bounds UAI 2022 Stability vs Implicit Bias of Gradient Methods on Separable Data and Beyond COLT 2022 STGN: an Implicit Regularization Method for Learning with Noisy Labels in Natural Language Processing EMNLP 2022 Stochastic Adaptive Activation Function NeurIPS 2022