Neural Network Optimization

Rethinking Influence Functions of Neural Networks in the Over-Parameterized Regime AAAI 2022 Revisiting Random Channel Pruning for Neural Network Compression CVPR 2022 Robustness in deep learning: The good (width), the bad (depth), and the ugly (initialization) NeurIPS 2022 Scaled ReLU Matters for Training Vision Transformers AAAI 2022 Scaling Up Your Kernels to 31x31: Revisiting Large Kernel Design in CNNs CVPR 2022 Searching the Deployable Convolution Neural Networks for GPUs CVPR 2022 Smooth Maximum Unit: Smooth Activation Function for Deep Networks Using Smoothing Maximum Technique CVPR 2022 Stochastic Adaptive Activation Function NeurIPS 2022 Subspace Adversarial Training CVPR 2022 Super-Acceleration with Cyclical Step-sizes AISTATS 2022 The Devil in Linear Transformer EMNLP 2022 The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models NeurIPS 2022 To update or not to update? Neurons at equilibrium in deep models NeurIPS 2022 Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis NeurIPS 2022 Towards Efficient and Scalable Sharpness-Aware Minimization CVPR 2022 Towards Efficient NLP: A Standard Evaluation and A Strong Baseline NAACL 2022 Towards Noise-adaptive, Problem-adaptive (Accelerated) Stochastic Gradient Descent ICML 2022 Towards One Shot Search Space Poisoning in Neural Architecture Search (Student Abstract) AAAI 2022 Towards Practical Adam: Non-Convexity, Convergence Theory, and Mini-Batch Acceleration JMLR 2022 Towards Practical Certifiable Patch Defense With Vision Transformer CVPR 2022 Towards Practical Control of Singular Values of Convolutional Layers NeurIPS 2022 Towards Theoretically Inspired Neural Initialization Optimization NeurIPS 2022 Training Scale-Invariant Neural Networks on the Sphere Can Happen in Three Regimes NeurIPS 2022 Training Your Sparse Neural Network Better with Any Mask ICML 2022 Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions NeurIPS 2022