Neural Network Optimization

TRAMS: Training-free Memory Selection for Long-range Language Modeling EMNLP 2023 Tuning Frequency Bias in Neural Network Training with Nonuniform Data ICLR 2023 Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias ICML 2023 Understanding Imbalanced Semantic Segmentation Through Neural Collapse CVPR 2023 Understanding Multi-phase Optimization Dynamics and Rich Nonlinear Behaviors of ReLU Networks NeurIPS 2023 Understanding Representation Learnability of Nonlinear Self-Supervised Learning AAAI 2023 Win: Weight-Decay-Integrated Nesterov Acceleration for Adaptive Gradient Algorithms ICLR 2023 Zero-Cost Operation Scoring in Differentiable Architecture Search AAAI 2023 A Closer Look at Parameter Contributions When Training Neural Language and Translation Models COLING 2022 A Closer Look at the Adversarial Robustness of Deep Equilibrium Models NeurIPS 2022 A Computable Definition of the Spectral Bias AAAI 2022 AdaBlock: SGD with Practical Block Diagonal Matrix Adaptation for Deep Learning AISTATS 2022 AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognition CVPR 2022 Adam Can Converge Without Any Modification On Update Rules NeurIPS 2022 Adaptive Accelerated (Extra-)Gradient Methods with Variance Reduction ICML 2022 Adversarial Branch Architecture Search for Unsupervised Domain Adaptation WACV 2022 A Momentumized, Adaptive, Dual Averaged Gradient Method JMLR 2022 An In-depth Study of Stochastic Backpropagation NeurIPS 2022 A Novel Differentiable Mixed-Precision Quantization Search Framework for Alleviating the Matthew Effect and Improving Robustness ACML 2022 A Sparse-Motif Ensemble Graph Convolutional Network against Over-smoothing IJCAI 2022 A Stochastic Momentum Accelerated Quasi-Newton Method for Neural Networks (Student Abstract) AAAI 2022 A Study on the Ramanujan Graph Property of Winning Lottery Tickets ICML 2022 A Unified Weight Initialization Paradigm for Tensorial Convolutional Neural Networks ICML 2022 Automatic differentiation of nonsmooth iterative algorithms NeurIPS 2022 BLOX: Macro Neural Architecture Search Benchmark and Algorithms NeurIPS 2022