Neural Network Optimization

Critical Initialization of Wide and Deep Neural Networks using Partial Jacobians: General Theory and Applications NeurIPS 2023 Curvature-Balanced Feature Manifold Learning for Long-Tailed Classification CVPR 2023 Deep Dive Into Gradients: Better Optimization for 3D Object Detection With Gradient-Corrected IoU Supervision CVPR 2023 Demystifying Prompts in Language Models via Perplexity Estimation EMNLP 2023 Differentiable Architecture Search With Random Features CVPR 2023 Dissecting the Effects of SGD Noise in Distinct Regimes of Deep Learning ICML 2023 DMIS: Dynamic Mesh-Based Importance Sampling for Training Physics-Informed Neural Networks AAAI 2023 DoG is SGD’s Best Friend: A Parameter-Free Dynamic Step Size Schedule ICML 2023 Dropout Reduces Underfitting ICML 2023 EfficientViT: Memory Efficient Vision Transformer With Cascaded Group Attention CVPR 2023 Equilibrium Point Learning ACML 2023 Eva: Practical Second-order Optimization with Kronecker-vectorized Approximation ICLR 2023 Expected Gradients of Maxout Networks and Consequences to Parameter Initialization ICML 2023 Experimental Observations of the Topology of Convolutional Neural Network Activations AAAI 2023 Fast Convergence in Learning Two-Layer Neural Networks with Separable Data AAAI 2023 Fast Saturating Gate for Learning Long Time Scales with Recurrent Neural Networks AAAI 2023 Fixed-Weight Difference Target Propagation AAAI 2023 From high-dimensional & mean-field dynamics to dimensionless ODEs: A unifying approach to SGD in two-layers networks COLT 2023 Global Vision Transformer Pruning With Hessian-Aware Saliency CVPR 2023 GWQ: Group-Wise Quantization Framework for Neural Networks ACML 2023 Implicit Bias of Gradient Descent for Mean Squared Error Regression with Two-Layer Wide Neural Networks JMLR 2023 Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data NeurIPS 2023 Improved Convergence in High Probability of Clipped Gradient Methods with Heavy Tailed Noise NeurIPS 2023 Improved Powered Stochastic Optimization Algorithms for Large-Scale Machine Learning JMLR 2023 Initialization Noise in Image Gradients and Saliency Maps CVPR 2023