Optimization & Theory

On the Explicit Role of Initialization on the Convergence and Implicit Bias of Overparametrized Linear Networks ICML 2021 On the Frequency Bias of Generative Models NeurIPS 2021 On the Generalization Power of Overfitted Two-Layer Neural Tangent Kernel Models ICML 2021 On the Implicit Bias of Initialization Shape: Beyond Infinitesimal Mirror Descent ICML 2021 On the Limits of Minimal Pairs in Contrastive Evaluation EMNLP 2021 On the Origin of Implicit Regularization in Stochastic Gradient Descent ICLR 2021 On the Predictability of Pruning Across Scales ICML 2021 On the Proof of Global Convergence of Gradient Descent for Deep ReLU Networks with Linear Widths ICML 2021 On the Random Conjugate Kernel and Neural Tangent Kernel ICML 2021 On the Sample Complexity of Learning under Geometric Stability NeurIPS 2021 On the Softmax Bottleneck of Recurrent Language Models AAAI 2021 On the Stochastic Stability of Deep Markov Models NeurIPS 2021 On the Theory of Implicit Deep Learning: Global Convergence with Implicit Layers ICLR 2021 On the Verification of Neural ODEs with Stochastic Guarantees AAAI 2021 Opening the Blackbox: Accelerating Neural Differential Equations by Regularizing Internal Solver Heuristics ICML 2021 OpEvo: An Evolutionary Method for Tensor Operator Optimization AAAI 2021 OPQ: Compressing Deep Neural Networks with One-shot Pruning-Quantization AAAI 2021 Optimal Gradient Checkpoint Search for Arbitrary Computation Graphs CVPR 2021 Optimal Quantization Using Scaled Codebook CVPR 2021 Optimal Sketching for Trace Estimation NeurIPS 2021 Optimal Transport for Deep Generative Models: State of the Art and Research Challenges IJCAI 2021 Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth ICML 2021 Optimizing Deeper Transformers on Small Datasets ACL 2021 Optimizing Information Theory Based Bitwise Bottlenecks for Efficient Mixed-Precision Activation Quantization AAAI 2021 Optimizing Reusable Knowledge for Continual Learning via Metalearning NeurIPS 2021