Neural Network Optimization

HMM-Free Encoder Pre-Training for Streaming RNN Transducer Interspeech 2021 HourNAS: Extremely Fast Neural Architecture Search Through an Hourglass Lens CVPR 2021 How Benign is Benign Overfitting ? ICLR 2021 How Much Over-parameterization Is Sufficient to Learn Deep ReLU Networks? ICLR 2021 How much pretraining data do language models need to learn syntax? EMNLP 2021 H-Transformer-1D: Fast One-Dimensional Hierarchical Attention for Sequences ACL 2021 Hyperparameter Power Impact in Transformer Language Model Training EMNLP 2021 Hyperparameter Tuning is All You Need for LISTA NeurIPS 2021 HyperSeg: Patch-Wise Hypernetwork for Real-Time Semantic Segmentation CVPR 2021 Identifying nonlinear dynamical systems with multiple time scales and long-range dependencies ICLR 2021 Impact of Aliasing on Generalization in Deep Convolutional Networks ICCV 2021 Implicit Bias of Linear RNNs ICML 2021 Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity NeurIPS 2021 Implicit Gradient Regularization ICLR 2021 Implicit Kernel Attention AAAI 2021 Implicit Regularization in Tensor Factorization ICML 2021 Implicit Self-Regularization in Deep Neural Networks: Evidence from Random Matrix Theory and Implications for Learning JMLR 2021 Implicit Sparse Regularization: The Impact of Depth and Early Stopping NeurIPS 2021 Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning ICLR 2021 Importance-based Neuron Allocation for Multilingual Neural Machine Translation ACL 2021 Improved, Deterministic Smoothing for L_1 Certified Robustness ICML 2021 Improving Adversarial Robustness via Channel-wise Activation Suppressing ICLR 2021 Improving Anytime Prediction with Parallel Cascaded Networks and a Temporal-Difference Loss NeurIPS 2021 Improving Deep Learning Interpretability by Saliency Guided Training NeurIPS 2021 Improving Molecular Graph Neural Network Explainability with Orthonormalization and Induced Sparsity ICML 2021