Neural Network Optimization

h-detach: Modifying the LSTM Gradient Towards Better Optimization ICLR 2019 Heavy-ball Algorithms Always Escape Saddle Points IJCAI 2019 Hierarchical Context Enabled Recurrent Neural Network for Recommendation AAAI 2019 Hierarchical Pointer Net Parsing EMNLP 2019 How degenerate is the parametrization of neural networks with the ReLU activation function? NeurIPS 2019 How to Initialize your Network? Robust Initialization for WeightNorm & ResNets NeurIPS 2019 HyperAdam: A Learnable Task-Adaptive Adam for Network Training AAAI 2019 IA-NET: Acceleration and Compression of Speech Enhancement Using Integer-Adder Deep Neural Network Interspeech 2019 Implicit Regularization of Accelerated Methods in Hilbert Spaces NeurIPS 2019 Improved Differentiable Architecture Search for Language Modeling and Named Entity Recognition EMNLP 2019 Improving Deep Transformer with Depth-Scaled Initialization and Merged Attention EMNLP 2019 Information Aggregation for Multi-Head Attention with Routing-by-Agreement NAACL 2019 Initialization of ReLUs for Dynamical Isometry NeurIPS 2019 InteractionNN: A Neural Network for Learning Hidden Features in Sparse Prediction IJCAI 2019 Invertible Convolutional Flow NeurIPS 2019 IRLAS: Inverse Reinforcement Learning for Architecture Search CVPR 2019 Is Deeper Better only when Shallow is Good? NeurIPS 2019 Jumpout : Improved Dropout for Deep Neural Networks with ReLUs ICML 2019 Kernel RNN Learning (KeRNL) ICLR 2019 Large-Scale Distributed Second-Order Optimization Using Kronecker-Factored Approximate Curvature for Deep Convolutional Neural Networks CVPR 2019 Large Scale Structure of Neural Network Loss Landscapes NeurIPS 2019 Layer Trajectory BLSTM Interspeech 2019 LCA: Loss Change Allocation for Neural Network Training NeurIPS 2019 Leap-LSTM: Enhancing Long Short-Term Memory for Text Categorization IJCAI 2019 Learning and Generalization in Overparameterized Neural Networks, Going Beyond Two Layers NeurIPS 2019