vanishing gradient

MomentumRNN: Integrating Momentum into Recurrent Neural Networks NeurIPS 2020 On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems NeurIPS 2020 RNNs Incrementally Evolving on an Equilibrium Manifold: A Panacea for Vanishing and Exploding Gradients? ICLR 2020 Target Propagation in Recurrent Neural Networks JMLR 2020 Temporal Pyramid Recurrent Neural Network AAAI 2020 Untangling tradeoffs between recurrence and self-attention in artificial neural networks NeurIPS 2020 Complex Unitary Recurrent Neural Networks Using Scaled Cayley Transform AAAI 2019 h-detach: Modifying the LSTM Gradient Towards Better Optimization ICLR 2019 Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics NeurIPS 2019 Propagation Mechanism for Deep and Wide Neural Networks CVPR 2019 Towards Non-Saturating Recurrent Units for Modelling Long-Term Dependencies AAAI 2019 Can recurrent neural networks warp time? ICLR 2018 Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks ICML 2018 How to Start Training: The Effect of Initialization and Architecture NeurIPS 2018 Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum ACL 2018 Non-parametric estimation of Jensen-Shannon Divergence in Generative Adversarial Network training AISTATS 2018 On the Local Hessian in Back-propagation NeurIPS 2018 Orthogonal Recurrent Neural Networks with Scaled Cayley Transform ICML 2018 Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization ICML 2018 Which Neural Net Architectures Give Rise to Exploding and Vanishing Gradients? NeurIPS 2018 Dilated Recurrent Neural Networks NeurIPS 2017 Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections ICML 2017 Mean Field Residual Networks: On the Edge of Chaos NeurIPS 2017 The Shattered Gradients Problem: If resnets are the answer, then what is the question? ICML 2017 Full-Capacity Unitary Recurrent Neural Networks NeurIPS 2016