conftrace_

vanishing gradient

43 papers

Explore in graph

Co-occurring keywords

recurrent neural network (1790) exploding gradient (13) gradient descent (1144) orthogonal matrix (29) long-term dependency (44) neural network (6616) residual network (211) neural network optimization (1293) sequence modeling (288) gradient propagation (34)

Papers

Complex Unitary Recurrent Neural Networks Using Scaled Cayley Transform AAAI 2019
Non-normal Recurrent Neural Network (nnRNN): learning long time dependencies while improving expressivity with transient dynamics NIPS 2019
Which Neural Net Architectures Give Rise to Exploding and Vanishing Gradients? NIPS 2018
How to Start Training: The Effect of Initialization and Architecture NIPS 2018
Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks ICML 2018
Orthogonal Recurrent Neural Networks with Scaled Cayley Transform ICML 2018
Long Short-Term Memory as a Dynamically Computed Element-wise Weighted Sum ACL 2018
Non-parametric estimation of Jensen-Shannon Divergence in Generative Adversarial Network training AISTATS 2018
On the Local Hessian in Back-propagation NIPS 2018
Stabilizing Gradients for Deep Neural Networks via Efficient SVD Parameterization ICML 2018
The Shattered Gradients Problem: If resnets are the answer, then what is the question? ICML 2017
Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections ICML 2017
Mean Field Residual Networks: On the Edge of Chaos NIPS 2017
Dilated Recurrent Neural Networks NIPS 2017
Tensor Switching Networks NIPS 2016
Residual Networks Behave Like Ensembles of Relatively Shallow Networks NIPS 2016
Full-Capacity Unitary Recurrent Neural Networks NIPS 2016
On the difficulty of training recurrent neural networks ICML 2013