conftrace_

vanishing gradient

43 papers

Explore in graph

Co-occurring keywords

recurrent neural network (1790) exploding gradient (13) gradient descent (1144) orthogonal matrix (29) long-term dependency (44) neural network (6616) residual network (211) neural network optimization (1293) sequence modeling (288) gradient propagation (34)

Papers

Only Strict Saddles in the Energy Landscape of Predictive Coding Networks? NIPS 2024
Recurrent neural networks: vanishing and exploding gradients are not the end of the story NIPS 2024
Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobians NIPS 2023
Why Capsule Neural Networks Do Not Scale: Challenging the Dynamic Parse-Tree Assumption AAAI 2023
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization ICML 2023
On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology ICML 2023
Clipped Hyperbolic Classifiers Are Super-Hyperbolic Classifiers CVPR 2022
AFRNN: Stable RNN with Top Down Feedback and Antisymmetry ACML 2022
Vanishing Curvature in Randomly Initialized Deep ReLU Networks AISTATS 2022
On the difficulty of learning chaotic dynamics with RNNs NIPS 2022
Implicit Bias of Linear RNNs ICML 2021
UnICORNN: A recurrent model for learning very long time dependencies ICML 2021
A unified framework for Hamiltonian deep neural networks L4DC 2021
Shuffling Recurrent Neural Networks AAAI 2021
Time Adaptive Recurrent Neural Network CVPR 2021
MomentumRNN: Integrating Momentum into Recurrent Neural Networks NIPS 2020
On the Almost Sure Convergence of Stochastic Gradient Descent in Non-Convex Problems NIPS 2020
Beyond exploding and vanishing gradients: analysing RNN training using attractors and smoothness AISTATS 2020
Target Propagation in Recurrent Neural Networks JMLR 2020
Linear Antisymmetric Recurrent Neural Networks L4DC 2020
Eigenvalue Normalized Recurrent Neural Networks for Short Term Memory AAAI 2020
Temporal Pyramid Recurrent Neural Network AAAI 2020
Untangling tradeoffs between recurrence and self-attention in artificial neural networks NIPS 2020
Propagation Mechanism for Deep and Wide Neural Networks CVPR 2019
Towards Non-Saturating Recurrent Units for Modelling Long-Term Dependencies AAAI 2019