conftrace_

2013 NeurIPS NeurIPS 2013

Compete to Compute

Abstract

Local competition among neighboring neurons is common in biological neural networks (NNs). We apply the concept to gradient-based, backprop-trained artificial multilayer NNs. NNs with competing linear units tend to outperform those with non-competing nonlinear units, and avoid catastrophic forgetting when training sets change over time.

🌱 Topic Pioneer - Continual Learning

📈 Trend Setter - Adversarial Learning

🧭 Keyword Pioneer - continual learning

🐣 Hot Topic Early Bird - continual learning

Authors

Rupesh K Srivastava, Jonathan Masci, Sohrob Kazerounian, Faustino Gomez, Jürgen Schmidhuber

Topics

Machine Learning > Core Methods > Classification Machine Learning > Learning Types > Continual Learning Deep Learning > Architectures > Neural Networks Artificial Intelligence > Core AI > Adversarial Learning Deep Learning > Learning Types > Deep Learning Machine Learning > Core Methods > Neural Networks Deep Learning > Learning Types > Continual Learning

Keywords

continual learning catastrophic forgetting feature learning deep learning local competition multilayer neural networks backprop training backpropagation training gradient-based learning neural network competing linear unit

More from NeurIPS 2013

All NeurIPS 2013 papers →

Generalizing Analytic Shrinkage for Arbitrary Covariance Structures

Higher Order Priors for Joint Intrinsic Image, Objects, and Attributes Estimation

Online Robust PCA via Stochastic Optimization

Heterogeneous-Neighborhood-based Multi-Task Local Learning Algorithms

Scalable Influence Estimation in Continuous-Time Diffusion Networks

More data speeds up training time in learning halfspaces over sparse vectors