conftrace_

2017 NeurIPS NeurIPS 2017

Exploring Generalization in Deep Learning

Abstract

With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustness. We study how these measures can ensure generalization, highlighting the importance of scale normalization, and making a connection between sharpness and PAC-Bayes theory. We then investigate how well the measures explain different observed phenomena.

📈 Trend Setter - Robustness

🐣 Hot Topic Early Bird - deep network

Authors

Behnam Neyshabur, Srinadh Bhojanapalli, David Mcallester, Nati Srebro

Topics

Machine Learning > Optimization & Theory > Learning Theory Machine Learning > Optimization & Theory > Theory Deep Learning > Learning Types > Representation Learning Machine Learning > Learning Types > Robustness Machine Learning > Optimization & Theory > Generalization

Keywords

pac-bayes theory generalization bound deep network sharpness measure norm-based control

More from NeurIPS 2017

All NeurIPS 2017 papers →

Identification of Gaussian Process State Space Models

Estimation of the covariance structure of heavy-tailed distributions

Robust Optimization for Non-Convex Objectives

Spherical convolutions and their application in molecular modelling

Safe Adaptive Importance Sampling

Introspective Classification with Convolutional Nets