conftrace_

2019 COLT COLT 2019

Open Problem: How fast can a multiclass test set be overfit?

Abstract

We ask how many measurements of the accuracy on a multiclass benchmark are needed to achieve a given amount of overfitting.

❓ The Questioner

Authors

Vitaly Feldman, Roy Frostig, Moritz Hardt

Topics

Machine Learning > Core Methods > Classification Machine Learning > Optimization & Theory > Learning Theory Machine Learning > Optimization & Theory > Generalization

Keywords

learning theory benchmark evaluation sample complexity multi-class classification test set test set overfitting

More from COLT 2019

All COLT 2019 papers →

Estimating the Mixing Time of Ergodic Markov Chains

Stochastic Approximation of Smooth and Strongly Convex Functions: Beyond the $O(1/T)$ Convergence Rate

Open Problem: Is Margin Sufficient for Non-Interactive Private Distributed Learning?

Open Problem: Do Good Algorithms Necessarily Query Bad Points?

Open Problem: Risk of Ruin in Multiarmed Bandits

Open Problem: Monotonicity of Learning