2012
NeurIPS
NeurIPS 2012
Gradient Weights help Nonparametric Regressors
Abstract
In regression problems over $\real^d$, the unknown function $f$ often varies more in some coordinates than in others. We show that weighting each coordinate $i$ with the estimated norm of the $i$th derivative of $f$ is an efficient way to significantly improve the performance of distance-based regressors, e.g. kernel and $k$-NN regressors. We propose a simple estimator of these derivative norms and prove its consistency. Moreover, the proposed estimator is efficiently learned online.
🌉
Interdisciplinary Bridge
- Artificial Intelligence and Mathematics & Optimization
🐣
Hot Topic Early Bird
- k-nearest neighbor
Authors
Topics
Machine Learning > Core Methods > Regression
Machine Learning > Learning Types > Unsupervised Learning
Machine Learning > Optimization & Theory > Theory
Mathematics & Optimization > Optimization > Continuous Optimization
Machine Learning > Learning Types > Online Learning
Machine Learning > Learning Types > Supervised Learning
Machine Learning > Core Methods > Feature Learning
Machine Learning > Core Methods > Kernel Methods
Keywords
feature learning
online learning
nonparametric regression
kernel regression
adaptive regression
nearest neighbor regression
feature weighting
distance-based regressors
derivative norm estimation
k-nearest neighbor
derivative estimation
kernel regressor
kernel methods
gradient weighting
derivative norm
k-nn regressor