conftrace_

2021 ICML ICML 2021

Near-Optimal Linear Regression under Distribution Shift

Abstract

Transfer learning is essential when sufficient data comes from the source domain, with scarce labeled data from the target domain. We develop estimators that achieve minimax linear risk for linear regression problems under distribution shift. Our algorithms cover different transfer learning settings including covariate shift and model shift. We also consider when data are generated from either linear or general nonlinear models. We show that linear minimax estimators are within an absolute constant of the minimax risk even among nonlinear estimators for various source/target distributions.

Authors

Qi Lei, Wei Hu, Jason Lee

Topics

Artificial Intelligence > Learning Paradigms > Transfer Learning Machine Learning > Core Methods > Regression Machine Learning > Optimization & Theory > Optimization Machine Learning > Application Areas > Domain Adaptation Machine Learning > Learning Types > Transfer Learning

Keywords

transfer learning domain adaptation covariate shift distribution shift linear regression minimax risk nonlinear model

More from ICML 2021

All ICML 2021 papers →

SUNRISE: A Simple Unified Framework for Ensemble Learning in Deep Reinforcement Learning

Achieving Near Instance-Optimality and Minimax-Optimality in Stochastic and Adversarial Linear Bandits Simultaneously

PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training

Stability and Generalization of Stochastic Gradient Methods for Minimax Problems

Scalable Evaluation of Multi-Agent Reinforcement Learning with Melting Pot

Better Training using Weight-Constrained Stochastic Dynamics