Skip to content
conftrace_
2007 NeurIPS NeurIPS 2007

Fitted Q-iteration in continuous action-space MDPs

Abstract

We consider continuous state, continuous action batch reinforcement learning where the goal is to learn a good policy from a sufficiently rich trajectory generated by another policy. We study a variant of fitted Q-iteration, where the greedy action selection is replaced by searching for a policy in a restricted set of candidate policies by maximizing the average action values. We provide a rigorous theoretical analysis of this algorithm, proving what we believe is the first finite-time bounds for value-function based algorithms for continuous state- and action-space problems.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Reinforcement Learning
📈 Trend Setter - Offline RL
🧭 Keyword Pioneer - batch learning
🐣 Hot Topic Early Bird - markov decision process

Authors

András Antos, Csaba Szepesvári, Rémi Munos

Topics

Machine Learning > Optimization & Theory > Learning Theory Reinforcement Learning > Methods > Deep RL Reinforcement Learning > Methods > Offline RL Reinforcement Learning > Methods > Policy Learning Machine Learning > Learning Types > Reinforcement Learning

Keywords

reinforcement learning markov decision process value function continuous control fitted q-iteration batch learning value function approximation batch reinforcement learning continuous state space continuous action space finite-time bound
Download PDF

More from NeurIPS 2007

All NeurIPS 2007 papers →
Probabilistic Matrix Factorization
Density Estimation under Independent Similarly Distributed Sampling Assumptions
Efficient Inference for Distributions on Permutations
Blind channel identification for speech dereverberation using l1-norm sparse learning
Predicting Brain States from fMRI Data: Incremental Functional Principal Component Regression
Agreement-Based Learning