Skip to content
conftrace_
2002 JMLR JMLR 2002

On the Convergence of Optimistic Policy Iteration

Abstract

We consider a finite-state Markov decision problem and establish the convergence of a special case of optimistic policy iteration that involves Monte Carlo estimation of Q -values, in conjunction with greedy policy selection. We provide convergence results for a number of algorithmic variations, including one that involves temporal difference learning (bootstrapping) instead of Monte Carlo estimation. We also indicate some extensions that either fail or are unlikely to go through. [abs] [pdf] [ps.gz] [ps]

🌱 Topic Pioneer - Policy Learning
📈 Trend Setter - Policy Learning
🧭 Keyword Pioneer - temporal difference learning
🐣 Hot Topic Early Bird - markov decision process

Authors

John N. Tsitsiklis

Topics

Reinforcement Learning > Methods > Policy Learning Reinforcement Learning > Applications > Value Iteration

Keywords

temporal difference learning markov decision process monte carlo estimation policy iteration greedy policy
Download PDF

More from JMLR 2002

All JMLR 2002 papers →
Round Robin Classification
Kernel Independent Component Analysis
Learning Monotone DNF from a Teacher that Almost Does Not Answer Membership Queries
Data-dependent margin-based generalization bounds for classification
Variational Learning of Clusters of Undercomplete Nonsymmetric Independent Components
Learning Precise Timing with LSTM Recurrent Networks