Unveiling Markov heads in Pretrained Language Models for Offline Reinforcement Learning

Wenhao Zhao; Qiushui Xu; Linjie Xu; Lei Song; Jinyu Wang; Chunlai Zhou; Jiang Bian

2025 ICML ICML 2025

Unveiling Markov heads in Pretrained Language Models for Offline Reinforcement Learning

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Reinforcement Learning

Authors

Wenhao Zhao, Qiushui Xu, Linjie Xu, Lei Song, Jinyu Wang, Chunlai Zhou, Jiang Bian

Topics

Deep Learning > Architectures > Transformers Reinforcement Learning > Methods > Offline RL Artificial Intelligence > Core AI > Large Language Models

Keywords

offline reinforcement learning adaptive learning decision transformer pre-trained language model markov head mixture of attention

Download PDF