Skip to content
conftrace_
2024 ICML ICML 2024

Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Data

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Reinforcement Learning

Authors

Fahim Tajwar, Anikait Singh, Archit Sharma, Rafael Rafailov, Jeff Schneider, Tengyang Xie, Stefano Ermon, Chelsea Finn, Aviral Kumar

Topics

Machine Learning > Optimization & Theory > Optimization Reinforcement Learning > Methods > Offline RL Artificial Intelligence > Core AI > Large Language Models

Keywords

on-policy sampling preference fine-tuning categorical distribution likelihood mode-seeking objectives negative gradient
Download PDF