Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Gradient-Aligned Iterative Fine-Tuning for Adaptive LLM Post-Training

AbstractPost-training has become the primary mechanism for grounding large language models (LLMs) to application-specific domains, enabling them to acquire new knowledge, behaviors, and policies beyond pre-training. Supervised fine-tuning (SFT) and preference optimization are the dominant paradigms for this adaptation, but are typically applied sequentially, requiring careful tuning of training schedules, regularization, and other hyperparameters.We propose Gradient-Aligned Iterative Fine-Tuning (GAIF), an adaptive post-training framework that combines three complementary components: a unified SFT+DPO objective optimized simultaneously; a prospective adaptive weighting mechanism that selects task weights based on next-step gradient geometry; and an iterative outer loop that regenerates preference pairs from the current model’s failures, keeping the supervision signal synchronized with the model’s evolving capabilities while adapting training toward arbitrary evaluation metrics, including non-differentiable ones.We evaluate GAIF on a production-scale generative movie search assistant, where a 7B LLM is adapted to the entertainment domain while remaining faithful to a fixed streaming catalog. The 7B model trained with GAIF outperforms standard SFT, sequential SFT+DPO, and a ~90B reference model in both ranking quality (NDCG) and output validity. Experiments on GSM8K mathematical reasoning further show the algorithm’s effectiveness extends beyond grounded retrieval.

More from EMNLP 2026

All EMNLP 2026 papers →