ContextRouter: Learning When to Use Context for Efficient LLM Machine Translation
Abstract
AbstractLLM-based machine translation systems can now do more than translate a source segment in isolation. At test time, they can translate directly, prepend previous context, or refine a context-aware draft. However, these interventions are not uniformly beneficial. Context can resolve discourse-sensitive choices, but irrelevant context and extra refinement increases cost and may perturb adequate translations. We introduce ContextRouter, a lightweight policy router that learns which intervention to apply for each source segment. During training, we first generate candidates for all contextual interventions and score them with a routing objective combining overall translation quality with incurred token costs. We then train our ContextRouter with group-relative supervision, so it learns within-example preferences rather than absolute global action scores. At inference, our router uses only test-time-available signals from the source, previous source-side context, and a no-context draft to determine and execute only the selected context intervention. On a filtered context-aware subset of TED, QED, and OpenSubtitles covering multiple languages and domains, our ContextRouter improves the quality-cost tradeoff over fixed no-context, always-context, always-refinement, QE-reranking, and score-regression policies, and human evaluation confirms that annotators prefer its chosen interventions. These results support treating context and refinement actions in LLM-MT not as always present defaults but conditional test-time interventions instead.