Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Exploring Language-Agnosticity in Function Vectors: A Case Study in Machine Translation

AbstractFunction vectors (FVs) are vector representations of tasks extracted from model activations during in-context learning. While prior work has shown that multilingual model representations can be language-agnostic, it remains unclear whether the same holds for function vectors. We study whether FVs exhibit language-agnosticity, using word-level machine translation as a case study. Across three decoder-only multilingual LLMs, we find that translation FVs extracted from word-to-word prompts in a single English→X direction transfer to other target languages, consistently improving the rank of correct translation tokens across multiple unseen languages. We further find that the highest-gain tokens span multiple languages and that translation FVs across directions share most of their top-ranked heads, indicating that the FV encodes a largely language-agnostic translation signal.

More from EMNLP 2026

All EMNLP 2026 papers →