ParaRater: Enhancing Cross-Lingual Transfer in LLMs with Meta-Learning
Abstract
AbstractMultilingual LLMs increasingly claim broad language coverage, yet their performance still lags far behind English in many languages. Because low resource data is scarce and noisy, improving these languages directly is difficult, making cross lingual transfer from English a natural alternative. Parallel corpora are central to such transfer, but it remains unclear which kinds of parallel data matter most. We propose ParaRater, a corpus selection method that identifies high value English text to translate into target languages, building parallel corpora that most efficiently improve target language performance. ParaRater uses meta learning to tie selection directly to downstream accuracy on native target language data, and applies a two stage filter to find samples that only help when both language versions are used in training, capturing truly impactful parallel data. Experiments across Chinese, German, French, Japanese, Thai, Arabic, Tagalog, and Russian, together with qualitative analyses, support these findings.