Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Want Better Synthetic Data? Steer It: Activation Steering for Low-Resource Language Generation

AbstractLarge language models (LLMs) have become an effective tool for synthetic data generation, including for low-resource languages, where generated data can improve downstream task performance. Current best-performing approaches typically rely on few-shot prompting with target-language examples, which increases inference costs and may reduce diversity through lexical anchoring. In this work, we investigate activation steering as an alternative for low-resource synthetic data generation. We study two steering strategies: Language Steering, which targets the linguistic identity of a language, and Quality Steering, which captures well-formedness by contrasting human-written and backtranslated text representations. We evaluate these methods across four open-source LLMs, multiple layers, and 11 typologically diverse languages by generating sentiment and topic classification data and fine-tuning smaller classifiers. Steering is applied in both zero-shot and few-shot prompting settings and compared against non-steered counterparts. Our results show that steering on early layers consistently improves the diversity of generated data while often yielding stronger downstream model performance, particularly for low-resource languages.

❓ The Questioner
🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →