Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Laniqo at WMT26 Multilingual Instruction Shared Task (MIST): Full-Parameter Knowledge Distillation with Task-Conditional Pivot Decoding Under a 10B-Parameter Constraint

AbstractWe describe our submission to the WMT26 Multilingual Instruction Shared Task (MIST). Our system is google/gemma-4-E4B-it (8B total / 4.5B effective parameters), fully fine-tuned via knowledge distillation from a larger teacher model (Gemma-4-31B-it) on teacher-generated responses spanning three sub-tasks and 24 languages. We find that (i) a two-judge quality-agreement filter on the distillation data provides no measurable benefit over using the teacher’s outputs unfiltered, at both LoRA and full-parameter training scale, and (ii) a task- conditional decoding strategy (using two-turn English drafting for open-ended generation and summarization, versus single-turn direct decod- ing for context-grounded question answering) significantly improves quality where applied and is harmless where it is not. Our final submission combines full-parameter distillation with this task-conditional decoding scheme.

More from EMNLP 2026

All EMNLP 2026 papers →