Layer-Aware Native Quantization for Compact Machine Translation: A WMT26 English-to-Chinese Submission
Abstract
AbstractThis paper describes a constrained English-to-Simplified-Chinese submission to the WMT26 Model Compression shared task. The submitted system is derived from google/gemma-3-12b-it and applies native 4-bit NormalFloat quantization only to the text-transformer MLP projections: gate_proj, up_proj, and down_proj. Attention projections, embeddings, normalization layers, the language-model head, and vision modules remain in BF16. This work continues the selected-layer compression line introduced in prior speech-translation work, but shifts it from storage-oriented codec compression to deployment-oriented native quantization for text MT. The central finding is that selective native quantization is a quality-preserving compromise: it recovers significant quality lost by global q4 while avoiding the BF16-like memory expansion of the earlier q3+Zstd codec. On a local 332-segment WMT25 English-Chinese reference set, the system obtains 28.30 chrF and 31.37 BLEU, compared with 27.67 chrF and 30.80 BLEU for the organizer global-q4 baseline. The submitted artifact is 10.99 GiB and reaches 14.64 GiB peak VRAM locally, placing it between the smaller global-q4 baseline and the higher-quality BF16 model.