Skip to content
conftrace_
2026 EMNLP EMNLP 2026

VIB-Simp: IB-Inspired Output-Logit Regularization for German Text Simplification

AbstractGerman text simplification requires reducing lexical and structural complexity while preserving meaning, yet standard fine-tuning optimizes reference imitation without directly controlling the predictive distribution where lexical choices are made. We propose VIB-Simp, a lightweight output-logit regularization framework inspired by the Variational Information Bottleneck (VIB) literature. Instead of estimating mutual information or adding latent variables, VIB-Simp regularizes the mean and variance of output logits using a closed-form Gaussian Kullback–Leibler (KL) surrogate, providing a lightweight way to stabilize lexical choices during fine-tuning. An optional unlikelihood-style lexical penalty further suppresses probability mass assigned to difficult lexical candidates.Experiments on TextComplexityDE and 20Min show consistent gains across LLaMA-3.2-3B, Qwen-2.5-3B, mBART, GerPT-2, and mGPT, with consistent SARI improvements. Additional baselines, length-controlled analysis, ablations, expert ratings, and logit diagnostics show that the gains are not explained by shortening or dictionary-only effects. The results support an IB-style view of simplification: preserve meaning-critical content while suppressing unnecessary lexical complexity.

More from EMNLP 2026

All EMNLP 2026 papers →