Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Models

AbstractQuantization is widely used to deploy large language models, but its effect on uncertainty behavior, such as confidence, margins, and abstention, is rarely treated as a primary objective. We frame calibration-data selection for quantization as a *target-dependent uncertainty-preservation problem*. Different deployments emphasize different regions of the input distribution, yet prior work mainly optimizes accuracy-oriented compression metrics or adjusts scores after quantization. We formalize this goal with distributional and boundary preservation risks, and provide a simple mixture-mismatch argument explaining why no single calibration recipe should be expected to fit all targets. We introduce Doubt-Preserving Quantization (DPQ), a lightweight pre-quantization recipe family that uses full-precision predictions to construct target-aligned calibration mixtures of high-doubt examples and generic anchors. Across 8 language models, 9 NLP benchmarks, and 22 comparison methods, the leading recipe changes with the preservation target: DPQ-r75 leads on SQuAD2 answerability-boundary preservation, while broad multiple-choice QA favors a different set of milder or single-signal recipes, with confidence-only leading and variants such as entropy-only and DPQ-r50 outperforming the boundary-heavy DPQ-r75. These results show that calibration data should be selected for the specific full-precision score behavior a deployment needs to preserve, rather than treated as a fixed quantization detail. Code is available at https://github.com/xi-xiaoran/DPQ.

More from EMNLP 2026

All EMNLP 2026 papers →