Skip to content
conftrace_
2026 EMNLP EMNLP 2026

From Evasive Servants to Dignified Peers via Dual Preference Optimization

AbstractCurrent aligned language models exhibit a dual failure mode we term the Evasive Servant: they sycophantically validate flawed user beliefs while deflecting responsibility with boilerplate disclaimers. We propose the Dignified Peer framework, which targets servility through anti-sycophancy and trustworthiness, and evasiveness through empathy and creativity. Operationalizing this framework requires addressing three concrete gaps: lack of fine-grained per-dimension supervision, objective imbalance during multi-objective optimization, and confound-prone evaluation. We introduce the PersonaKnob dataset, which encodes a compositional partial-order structure of multi-persona preferences, and pair it with Lag-DPO, a tolerance-constrained Lagrangian DPO algorithm that dynamically rebalances dimensions to mitigate collapse. For evaluation, we adapt the Many-Facet Rasch Model (MFRM) to disentangle model capability from judge severity, rubric difficulty, and question complexity, and we validate MFRM scores against human pairwise preferences (Spearman 𝜌 ∈ [0.67, 0.85] across dimensions). Across two open-source backbones, Lag-DPO improves all four dimensions in distribution and reduces sycophancy while preserving safe-refusal balance on out-of-distribution benchmarks; we report these results with explicit weight-sensitivity and uncertainty analyses rather than claim a fully resolved Dignified Peer.

More from EMNLP 2026

All EMNLP 2026 papers →