Skip to content
conftrace_
2026 EMNLP EMNLP 2026

VISA: Value Injection via Shielded Adaptation for Personalized LLM Alignment

AbstractPersonalizing large language models often requires adapting them to new domains, tasks, or user preferences. However, such adaptation can unintentionally shift a model’s value profile, even when the training data is primarily knowledge-oriented. We refer to this failure mode as value drift, a form of value alignment tax that emerges when learning new knowledge interferes with pre-existing alignment. To address this problem, we propose VISA, a modular framework for Value Injection via Shielded Adaptation. VISA keeps the base model as a stable content generator and trains a lightweight Value Rewriter to adjust the value expression of generated responses toward a target value vector. The target is constructed through a Value Detector and an Instruction Translator, while the rewriter is optimized with Group Relative Policy Optimization using rewards for value alignment and semantic consistency. We further construct VCR-45K, a value-conditioned rewriting dataset for evaluating controllable value adaptation. Experiments show that VISA substantially improves semantic preservation over prompting and fine-tuning baselines while maintaining competitive value alignment, enabling low-cost personalized alignment with reduced value drift.

More from EMNLP 2026

All EMNLP 2026 papers →