Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Is LoRA Robust to Adversarial Attacks? Expectation-Based Regularization for Parameter-Efficient Adversarial Training

AbstractLow-Rank Adaptation (LoRA) is widely used for parameter-efficient fine-tuning, yet its robustness to textual adversarial perturbations remains understudied. Existing adversarial training methods often improve robustness by solving costly min-max objectives with inner perturbation optimization with suboptimal results. In this research, we propose Expectation-Regularized LoRA (ER-LoRA), a simple expectation-based regularization approach that improves the robustness of LoRA without requiring inner maximization or adversarial example generation. ER-LoRA smooths predictions over a LoRA-aware low-dimensional perturbation distribution and admits a closed-form curvature-aware objective via a local Gaussian approximation. It adds no trainable parameters and keeps the LoRA architecture unchanged. Experiments on adversarial NLP benchmarks and textual attacks show that ER-LoRA improves robustness while preserving clean accuracy and parameter efficiency.

More from EMNLP 2026

All EMNLP 2026 papers →