Skip to content
conftrace_
2026 ICML ICML 2026

Toward Subspace-Perturbed Trajectory-Aware Backdoor Attacks in Deep Reinforcement Learning

Deep Reinforcement Learning agents are increasingly used in safety-critical domains but remain vulnerable to stealthy backdoor attacks. Existing outer-loop attacks face a trade-off between perceptual stealth, poisoning efficiency, and value-function consistency, often making the attack ineffective or easily exposed. To address these challenges, we propose SpecDRL, a unified framework that (1) embeds triggers in the least sensitive subspaces of the state manifold via Subspace-Aware Injection, exploiting perceptual blind spots, (2) selects the most influential time steps for poisoning through Value-Guided Strategic Sampling based on Return-to-Go and Temporal-Difference error, and (3) preserves reward integrity via Bellman-Consistent Dynamic Reward Poisoning, which analytically enforces $\epsilon$consistency of value functions and bounds global return deviations. Experiments across 12 Atari environments demonstrate that SpecDRL achieves near-100% attack success, accelerates backdoor convergence, and maintains benign task performance.

More from ICML 2026

All ICML 2026 papers →