When LLM Plans Fail: Solver-Verified Remediation Under Operational Constraints
Abstract
AbstractWe study postmortem remediation planning for large-scale cloud incidents, focusing on whether generated actions are feasible, safe, and executable under real operational constraints. While LLM-based systems can produce structured corrective actions, we find that 25–35% of generated plans across 8,200 real-world incidents violate at least one feasibility constraint due to hidden dependencies, policy limits, or risk budgets.We introduce Solver-Verified Remediation Planning (SVRP), a decision-centric framework that separates action proposal from feasibility verification. An LLM proposes candidate actions, which are verified using an explicit constraint system and satisfiability solving; minimal unsatisfiable cores guide targeted repair or abstention. SVRP raises feasibility from 62.4% to over 93.1%, reduces unsafe actions by more than 50%, and improves expert-rated executability from 3.7 to 4.7, while preserving semantic intent. These results suggest that remediation failures often reflect structural constraints that benefit from solver-backed verification in addition to improved prompting.