Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Targeted Token-Level Unlearning for Repairing Text-to-SQL Models

AbstractText-to-SQL models may exhibit recurring error generation patterns that are reproducible under a fixed model and difficult to eliminate through continued training. Existing approaches either perform post-hoc correction at the output level or require access to the target domain’s training data for preference optimization, neither of which directly targets these reproducible local decision tendencies. To address these limitations, we propose Targeted Token-level Unlearning for SQL (TTU-SQL), a machine-unlearning-based framework that uses a probe dataset to detect and correct systematic errors without requiring access to the model’s original training data. The framework comprises three components: (1) AST-based token-level divergence detection that precisely localizes erroneous decision points; (2) a targeted error-type extraction procedure that selects three specific types of forgettable errors based on AST divergence and logit-level behavior; and (3) Part-NPO, which applies negative preference optimization only to localized divergence tokens to suppress erroneous continuations while preserving useful SQL generation knowledge. Across the Spider and BIRD datasets and all evaluated model backbones, TTU-SQL yields consistent execution-accuracy gains over equal-budget training-based repair baselines.

More from EMNLP 2026

All EMNLP 2026 papers →