2026
EMNLP
EMNLP 2026
Does the Greater-Than Circuit Generalize? A Reproducibility Study Across Models and Numerical Entailment
Abstract
AbstractWe reproduce the canonical GPT-2-small “greater-than” circuit in its original stack and independently, then test how far it generalizes across eight models (124M–12B) and to numerical entailment. The reported numbers replicate, and the late-MLP structure recurs at matched relative depth well above a size-matched null. Sparse faithfulness is high everywhere too, but only when the edge budget scales with the model’s own graph; since these graphs differ 96-fold in size, a fixed budget makes faithfulness appear to collapse with scale. On numerical entailment, no circuit-bearing model passes the 0.6 gate, while the one model that solves the task well cannot be posed the circuit’s own task.
❓
The Questioner