Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Does the Greater-Than Circuit Generalize? A Reproducibility Study Across Models and Numerical Entailment

AbstractWe reproduce the canonical GPT-2-small “greater-than” circuit in its original stack and independently, then test how far it generalizes across eight models (124M–12B) and to numerical entailment. The reported numbers replicate, and the late-MLP structure recurs at matched relative depth well above a size-matched null. Sparse faithfulness is high everywhere too, but only when the edge budget scales with the model’s own graph; since these graphs differ 96-fold in size, a fixed budget makes faithfulness appear to collapse with scale. On numerical entailment, no circuit-bearing model passes the 0.6 gate, while the one model that solves the task well cannot be posed the circuit’s own task.

❓ The Questioner

More from EMNLP 2026

All EMNLP 2026 papers →