Skip to content
conftrace_
2026 EMNLP EMNLP 2026

When Symbolic Analyses Mislead: Resolution Sensitivity in Neural Language Representations

AbstractRecent representation analyses have reported structure-like or invariant behavior even in randomly initialized transformers, motivating the interpretation that aspects of linguistic structure may be architectural rather than learned. We show that such conclusions can be artifacts of coarse symbolic discretization. We formalize symbolic analysis of neural representations as operating over discretized activation trajectories and identify symbolic resolution as a critical, previously uncontrolled variable. Varying resolution reveals a sharp collapse of apparent similarity in random models, while pretrained models maintain stable, operator-selective differentiation across all tested resolutions. We introduce resolution stability as a necessary validity criterion for symbolic and automata-based analyses, providing a falsifiable, architecture-agnostic diagnostic for auditing internal linguistic structure in neural language models.

More from EMNLP 2026

All EMNLP 2026 papers →