Skip to content
conftrace_
2026 EMNLP EMNLP 2026

LoopViT: Scaling Visual ARC with Looped Transformers

AbstractRecent ARC-AGI systems have shown that pure vision can solve abstract grid reasoning without language backbones, but most strong models still rely on fixed-depth feed-forward computation. We study whether recurrent computation is a better scaling axis for this setting. We propose Loop-ViT, a vision-native recurrent model that reuses a weight-tied Hybrid Block across multiple steps, with local convolutions for image-token refinement and attention for global rule induction. We also use an entropy-based exit rule to reduce unnecessary iterations at inference time. On ARC-AGI-1, Loop-ViT improves the parameter-efficiency trade-off over fixed-depth vision baselines and reaches 65.8% accuracy with an 18M model. Overall, the results suggest that repeated computation is a useful inductive bias for ARC-style visual reasoning. The code is available at https://github.com/WenjieShu/LoopViT.

More from EMNLP 2026

All EMNLP 2026 papers →