Skip to content
conftrace_
2026 EMNLP EMNLP 2026

RIFT: Relation Modeling and Transfer for Visual Figurative Entailment

AbstractVisual figurative entailment requires determining whether an image supports the nonliteral meaning of a textual claim, posing challenges for multimodal figurative understanding. Existing methods typically use fixed image-text fusion and struggle to capture sample-specific relations between visual evidence and figurative claims. Moreover, the limited scale of annotated visual figurative data makes it difficult to learn robust figurative relations from visual supervision alone. To address these challenges, we propose RIFT, which generates a claim-conditioned visual description and uses a relation-gated multimodal mixture of experts to model interactions among the image, description, and claim. It further retrieves figurative reasoning patterns from a FLUTE-based relation-level rationale memory and employs a confidence routing gate to modulate this external guidance according to retrieval reliability. The resulting internal and external relation representations are integrated as latent prefix tokens for joint entailment prediction and explanation generation. On V-FLUTE, RIFT achieves 88.52 F1@0, 76.07 F1@53, and 59.47 F1@60, outperforming MAPPER by 3.98, 6.38, and 7.75 points, respectively. These improvements, particularly on explanation-constrained metrics, show that relation modeling and cross-modal transfer benefit both prediction and explanation quality.

More from EMNLP 2026

All EMNLP 2026 papers →