Skip to content
conftrace_
2026 EMNLP EMNLP 2026

M-QUEST - Meme Question-Understanding Evaluation on Semantics and Toxicity

AbstractInternet memes are a powerful form of online communication, yet their multimodal nature and reliance on commonsense and cultural knowledge make automated toxicity detection challenging. Prior work has studied individual dimensions of memes: textual content via optical character recognition, visual content via object recognition, and toxicity via hate speech detection, but lacks a unified architecture that formally identifies the semantic dimensions contributing to meme toxicity. We present three contributions: (1) a semantic framework identifying the dimensions necessary for meme interpretation; (2) a corresponding benchmark, M-QUEST, of 609 question-answer pairs for 307 memes, generated via a semi-automatic process grounded in the framework; and (3) an empirical evaluation of 12 VLMs on M-QUEST. The results show that the instruction-tuned models with reasoning capabilities significantly outperform others, whereas pragmatic inference remains challenging across all models.

More from EMNLP 2026

All EMNLP 2026 papers →