Skip to content
conftrace_
2026 EMNLP EMNLP 2026

SALM: Social Arabic Language Model

AbstractArabic NLP faces a fundamental mismatch between training and deployment conditions. While most language models are trained predominantly on Modern Standard Arabic (MSA), real-world communication occurs largely in spoken dialects. To bridge this gap, we leverage social media content from Reddit, Twitter, and YouTube as a scalable proxy for spoken Arabic. We construct a 1.04-billion-token pretraining corpus and train SALM, a compact 336M-parameter decoder-only language model pre-trained via next-token prediction alone, no masked language modelling, no dialect labels, no task-specific signal. Despite an order-of-magnitude smaller training corpus than existing dialectal PLMs and a generative training objective that provides no direct classification signal, SALM matches MARBERT, explicitly pre-trained on one billion dialectal tweets, within 0.7 percentage points on MADAR-26 dialect identification accuracy and outperforms it on macro-F1. Under full fine-tuning, SALM surpasses all baselines, achieving 69.1% accuracy on 26-way city-level dialect classification. Under the same frozen probe on binary Arabic sentiment (AJGT), SALM achieves 92.2% accuracy, topping all Arabic PLMs, encoder and decoder alike. We release the corpus and trained model to facilitate future research on Arabic spoken language modelling.

More from EMNLP 2026

All EMNLP 2026 papers →