Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Soloeval at BAREC Shared Task 2026: QWK-Aligned Soft-Label Ordinal Classification with Auxiliary Surface Features for Arabic Readability Assessment

AbstractWe describe Soloeval’s submission to the Sentence-Level Open Track of the BAREC Shared Task 2026 on fine-grained Arabic readability assessment. Rather than treating the 19-level readability scale as a scalar regression target, we fine-tune pretrained Arabic BERT encoders (MARBERTv2 and CAMeLBERT-Mix) with a soft-label cross-entropy objective whose target distribution is a Gaussian centered on the gold level, chosen to more directly reflect the Quadratic Weighted Kappa (QWK) evaluation metric’s quadratic distance penalty. We combine this with lightweight auxiliary surface features (sentence length, word count, diacritic density) and K-fold cross-validation. Our best single-model configuration, CAMeLBERT-Mix trained with F=3 folds, achieves a QWK of 80.6 on the BAREC 2026 blind test leaderboard for the Open track. We report our full hyperparameter search, discuss the gap between our dev- and test-set results, and outline what we would try next.

More from EMNLP 2026

All EMNLP 2026 papers →