ArabicMTDiagnostics at AlexandriaX-2026: Detecting and Classifying Errors in Dialectal Arabic Machine Translation Outputs
Abstract
AbstractDialectal Arabic MT error detection asks a sys tem to locate error spans in a machine trans lation and label each with a linguistic error category. It is hard because the dialects lack a standardised orthography and the error cate gories are severely imbalanced. In Subtask 3 of the AlexandriaX 2026 shared task, the in put is an English sentence and its machine translation into one of five Arabic dialects, and the output is character-level error spans with their LQM categories. The central difficulty is collapse onto the majority sociolinguistics class, 58% of training spans, leaving rare cat egories undetected. To address this, we fine tune NileChat-3B-Base with QLoRA on a sin gle 8 GB consumer GPU, casting the task as schema-prompted JSON generation and com bining character-position annotations that let the model copy offsets rather than count them, a category guide with one worked example per error type, and self-consistency decoding merging five samples by character-level union voting. Our system scores 0.3484 Overall on the official test set. We report two negative results: early stopping on validation loss de grades performance, since rare categories are acquired only in later epochs, and fine-tuning on the skewed distribution lowers macro-F1 be low the un-fine-tuned baseline until the cate gory guide is added