Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Same Intent, Different Script: Cross-Script Safety Degradation in LLMs Across Punjabi Sociolinguistic Variants

AbstractMultilingual safety alignment typically assumes standard, canonical scripts, overlooking orthographic variation. We introduce PunjabiSafetyBench, a parallel safety benchmark evaluating 300 harmful intents and 500 benign controls across six sociolinguistic script variants (Gurmukhi, Shahmukhi, Romanized, code-mixed, Urdu, English). Evaluating six frontier LLMs, we reveal severe safety degradation on non-canonical scripts: transitioning queries to Romanized Punjabi or Shahmukhi drives significant guardrail bypasses in open-weight models, while proprietary models remain robust. Furthermore, we expose a localization gap where models overlook regional harmful slang despite successfully refusing standard clinical equivalents. Our findings demonstrate that modern guardrails are fragile to surface orthography, necessitating safety alignment on informal, non-standard scripts.

More from EMNLP 2026

All EMNLP 2026 papers →