Skip to content
conftrace_
2026 EMNLP EMNLP 2026

HPLT3-PerRef: A Refined Large-Scale Persian Web Corpus via Supervised Document Quality Filtering

AbstractLarge-scale web corpora are valuable resources for natural language processing (NLP), linguistic research, and language model training, particularly for digitally low-resource languages such as Persian. However, web-derived text often contains document-level noise, even after quality-filtering pipelines. We present HPLT3 Persian Refined (HPLT3-PerRef), a refined Persian web corpus containing 105.9 billion tokens and 102.37 million documents, obtained by applying a supervised quality-filtering pipeline to the Persian subset of HPLT3. We manually annotated 6,144 Persian web documents and trained a multilingual transformer-based classifier to distinguish high- and low-quality documents. The classifier removed approximately 20% of the original corpus while preserving broad domain coverage and complements HPLT3’s heuristic-based Web Document Score (WDS) by identifying additional low-quality documents. We publicly release the corpus, the trained classifier, and the complete source code.

More from EMNLP 2026

All EMNLP 2026 papers →