conftrace_

text deduplication

3 papers

Explore in graph

Co-occurring keywords · papers mentioning both

near-duplicate detection (1) digitized book (1) text correction (1) cross-encoder model (1) optical character recognition (1) bi-encoder model (1) multilingual dataset (1) neural network embedding (1) data cleaning (1) language model (1)

Papers

CulturaX: A Cleaned, Enormous, and Multilingual Dataset for Large Language Models in 167 Languages COLING 2024 Noise-Robust De-Duplication at Scale ICLR 2023 Cleaning Dirty Books: Post-OCR Processing for Previously Scanned Texts EMNLP 2021