byte pair encoding

On the Effectiveness of Acoustic BPE in Decoder-Only TTS Interspeech 2024 PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control ICML 2024 Tokenization Falling Short: On Subword Robustness in Large Language Models EMNLP 2024 Tokenization Is More Than Compression EMNLP 2024 Tokenization Matters: Navigating Data-Scarce Tokenization for Gender Inclusive Language Technologies NAACL 2024 Two Counterexamples to Tokenization and the Noiseless Channel COLING 2024 Unpacking Tokenization: Evaluating Text Compression and its Correlation with Model Performance ACL 2024 A Formal Perspective on Byte-Pair Encoding ACL 2023 Byte Pair Encoding for Symbolic Music EMNLP 2023 CodeBPE: Investigating Subtokenization Options for Large Language Model Pretraining on Source Code ICLR 2023 Effects of sub-word segmentation on performance of transformer language models EMNLP 2023 Optimizing the Size of Subword Vocabularies in Dialect Classification EACL 2023 Phonetic-assisted Multi-Target Units Modeling for Improving Conformer-Transducer ASR system Interspeech 2023 Subword Segmental Machine Translation: Unifying Segmentation and Target Sentence Generation ACL 2023 Tokenization with Factorized Subword Encoding ACL 2023 What changes when you randomly choose BPE merge operations? Not much. EACL 2023 BPE vs. Morphological Segmentation: A Case Study on Machine Translation of Four Polysynthetic Languages ACL 2022 Impact of Sequence Length and Copying on Clause-Level Inflection EMNLP 2022 Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages ACL 2022 Pixel-Level BPE for Auto-Regressive Image Generation COLING 2022 Segmenting Numerical Substitution Ciphers EMNLP 2022 Subword Segmental Language Modelling for Nguni Languages EMNLP 2022 Acoustic Data-Driven Subword Modeling for End-to-End Speech Recognition Interspeech 2021 AlloST: Low-Resource Speech Translation Without Source Transcription Interspeech 2021 A Statistical Extension of Byte-Pair Encoding ACL 2021