Skip to content
conftrace_
2026 EMNLP EMNLP 2026

STEB: Style Text Embedding Benchmark

AbstractWhile semantic embeddings are rigorously evaluated on the Massive Text Embedding Benchmark, the evaluation of style embeddings remains fragmented, with each work relying on their own set of tasks and datasets. To bridge this gap, we introduce the Style Text Embedding Benchmark, a comprehensive open-source benchmark intended to standardize the evaluation of style embeddings. STEB encompasses 96 datasets across 7 languages, spanning applications such as authorship verification, authorship retrieval, AI-text detection, probing of llinguistic features, and others. We find that semantic embeddings consistently fail in stylistic tasks, and that there is no style embedding that is universally superior across all tasks evaluated. STEB comes with an open-source code base and leaderboard at https://github.com/rrivera1849/STEB.

More from EMNLP 2026

All EMNLP 2026 papers →