Who Wrote the Book? Detecting and Attributing LLM Ghostwriters
Abstract
AbstractOut-of-distribution (OOD) generalisation is essential for authorship attribution (AA), especially when involving new authors, yet this aspect remains insufficiently supported by existing AA datasets. In this paper, we introduce GHOSTWRITEBENCH, a dataset for LLM AA task, designed to test generalisation across multiple OOD dimensions, including unseen domain and authors. GHOSTWRITEBENCH comprises long-form texts (50K+ words per book) generated by ten frontier LLMs. We find that existing methods degrade substantially in unseen author scenarios — dropping by as much as 70% even under high-resource settings. We propose TRACE, a novel lightweight and interpretable fingerprinting method that captures token-level transition patterns (i.e., word rank and entropy) using a lightweight language model. Experiments show robustness of TRACE in OOD settings and establish it as a strong baseline on GHOSTWRITEBENCH.