Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Fine-Tuning for Social-Media User Simulation: A Study Across Models and Evaluators

AbstractDoes fine-tuning on a user’s social-media history make an LLM a better simulator of that user? We study this question using QLoRA, conditioning models on a user’s chronological action history and a reply thread before generating that user’s reply. Across 12 matched baseline–QLoRA pairs from four model families (0.8B–32B), plus a baseline-only 122B model, LLM-as-judge believability evaluations show that QLoRA improves several smaller models, has mixed effects on stronger models, and does not lift a sub-10B model into the strongest large-model tier. However, a real-versus-generated classifier gives a different result: QLoRA makes Qwen, OLMo, and Llama replies much less detectable, while Gemma remains readily detectable. Contextual believability and detector AUC are uncorrelated across systems. We further explore LLM evaluations: strong LLM judges agree on broad quality tiers, and a blind 200-comparison audit finds human–judge agreement of 66% when three strong judges agree and 47% when they split. The measured effect of QLoRA therefore depends on the evaluation axis.

More from EMNLP 2026

All EMNLP 2026 papers →