Whose Toxicity? Persona Conditioning and Model–Annotator Label Agreement by Gender in Indonesian Toxicity Classification
Abstract
AbstractToxicity judgments can depend on who is judging. We study whether persona prompting captures this variation by evaluating ten Indonesian-capable LLMs under four persona conditions on 7,524 IndoDiscourse posts, each labeled by at least one female and one male annotator, comprising 24,655 individual judgments. Persona cues substantially affect toxicity predictions, but their effects are model-specific, show no common direction, and are small between the two gender personas. Yet this steerability does not amount to perspective adoption: switching between gender personas does not selectively increase label agreement with the corresponding annotator group. Annotation-level mixed-effects models and annotator-level permutation tests likewise find no reliable female–male agreement gap. The apparent gap obtained from aggregated group labels instead follows from annotator label rates and aggregation choices.