CHRONOEXPERT: A Temporal Mixture-of-Experts for Knowledge Editing in Large Language Models
Abstract
AbstractKnowledge editing for large language models typically treats a fact update as a replacement in which a new object overwrites the old. Many real-world facts, however, evolve through sequences: the same subject–relation pair maps to different objects across disjoint intervals, each correct within its validity window. Existing temporal-editing benchmarks probe latest-value or explicit-time recall, so they cannot determine whether an edited model retains a coherent timeline. We introduce TKE-Bench, a diagnostic benchmark that operationalizes temporal accessibility along three reasoning-oriented axes with paraphrased variants. On TKE-Bench, all evaluated editing baselines collapse on chronologically structured queries, revealing catastrophic temporal interference (CTI). We trace CTI to a geometric property implicit in locate-and-edit methods: mid-layer subject-token keys are nearly collinear across time slices of the same subject, so stacked updates write along one direction and the latest edit overwrites every earlier value. Building on this, we propose ChronoExpert, a temporal mixture-of-experts that stores each time slice in an independent lightweight expert and activates only the matching one at inference, leaving the base model otherwise untouched. Across multiple LLM backbones, ChronoExpert consistently outperforms the strongest editing baselines and stays robust as chains grow longer.