Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Deference with attributions in LLM Cascades

AbstractIn this paper, we address the problem of iteratively choosing to defer from a small Language Model (LM) to a larger one in order to balance accuracy and cost in a closed question-and-answer setting. Our deferral strategy is defined as follows: we identify the important sentences in the context, per the models, measure the differences in the ranking of their importance with the normalized Kendall Tau Distance and threshold that value to create a binary decision. We find that we can tune the desired cost or accuracy using this threshold value. Although it is outperformed on a dataset by a simple uncertainty average threshold, our method provides a more interpretable mechanism. Indeed, the attributions are found using the attributions of the generating LLM, which can be easily checked by a human. We find that our method offers a good interpolation of base model accuracies, especially in low cost regimes.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

More from EMNLP 2026

All EMNLP 2026 papers →