Exemplar Choice Matters for Morphological Reasoning: Model Confidence Vs Exemplar Similarity
Abstract
AbstractLLMs have shown impressive linguistic competencies in a wide range of NLP tasks, yet morphological generalization remains challenging. Models can benefit from in-context learning, where the choice of few-shot examples significantly influences the outcome. We probe LLMs’ morphological capabilities by assessing how two targeted in-context exemplar selection strategies, Iterative Confidence-Based Exemplar Selection (i-CES) and Similarity-Based Exemplar Selection (SES), influence the model’s ability for inflectional and derivational modeling across multiple languages. Our experiments with LLaMA 3.3-70B-instruct and Qwen2.5-32B-Instruct models indicate that SES is more effective for accessing latent linguistic knowledge, showing a substantial performance advantage over i-CES in both derivational and inflectional tasks. In contrast, i-CES offers no clear advantage over random sampling.