Does Sensitivity to Pragmatic Norms Emerge with More Parameters, More Data, or More Conversations?
Abstract
AbstractWhile previous work has examined the ability of language models (LMs) to capture pragmatic patterns, it remains understudied to what extent these abilities are shaped by training conditions. To address this gap, we test several models using an existing evaluation paradigm targeting LMs’ sensitivity to conversational norms (Askari et al. 2025). Across three experimental settings, we investigate the role of model size, training dynamics, and data composition in shaping pragmatic competence. Our results show that different pragmatic abilities exhibit distinct learning dynamics across models with varying sizes. While size or data scaling lead to inconsistent gains, exposure to conversational data causes moderate improvements in a modest training setting. Together, our findings suggest that pragmatic competence in LMs is heterogeneous and highlight the importance of targeted training conditions over scale alone for developing pragmatically sensitive models.