Skip to content
conftrace_
2026 EMNLP EMNLP 2026

Improving Efficiency of Ternary Large Language Models through Pruning

AbstractAs the environmental impact of Large Language Models (LLMs) grows, improving model efficiency has become critical for sustainable NLP. Recent advances in model compression have shown that ternary quantisation (i.e., using only {-1, 0, 1} as weights) substantially reduces memory footprint, inference time, and energy usage of LLMs with minimal performance degradation. However, whether these already efficient models can be further pruned to reduce resource requirements without compromising effectiveness remains unexplored. We investigate this question across six models, four sparsity levels, and six structured and unstructured pruning techniques. We find that even minimal pruning yields substantial efficiency gains: pruning fewer than 5% of parameters reduces inference time and energy consumption by up to 21%, and memory usage by up to 13%, with minimal accuracy degradation. We also find that, for pruned models with less than a 5% accuracy drop, the most energy and inference-time efficient configurations are achieved by pruning entire attention layers, whereas the most memory efficient configurations are obtained by pruning entire MLP layers. These findings highlight pruning as a complementary technique to ternary quantisation for reducing environmental footprint of LLMs while maintaining model effectiveness.

More from EMNLP 2026

All EMNLP 2026 papers →