Improving Efficiency of Ternary Large Language Models through Pruning
Abstract
AbstractAs the environmental impact of Large Language Models (LLMs) grows, improving model efficiency has become critical for sustainable NLP. Recent advances in model compression have shown that ternary quantisation (i.e., using only {-1, 0, 1} as weights) substantially reduces memory footprint, inference time, and energy usage of LLMs with minimal performance degradation. However, whether these already efficient models can be further pruned to reduce resource requirements without compromising effectiveness remains unexplored. We investigate this question across six models, four sparsity levels, and six structured and unstructured pruning techniques. We find that even minimal pruning yields substantial efficiency gains: pruning fewer than 5% of parameters reduces inference time and energy consumption by up to 21%, and memory usage by up to 13%, with minimal accuracy degradation. We also find that, for pruned models with less than a 5% accuracy drop, the most energy and inference-time efficient configurations are achieved by pruning entire attention layers, whereas the most memory efficient configurations are obtained by pruning entire MLP layers. These findings highlight pruning as a complementary technique to ternary quantisation for reducing environmental footprint of LLMs while maintaining model effectiveness.