attention head pruning

BlockPruner: Fine-grained Pruning for Large Language Models ACL 2025 SlimLLM: Accurate Structured Pruning for Large Language Models ICML 2025 Structured Pruning for Diverse Best-of-N Reasoning Optimization ACL 2025 Cutting Off the Head Ends the Conflict: A Mechanism for Interpreting and Mitigating Knowledge Conflicts in Language Models ACL 2024 Uncovering Syllable Constituents in the Self-Attention-Based Speech Representations of Whisper EMNLP 2024 A Closer Look at Transformer Attention for Multilingual Translation EMNLP 2023 Structured Pruning for Efficient Generative Pre-trained Language Models ACL 2023 Probing via Prompting NAACL 2022 Token and Head Adaptive Transformers for Efficient Natural Language Processing COLING 2022 Pruning Neural Machine Translation for Speed Using Group Lasso EMNLP 2021 Self-Attention Attribution: Interpreting Information Interactions Inside Transformer AAAI 2021 On the weak link between importance and prunability of attention heads EMNLP 2020 Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned ACL 2019