Skip to content
conftrace_

head pruning

5 papers

Explore in graph

Co-occurring keywords · papers mentioning both

attention head (2) model compression (2) attention normalization (1) attention compression (1) multi-headed attention (1) transformer compression (1) peer distillation (1) parameter redundancy (1) group constraint (1) self-supervised grouping (1)

Papers

Defense against Backdoor Attack on Pre-trained Language Models via Head Pruning and Attention Normalization ICML 2024 Finding the Pillars of Strength for Multi-Head Attention ACL 2023 Towards Lossless Head Pruning through Automatic Peer Distillation for Language Models IJCAI 2023 Token and Head Adaptive Transformers for Efficient Natural Language Processing COLING 2022 Are Sixteen Heads Really Better than One? NeurIPS 2019