Skip to content
conftrace_

attention redundancy

3 papers

Explore in graph

Co-occurring keywords · papers mentioning both

attention head (2) transformer baseline (1) zero-shot pruning (1) model regularization (1) global attention (1) mixed attention (1) representation diversity (1) feature regularization (1) multi-head self-attention (1) model generalization (1)

Papers

The Principle of Diversity: Training Stronger Vision Transformers Calls for Reducing All Levels of Redundancy CVPR 2022 On Attention Redundancy: A Comprehensive Study NAACL 2021 Mixed Multi-Head Self-Attention for Neural Machine Translation EMNLP 2019