conftrace_

attention head

146 papers

Explore in graph

Co-occurring keywords

attention mechanism (4001) large language model (13587) language model (4599) mechanistic interpretability (133) model compression (3302) transformer model (1996) in-context learning (1405) representation learning (6206) transformer architecture (1560) model pruning (237)

Papers

Self-Attention Attribution: Interpreting Information Interactions Inside Transformer AAAI 2021
Probing for Bridging Inference in Transformer Language Models NAACL 2021
Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling NIPS 2021
The Stem Cell Hypothesis: Dilemma behind Multi-Task Learning with Transformer Encoders EMNLP 2021
On Attention Redundancy: A Comprehensive Study NAACL 2021
Chasing Sparsity in Vision Transformers: An End-to-End Exploration NIPS 2021
Block Pruning For Faster Transformers EMNLP 2021
BERTologiCoMix: How does Code-Mixing interact with Multilingual BERT? EACL 2021
Enlivening Redundant Heads in Multi-head Self-attention for Machine Translation EMNLP 2021
What Does BERT with Vision Look At? ACL 2020
On the weak link between importance and prunability of attention heads EMNLP 2020
Heads-up! Unsupervised Constituency Parsing via Self-Attention Heads AACL 2020
On the Computational Power of Transformers and Its Implications in Sequence Modeling EMNLP 2020
Investigating Gender Bias in Language Models Using Causal Mediation Analysis NIPS 2020
BERTering RAMS: What and How Much does BERT Already Know About Event Arguments? - A Study on the RAMS Dataset EMNLP 2020
Sarah’s Participation in WAT 2019 EMNLP 2019
Mixed Multi-Head Self-Attention for Neural Machine Translation EMNLP 2019
Revealing the Dark Secrets of BERT IJCNLP 2019
Adaptively Sparse Transformers EMNLP 2019
Are Sixteen Heads Really Better than One? NIPS 2019
Multi-Head Attention with Disagreement Regularization EMNLP 2018