attention head

Memory Injections: Correcting Multi-Hop Reasoning Failures During Inference in Transformer-Based Language Models EMNLP 2023 Plausibility Processing in Transformer Language Models: Focusing on the Role of Attention Heads in GPT EMNLP 2023 Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale ACL 2023 Self-Distillation into Self-Attention Heads for Improving Transformer-based End-to-End Neural Speaker Diarization Interspeech 2023 Towards Lossless Head Pruning through Automatic Peer Distillation for Language Models IJCAI 2023 VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers EMNLP 2023 Adapting a Language Model While Preserving its General Knowledge EMNLP 2022 Alleviating the Inequality of Attention Heads for Neural Machine Translation COLING 2022 Breaking Down Multilingual Machine Translation ACL 2022 Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruning NAACL 2022 Probing via Prompting NAACL 2022 Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models NAACL 2022 What changed? Investigating Debiasing Methods using Causal Mediation Analysis NAACL 2022 XAI for Transformers: Better Explanations through Conservative Propagation ICML 2022 BERTologiCoMix: How does Code-Mixing interact with Multilingual BERT? EACL 2021 Block Pruning For Faster Transformers EMNLP 2021 Chasing Sparsity in Vision Transformers: An End-to-End Exploration NeurIPS 2021 Chop Chop BERT: Visual Question Answering by Chopping VisualBERT’s Heads IJCAI 2021 Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks ACL 2021 Enlivening Redundant Heads in Multi-head Self-attention for Machine Translation EMNLP 2021 On Attention Redundancy: A Comprehensive Study NAACL 2021 Pay Better Attention to Attention: Head Selection in Multilingual and Multi-Domain Sequence Modeling NeurIPS 2021 Probing for Bridging Inference in Transformer Language Models NAACL 2021 Self-Attention Attribution: Interpreting Information Interactions Inside Transformer AAAI 2021 The Heads Hypothesis: A Unifying Statistical Approach Towards Understanding Multi-Headed Attention in BERT AAAI 2021