conftrace_

attention head

146 papers

Explore in graph

Co-occurring keywords

attention mechanism (4001) large language model (13587) language model (4599) mechanistic interpretability (133) model compression (3302) transformer model (1996) in-context learning (1405) representation learning (6206) transformer architecture (1560) model pruning (237)

Papers

Deciphering Stereotypes in Pre-Trained Language Models EMNLP 2023
Plausibility Processing in Transformer Language Models: Focusing on the Role of Attention Heads in GPT EMNLP 2023
Memory Injections: Correcting Multi-Hop Reasoning Failures During Inference in Transformer-Based Language Models EMNLP 2023
Humans and language models diverge when predicting repeating text EMNLP 2023
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model NIPS 2023
Rethinking the Role of Scale for In-Context Learning: An Interpretability-based Case Study at 66 Billion Scale ACL 2023
Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time ICML 2023
HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation ACL 2023
Causal interventions expose implicit situation models for commonsense language understanding ACL 2023
Self-Distillation into Self-Attention Heads for Improving Transformer-based End-to-End Neural Speaker Diarization INTERSPEECH 2023
Humans and language models diverge when predicting repeating text CONLL 2023
Gradient-based Intra-attention Pruning on Pre-trained Language Models ACL 2023
Characterizing Mechanisms for Factual Recall in Language Models EMNLP 2023
Breaking Down Multilingual Machine Translation ACL 2022
Probing via Prompting NAACL 2022
XAI for Transformers: Better Explanations through Conservative Propagation ICML 2022
Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models NAACL 2022
Adapting a Language Model While Preserving its General Knowledge EMNLP 2022
What changed? Investigating Debiasing Methods using Causal Mediation Analysis NAACL 2022
Gender Biases and Where to Find Them: Exploring Gender Bias in Pre-Trained Transformer-based Language Models Using Movement Pruning NAACL 2022
Alleviating the Inequality of Attention Heads for Neural Machine Translation COLING 2022
Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks ACL 2021
Chop Chop BERT: Visual Question Answering by Chopping VisualBERT’s Heads IJCAI 2021
Contributions of Transformer Attention Heads in Multi- and Cross-lingual Tasks IJCNLP 2021
The Heads Hypothesis: A Unifying Statistical Approach Towards Understanding Multi-Headed Attention in BERT AAAI 2021