conftrace_

attention head

146 papers

Explore in graph

Co-occurring keywords

attention mechanism (4001) large language model (13587) language model (4599) mechanistic interpretability (133) model compression (3302) transformer model (1996) in-context learning (1405) representation learning (6206) transformer architecture (1560) model pruning (237)

Papers

Steering Prepositional Phrases in Language Models: A Case of with-headed Adjectival and Adverbial Complements in Gemma-2 EMNLP 2025
Evil twins are not that evil: Qualitative insights into machine-generated prompts EMNLP 2025
Beyond Position: the emergence of wavelet-like properties in Transformers ACL 2025
Inferring Functionality of Attention Heads from their Parameters ACL 2025
SEAL: Scaling to Emphasize Attention for Long-Context Retrieval ACL 2025
Learning Task Representations from In-Context Learning ACL 2025
Hierarchical Safety Realignment: Lightweight Restoration of Safety in Pruned Large Vision-Language Models ACL 2025
Time Course MechInterp: Analyzing the Evolution of Components and Knowledge in Large Language Models ACL 2025
Word Form Matters: LLMs’ Semantic Reconstruction under Typoglycemia ACL 2025
How do autoregressive transformers solve full addition? EMNLP 2025
Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Information ACL 2025
Dissecting Persona-Driven Reasoning in Language Models via Activation Patching EMNLP 2025
Align Attention Heads Before Merging Them: An Effective Way for Converting MHA to GQA EMNLP 2025
Distributional Surgery for Language Model Activations EMNLP 2025
Toward Inclusive Language Models: Sparsity-Driven Calibration for Systematic and Interpretable Mitigation of Social Biases in LLMs EMNLP 2025
Understanding and Controlling Repetition Neurons and Induction Heads in In-Context Learning IJCNLP 2025
Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers NIPS 2024
SnapKV: LLM Knows What You are Looking for Before Generation NIPS 2024
Cluster-Learngene: Inheriting Adaptive Clusters for Vision Transformers NIPS 2024
Talking Heads: Understanding Inter-Layer Communication in Transformer Language Models NIPS 2024
Abrupt Learning in Transformers: A Case Study on Matrix Completion NIPS 2024
How does GPT-2 Predict Acronyms? Extracting and Understanding a Circuit via Mechanistic Interpretability AISTATS 2024
Better Explain Transformers by Illuminating Important Information EACL 2024
Non-Linear Inference Time Intervention: Improving LLM Truthfulness INTERSPEECH 2024
Exploring Semantics in Pretrained Language Model Attention NAACL 2024