attention head

LoFiT: Localized Fine-tuning on LLM Representations NeurIPS 2024 Mechanistic Understanding and Mitigation of Language Model Non-Factual Hallucinations EMNLP 2024 Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs ICLR 2024 Multimodal Task Vectors Enable Many-Shot Multimodal In-Context Learning NeurIPS 2024 Non-Linear Inference Time Intervention: Improving LLM Truthfulness Interspeech 2024 On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task EMNLP 2024 Prompting a Pretrained Transformer Can Be a Universal Approximator ICML 2024 SnapKV: LLM Knows What You are Looking for Before Generation NeurIPS 2024 Successor Heads: Recurring, Interpretable Attention Heads In The Wild ICLR 2024 Talking Heads: Understanding Inter-Layer Communication in Transformer Language Models NeurIPS 2024 Tell Your Model Where to Attend: Post-hoc Attention Steering for LLMs ICLR 2024 The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Models ACL 2024 When Parts Are Greater Than Sums: Individual LLM Components Can Outperform Full Models EMNLP 2024 Whispers that Shake Foundations: Analyzing and Mitigating False Premise Hallucinations in Large Language Models EMNLP 2024 Causal interventions expose implicit situation models for commonsense language understanding ACL 2023 Characterizing Mechanisms for Factual Recall in Language Models EMNLP 2023 Deciphering Stereotypes in Pre-Trained Language Models EMNLP 2023 Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time ICML 2023 Does Deep Learning Learn to Abstract? A Systematic Probing Framework ICLR 2023 GOHSP: A Unified Framework of Graph and Optimization-Based Heterogeneous Structured Pruning for Vision Transformer AAAI 2023 Gradient-based Intra-attention Pruning on Pre-trained Language Models ACL 2023 HiFi: High-Information Attention Heads Hold for Parameter-Efficient Model Adaptation ACL 2023 Humans and language models diverge when predicting repeating text CoNLL 2023 Inference-Time Intervention: Eliciting Truthful Answers from a Language Model NeurIPS 2023 Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 Small ICLR 2023