feed-forward layer

On the Role of Transformer Feed-Forward Layers in Nonlinear In-Context Learning ALT 2026 Transformers Can Overcome the Curse of Dimensionality: A Theoretical Study from an Approximation Perspective JMLR 2026 Counting in Small Transformers: The Delicate Interplay between Attention and Feed-Forward Layers ICML 2025 Distributional Associations vs In-Context Reasoning: A Study of Feed-forward and Attention Layers ICLR 2025 LLM Braces: Straightening Out LLM Predictions with Relevant Sub-Updates ACL 2025 The Role of Outgoing Connection Heterogeneity in Feedforward Layers of Large Language Models EMNLP 2025 An Investigation of Neuron Activation as a Unified Lens to Explain Chain-of-Thought Eliciting Arithmetic Reasoning of LLMs ACL 2024 Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable Tasks ICML 2024 Neuron Specialization: Leveraging Intrinsic Task Modularity for Multilingual Machine Translation EMNLP 2024 Scalable Neural Network Kernels ICLR 2024 Understanding the Expressive Power and Mechanisms of Transformer for Sequence Modeling NeurIPS 2024 LM-Debugger: An Interactive Tool for Inspection and Intervention in Transformer-Based Language Models EMNLP 2022 Hash Layers For Large Sparse Models NeurIPS 2021 Transformer Feed-Forward Layers Are Key-Value Memories EMNLP 2021 Improving Transformer-Based Speech Recognition Systems with Compressed Structure and Speech Attributes Augmentation Interspeech 2019 Breaking the Activation Function Bottleneck through Adaptive Parameterization NeurIPS 2018