conftrace_

softmax

10 papers

Explore in graph

Co-occurring keywords · papers mentioning both

temperature (3) attention (2) transformer (2) undirected topic model (1) document modeling (1) topic vectors (1) latent topic analysis (1) distributionally robust optimization (1) generative model (1) generalization bound (1)

Papers

Counting in Small Transformers: The Delicate Interplay between Attention and Feed-Forward Layers ICML 2025 In-Context Deep Learning via Transformer Models ICML 2025 LASER: Attention with Exponential Transformation ICML 2025 Softmax is not Enough (for Sharp Size Generalisation) ICML 2025 Eureka-Moments in Transformers: Multi-Step Tasks Reveal Softmax Induced Optimization Problems ICML 2024 Is Temperature Sample Efficient for Softmax Gaussian Mixture of Experts? ICML 2024 MultiMax: Sparse and Multi-Modal Attention Learning ICML 2024 To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO ICML 2024 Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features ICML 2024 Replicated Softmax: an Undirected Topic Model NeurIPS 2009