conftrace_

self-attention anisotropy

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

angular distance (1) representation degeneration (1) hidden representation (1) cross-entropy loss (1) transformer architecture (1)

Papers

Anisotropy Is Inherent to Self-Attention in Transformers EACL 2024