conftrace_

multi-query attention

3 papers

Explore in graph

Also known as

MQA

Co-occurring keywords

grouped-query attention (6) memory efficiency (261) cross-layer attention (2) autoregressive decoding (33) inference speedup (37) memory reduction (13) key-value caching (1) transformer decoding (3) large language model (13587) transformer model (1996)

Papers

MLKV: Multi-Layer Key-Value Heads for Memory Efficient Transformer Decoding NAACL 2025
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention NIPS 2024
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints EMNLP 2023