conftrace_

multi-head attention

172 papers

Explore in graph

Also known as

MHA

Co-occurring keywords

attention mechanism (4001) transformer architecture (1560) neural network (6616) neural machine translation (2310) text classification (6793) machine translation (2499) representation learning (6206) transformer model (1996) neural network optimization (1293) mixture of expert (640)

Papers

More Than Catastrophic Forgetting: Integrating General Capabilities For Domain-Specific LLMs EMNLP 2024
LingWav2Vec2: Linguistic-augmented wav2vec 2.0 for Vietnamese Mispronunciation Detection INTERSPEECH 2024
LongHeads: Multi-Head Attention is Secretly a Long Context Processor EMNLP 2024
How Transformers Utilize Multi-Head Attention in In-Context Learning? A Case Study on Sparse Linear Regression NIPS 2024
A Theoretical Understanding of Self-Correction through In-context Alignment NIPS 2024
Training Dynamics of Multi-Head Softmax Attention for In-Context Learning: Emergence, Convergence, and Optimality (extended abstract) COLT 2024
Pit One Against Many: Leveraging Attention-head Embeddings for Parameter-efficient Multi-head Attention EMNLP 2023
Mao-Zedong at SemEval-2023 Task 4: Label Represention Multi-Head Attention Model with Contrastive Learning-Enhanced Nearest Neighbor Mechanism for Multi-Label Text Classification SEMEVAL 2023
SEF-Net: Speaker Embedding Free Target Speaker Extraction Network INTERSPEECH 2023
RADIANT: Better rPPG Estimation Using Signal Embeddings and Transformer WACV 2023
GRAVO: Learning to Generate Relevant Audio from Visual Features with Noisy Online Videos INTERSPEECH 2023
Singularformer: Learning to Decompose Self-Attention to Linearize the Complexity of Transformer IJCAI 2023
Finding the Pillars of Strength for Multi-Head Attention ACL 2023
On the Expressivity Role of LayerNorm in Transformers’ Attention ACL 2023
Multimodal Locally Enhanced Transformer for Continuous Sign Language Recognition INTERSPEECH 2023
Multi-Head State Space Model for Speech Recognition INTERSPEECH 2023
End-to-end clinical temporal information extraction with multi-head attention ACL 2023
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling ACL 2023
Exploring Self-Distillation Based Relational Reasoning Training for Document-Level Relation Extraction AAAI 2023
What can a Single Attention Layer Learn? A Study Through the Random Features Lens NIPS 2023
Attention Gate Between Capsules in Fully Capsule-Network Speech Recognition INTERSPEECH 2023
Transforming Visual Scene Graphs to Image Captions ACL 2023
3DRP-Net: 3D Relative Position-aware Network for 3D Visual Grounding EMNLP 2023
Part Aware Contrastive Learning for Self-Supervised Action Recognition IJCAI 2023
Nested Deformable Multi-Head Attention for Facial Image Inpainting WACV 2023