conftrace_

transformer optimization

24 papers

Explore in graph

Co-occurring keywords

model compression (3302) attention mechanism (4001) memory efficiency (261) dynamic computation (16) key-value cache (55) efficient computing (781) low-precision computation (2) small dataset (13) gpu optimization (27) model initialization (32)

Papers

Conv-Basis: A New Paradigm for Efficient Attention Inference and Gradient Computation in Transformers EMNLP 2025
TreeKV: Smooth Key-Value Cache Compression with Tree Structures IJCAI 2025
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization EMNLP 2025
AsymKV: Enabling 1-Bit Quantization of KV Cache with Layer-Wise Asymmetric Quantization Configurations COLING 2025
p-MoD: Building Mixture-of-Depths MLLMs via Progressive Ratio Decay ICCV 2025
LAWCAT: Efficient Distillation from Quadratic to Linear Attention with Convolution across Tokens for Long Context Modeling EMNLP 2025
The Warmup Dilemma: How Learning Rate Strategies Impact Speech-to-Text Model Convergence ACL 2025
FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision NIPS 2024
CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling EMNLP 2024
KV Cache Compression, But What Must We Give in Return? A Comprehensive Benchmark of Long Context Capable Approaches EMNLP 2024
MemoryFormer : Minimize Transformer Computation by Removing Fully-Connected Layers NIPS 2024
Sequence Parallelism: Long Sequence Training from System Perspective ACL 2023
Context Compression for Auto-regressive Transformers with Sentinel Tokens EMNLP 2023
Leap-of-Thought: Accelerating Transformers via Dynamic Token Routing EMNLP 2023
You Need Multiple Exiting: Dynamic Early Exiting for Accelerating Unified Vision Language Model CVPR 2023
Pit One Against Many: Leveraging Attention-head Embeddings for Parameter-efficient Multi-head Attention EMNLP 2023
Speeding up Transformer Decoding via an Attention Refinement Network COLING 2022
Optimizing Deeper Transformers on Small Datasets ACL 2021
Faster Depth-Adaptive Transformers AAAI 2021
Optimizing Deeper Transformers on Small Datasets IJCNLP 2021
Value-aware Approximate Attention EMNLP 2021
Improving Transformer Optimization Through Better Initialization ICML 2020
Making Asynchronous Stochastic Gradient Descent Work for Transformers EMNLP 2019
Marian: Cost-effective High-Quality Neural Machine Translation in C++ ACL 2018