Efficient Computing

FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision NeurIPS 2024 FLORA: Fine-Grained Low-Rank Architecture Search for Vision Transformer WACV 2024 Forgetting, Ignorance or Myopia: Revisiting Key Challenges in Online Continual Learning NeurIPS 2024 Functional Interpolation for Relative Positions improves Long Context Transformers ICLR 2024 Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models NeurIPS 2024 GTP-ViT: Efficient Vision Transformers via Graph-Based Token Propagation WACV 2024 Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models AAAI 2024 HW-GPT-Bench: Hardware-Aware Architecture Benchmark for Language Models NeurIPS 2024 IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs ICLR 2024 In defense of parameter sharing for model-compression ICLR 2024 InfiniPot: Infinite Context Processing on Memory-Constrained LLMs EMNLP 2024 JumpCoder: Go Beyond Autoregressive Coder via Online Modification ACL 2024 KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization NeurIPS 2024 Ladder: Enabling Efficient Low-Precision Deep Learning Computing through Hardware-aware Tensor Transformation OSDI 2024 LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding ACL 2024 Learning Discretized Neural Networks under Ricci Flow JMLR 2024 LeMeViT: Efficient Vision Transformer with Learnable Meta Tokens for Remote Sensing Image Interpretation IJCAI 2024 LibAMM: Empirical Insights into Approximate Computing for Accelerating Matrix Multiplication NeurIPS 2024 Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning ACL 2024 LION: Implicit Vision Prompt Tuning AAAI 2024 LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment NeurIPS 2024 LLMem: Estimating GPU Memory Usage for Fine-Tuning Pre-Trained LLMs IJCAI 2024 LLM in a flash: Efficient Large Language Model Inference with Limited Memory ACL 2024 LocMoE: A Low-overhead MoE for Large Language Model Training IJCAI 2024 LoNAS: Elastic Low-Rank Adapters for Efficient Large Language Models COLING 2024