Efficient Computing

LongHeads: Multi-Head Attention is Secretly a Long Context Processor EMNLP 2024 LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression ACL 2024 LORS: Low-rank Residual Structure for Parameter-Efficient Network Stacking CVPR 2024 Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training EMNLP 2024 Megalodon: Efficient LLM Pretraining and Inference with Unlimited Context Length NeurIPS 2024 Mesa-Extrapolation: A Weave Position Encoding Method for Enhanced Extrapolation in LLMs NeurIPS 2024 Mixture of In-Context Experts Enhance LLMs' Long Context Awareness NeurIPS 2024 MobileQuant: Mobile-friendly Quantization for On-device Language Models EMNLP 2024 Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs ICLR 2024 MoPE-CLIP: Structured Pruning for Efficient Vision-Language Models with Module-wise Pruning Error Metric CVPR 2024 MOSEL: Inference Serving Using Dynamic Modality Selection EMNLP 2024 Multisize Dataset Condensation ICLR 2024 NN4SysBench: Characterizing Neural Network Verification for Computer Systems NeurIPS 2024 Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers NeurIPS 2024 Not All Experts are Equal: Efficient Expert Pruning and Skipping for Mixture-of-Experts Large Language Models ACL 2024 On Context Utilization in Summarization with Large Language Models ACL 2024 On Hardware-efficient Inference in Probabilistic Circuits UAI 2024 Optimized Speculative Sampling for GPU Hardware Accelerators EMNLP 2024 Orchid: Flexible and Data-Dependent Convolution for Sequence Modeling NeurIPS 2024 Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descent NeurIPS 2024 PINNACLE: PINN Adaptive ColLocation and Experimental points selection ICLR 2024 Pipeline Parallelism with Controllable Memory NeurIPS 2024 Predicting Carpark Availability in Singapore with Cross-Domain Data: A New Dataset and A Data-Driven Approach IJCAI 2024 Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language Model EMNLP 2024 Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization EMNLP 2024