model quantization

279 papers

Explore in graph

Also known as

PTQ INT8 QDNN

Co-occurring keywords

model compression (3283) large language model (12755) knowledge distillation (3680) weight quantization (133) post-training quantization (124) neural network optimization (1293) efficient computing (779) neural network (6616) activation quantization (47) efficient inference (225)

Papers

QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models EMNLP 2024

Instance-Aware Group Quantization for Vision Transformers CVPR 2024

Sign Value Constraint Decomposition for Efficient 1-Bit Quantization of Speech Translation Tasks INTERSPEECH 2024

QBB: Quantization with Binary Bases for LLMs NIPS 2024

Prefixing Attention Sinks can Mitigate Activation Outliers for Large Language Model Quantization EMNLP 2024

QTIP: Quantization with Trellises and Incoherence Processing NIPS 2024

RA-LoRA: Rank-Adaptive Parameter-Efficient Fine-Tuning for Accurate 2-bit Quantized Large Language Models ACL 2024

Understanding and Minimising Outlier Features in Transformer Training NIPS 2024

QDyLoRA: Quantized Dynamic Low-Rank Adaptation for Efficient Large Language Model Tuning EMNLP 2024

LLM-QAT: Data-Free Quantization Aware Training for Large Language Models ACL 2024

MobileQuant: Mobile-friendly Quantization for On-device Language Models EMNLP 2024

ApiQ: Finetuning of 2-Bit Quantized Large Language Model EMNLP 2024

PTMQ: Post-training Multi-Bit Quantization of Neural Networks AAAI 2024

DB-LLM: Accurate Dual-Binarization for Efficient LLMs ACL 2024

xCOMET-lite: Bridging the Gap Between Efficiency and Quality in Learned MT Evaluation Metrics EMNLP 2024

IntactKV: Improving Large Language Model Quantization by Keeping Pivot Tokens Intact ACL 2024

The Inhibitor: ReLU and Addition-Based Attention for Efficient Transformers (Student Abstract) AAAI 2024

Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models AAAI 2024

One-pass Multiple Conformer and Foundation Speech Systems Compression and Quantization Using An All-in-one Neural Model INTERSPEECH 2024

Retraining-Free Model Quantization via One-Shot Weight-Coupling Learning CVPR 2024

Efficient Multi-task LLM Quantization and Serving for Multiple LoRA Adapters NIPS 2024

FlattenQuant: Breaking through the Inference Compute-bound for Large Language Models with Per-tensor Quantization COLING 2024

Fed-QSSL: A Framework for Personalized Federated Learning under Bitwidth and Data Heterogeneity AAAI 2024

TinyAgent: Function Calling at the Edge EMNLP 2024

BitDelta: Your Fine-Tune May Only Be Worth One Bit NIPS 2024