conftrace_
← Authors

Sihwa Lee

3 papers · 2022–2024 · 3 conferences · across top CS/AI conferences

Achievements

Jump to papers ↓
🌍 Conference Polyglot (3) 🌉 Interdisciplinary Bridge 🧭 Keyword Pioneer 🐝 Cross-Pollinator (15)

Conferences

ACL (1) EMNLP (1) NIPS (1)

Top co-authors

Minsoo Kim (3) Jungwook Choi (3) Du-Seong Chang (2) Wonyong Sung (2) Janghwan Lee (1) Sukjin Hong (1) Suk-Jin Hong (1)

Keywords

knowledge distillation (2) model compression (2) neural network optimization (1) language model (1) parameter-efficient fine-tuning (1) low-rank adaptation (1) transformer encoder (1) quantization-aware training (1) quantization aware training (1) attention recovery (1) model quantization (1)

Papers

RA-LoRA: Rank-Adaptive Parameter-Efficient Fine-Tuning for Accurate 2-bit Quantized Large Language Models ACL 2024 Token-Scaled Logit Distillation for Ternary Weight Generative Language Models NIPS 2023 Understanding and Improving Knowledge Distillation for Quantization Aware Training of Large Transformer Encoders EMNLP 2022