Knowledge Distillation

Lightweight Zero-shot Text-to-Speech with Mixture of Adapters Interspeech 2024 LiSA: LiDAR Localization with Semantic Awareness CVPR 2024 Llama SLayer 8B: Shallow Layers Hold the Key to Knowledge Injection EMNLP 2024 LLM-based Multi-Level Knowledge Generation for Few-shot Knowledge Graph Completion IJCAI 2024 LLMR: Knowledge Distillation with a Large Language Model-Induced Reward COLING 2024 LLMs-as-Instructors: Learning from Errors Toward Automating Model Improvement EMNLP 2024 LLM See, LLM Do: Leveraging Active Inheritance to Target Non-Differentiable Objectives EMNLP 2024 LLM vs Small Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model AAAI 2024 LoftQ: LoRA-Fine-Tuning-aware Quantization for Large Language Models ICLR 2024 Logit Standardization in Knowledge Distillation CVPR 2024 LoQT: Low-Rank Adapters for Quantized Pretraining NeurIPS 2024 LoRA-Guard: Parameter-Efficient Guardrail Adaptation for Content Moderation of Large Language Models EMNLP 2024 LoraRetriever: Input-Aware LoRA Retrieval and Composition for Mixed Tasks in the Wild ACL 2024 Low-Rank Knowledge Decomposition for Medical Foundation Models CVPR 2024 Low-Rank Rescaled Vision Transformer Fine-Tuning: A Residual Design Approach CVPR 2024 LP-OVOD: Open-Vocabulary Object Detection by Linear Probing WACV 2024 LungAdapter: Efficient Adapting Audio Spectrogram Transformer for Lung Sound Classification Interspeech 2024 M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation ACL 2024 Machine Learning-Powered Combinatorial Clock Auction AAAI 2024 MadEye: Boosting Live Video Analytics Accuracy with Adaptive Camera Configurations NSDI 2024 Make Graph Neural Networks Great Again: A Generic Integration Paradigm of Topology-Free Patterns for Traffic Speed Prediction IJCAI 2024 Masked Distillation Advances Self-Supervised Transformer Architecture Search ICLR 2024 MCAD: Multi-teacher Cross-modal Alignment Distillation for efficient image-text retrieval NAACL 2024 MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning ACL 2024 MEND: Meta Demonstration Distillation for Efficient and Effective In-Context Learning ICLR 2024