← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Revisiting Knowledge Distillation for Autoregressive Language Models ACL 2024
Superfiltering: Weak-to-Strong Data Filtering for Fast Instruction-Tuning ACL 2024
ADAM: Dense Retrieval Distillation with Adaptive Dark Examples ACL 2024
3MVRD: Multimodal Multi-task Multi-teacher Visually-Rich Form Document Understanding ACL 2024
AraCLIP: Cross-Lingual Learning for Effective Arabic Image Retrieval ACL 2024
LEMON: Reviving Stronger and Smaller LMs from Larger LMs with Linear Parameter Fusion ACL 2024
MELoRA: Mini-Ensemble Low-Rank Adapters for Parameter-Efficient Fine-Tuning ACL 2024
Text-like Encoding of Collaborative Information in Large Language Models for Recommendation ACL 2024
Improving Text Embeddings with Large Language Models ACL 2024
Investigating and Mitigating the Multimodal Hallucination Snowballing in Large Vision-Language Models ACL 2024
M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation ACL 2024
Ranking Entities along Conceptual Space Dimensions with LLMs: An Analysis of Fine-Tuning Strategies ACL 2024
KnowledgeSG: Privacy-Preserving Synthetic Text Generation with Knowledge Distillation from Server EMNLP 2024
RA2FD: Distilling Faithfulness into Efficient Dialogue Systems EMNLP 2024
FIRST: Teach A Reliable Large Language Model Through Efficient Trustworthy Distillation EMNLP 2024
Optimizing Chinese Lexical Simplification Across Word Types: A Hybrid Approach EMNLP 2024
Improve Student’s Reasoning Generalizability through Cascading Decomposed CoTs Distillation EMNLP 2024
SynthesizRR: Generating Diverse Datasets with Retrieval Augmentation EMNLP 2024
xCOMET-lite: Bridging the Gap Between Efficiency and Quality in Learned MT Evaluation Metrics EMNLP 2024
BMRetriever: Tuning Large Language Models as Better Biomedical Text Retrievers EMNLP 2024
INDUS: Effective and Efficient Language Models for Scientific Applications EMNLP 2024
RRADistill: Distilling LLMs’ Passage Ranking Ability for Long-Tail Queries Document Re-Ranking on a Search Engine EMNLP 2024
Visual Editing with LLM-based Tool Chaining: An Efficient Distillation Approach for Real-Time Applications EMNLP 2024
GOVERN: Gradient Orientation Vote Ensemble for Multi-Teacher Reinforced Distillation EMNLP 2024
BSharedRAG: Backbone Shared Retrieval-Augmented Generation for the E-commerce Domain EMNLP 2024