conftrace_
← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Sharing Encoder Representations across Languages, Domains and Tasks in Large-Scale Spoken Language Understanding ACL 2023
Domain-specific transformer models for query translation ACL 2023
A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Training ACL 2023
Long-Tailed Question Answering in an Open World ACL 2023
Attention-Based Depth Distillation with 3D-Aware Positional Encoding for Monocular 3D Object Detection AAAI 2023
Jointly Reparametrized Multi-Layer Adaptation for Efficient and Private Tuning ACL 2023
Adaptive Contrastive Knowledge Distillation for BERT Compression ACL 2023
CKDST: Comprehensively and Effectively Distill Knowledge from Machine Translation to End-to-End Speech Translation ACL 2023
When Gradient Descent Meets Derivative-Free Optimization: A Match Made in Black-Box Scenario ACL 2023
G3R: A Graph-Guided Generate-and-Rerank Framework for Complex and Cross-domain Text-to-SQL Generation ACL 2023
f-Divergence Minimization for Sequence-Level Knowledge Distillation ACL 2023
Labels are not necessary: Assessing peer-review helpfulness using domain adaptation based on self-training ACL 2023
Federated Domain Adaptation for Named Entity Recognition via Distilling with Heterogeneous Tag Sets ACL 2023
Boost Transformer-based Language Models with GPU-Friendly Sparsity and Quantization ACL 2023
CocaCLIP: Exploring Distillation of Fully-Connected Knowledge Interaction Graph for Lightweight Text-Image Retrieval ACL 2023
Triplet-Free Knowledge-Guided Response Generation ACL 2023
LLM-Pruner: On the Structural Pruning of Large Language Models NIPS 2023
Adaptive Mixing of Auxiliary Losses in Supervised Learning AAAI 2023
Operator Selection and Ordering in a Pipeline Approach to Efficiency Optimizations for Transformers ACL 2023
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling ACL 2023
Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models NIPS 2023
Propagating Knowledge Updates to LMs Through Distillation NIPS 2023
Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Tasks NIPS 2023
Cappy: Outperforming and Boosting Large Multi-Task LMs with a Small Scorer NIPS 2023
ZipLM: Inference-Aware Structured Pruning of Language Models NIPS 2023