← Keywords

knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

Distill, Adapt, Distill: Training Small, In-Domain Models for Neural Machine Translation ACL 2020
Balancing Cost and Benefit with Tied-Multi Transformers ACL 2020
CASIA’s System for IWSLT 2020 Open Domain Translation ACL 2020
Distributed Distillation for On-Device Learning NIPS 2020
Block-Wisely Supervised Neural Architecture Search With Knowledge Distillation CVPR 2020
Creating Something From Nothing: Unsupervised Knowledge Distillation for Cross-Modal Hashing CVPR 2020
Spatio-Temporal Graph for Video Captioning With Knowledge Distillation CVPR 2020
FastBERT: a Self-distilling BERT with Adaptive Inference Time ACL 2020
Multi-label Contrastive Predictive Coding NIPS 2020
Classification Under Misspecification: Halfspaces, Generalized Linear Models, and Evolvability NIPS 2020
Fast, Accurate, and Simple Models for Tabular Data via Augmented Distillation NIPS 2020
Reverse-engineering recurrent neural network solutions to a hierarchical inference task for mice NIPS 2020
Residual Distillation: Towards Portable Deep Neural Networks without Shortcuts NIPS 2020
Kernel Based Progressive Distillation for Adder Neural Networks NIPS 2020
CompRess: Self-Supervised Learning by Compressing Representations NIPS 2020
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge NIPS 2020
Knowledge Distillation in Wide Neural Networks: Risk Bound, Data Efficiency and Imperfect Teacher NIPS 2020
ExpandNets: Linear Over-parameterization to Train Compact Convolutional Networks NIPS 2020
Self-Distillation Amplifies Regularization in Hilbert Space NIPS 2020
DeepI2I: Enabling Deep Hierarchical Image-to-Image Translation by Transferring from GANs NIPS 2020
Data Diversification: A Simple Strategy For Neural Machine Translation NIPS 2020
Movement Pruning: Adaptive Sparsity by Fine-Tuning NIPS 2020
Black-Box Ripper: Copying black-box models using generative evolutionary algorithms NIPS 2020
Knowledge Transfer in Multi-Task Deep Reinforcement Learning for Continuous Control NIPS 2020
AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search IJCAI 2020