knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

SoundNet: Learning Sound Representations from Unlabeled Video NIPS 2016

Bayesian dark knowledge NIPS 2015

Do Deep Nets Really Need to be Deep? NIPS 2014

Near-Maximum Entropy Models for Binary Neural Representations of Natural Images NIPS 2007

Distilling dialogues - A method using natural dialogue corpora for dialogue systems development NAACL 2000