knowledge distillation

3680 papers

Explore in graph

Also known as

CD DMD LORA KL DNA SELF-DISTILLATION TKD NBOD AD KD AOTD KI GID FD MKD SEQKD

Co-occurring keywords

model compression (3283) large language model (12755) transfer learning (5442) domain adaptation (4578) representation learning (6174) neural network (6616) language model (4573) continual learning (1164) catastrophic forgetting (939) contrastive learning (3979)

Papers

A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation ICCV 2023

Acoustic NLOS Imaging with Cross Modal Knowledge Distillation IJCAI 2023

Recycle-and-Distill: Universal Compression Strategy for Transformer-based Speech SSL Models with Attention Map Reusing and Masking Distillation INTERSPEECH 2023

Teaching What You Should Teach: A Data-Based Distillation Method IJCAI 2023

Protecting Language Generation Models via Invisible Watermarking ICML 2023

Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning ICCV 2023

Knowledge Distillation Approach for Efficient Internal Language Model Estimation INTERSPEECH 2023

Distilling Internet-Scale Vision-Language Models into Embodied Agents ICML 2023

Vertical Federated Graph Neural Network for Recommender System ICML 2023

Data Efficient Neural Scaling Law via Model Reusing ICML 2023

Understand and Modularize Generator Optimization in ELECTRA-style Pretraining ICML 2023

SpotEM: Efficient Video Search for Episodic Memory ICML 2023

Semantic Segmentation with Bidirectional Language Models Improves Long-form ASR INTERSPEECH 2023

Incorporating Ultrasound Tongue Images for Audio-Visual Speech Enhancement through Knowledge Distillation INTERSPEECH 2023

Understanding Self-Distillation in the Presence of Label Noise ICML 2023

Parallel Neurosymbolic Integration with Concordia ICML 2023

Towards Understanding Ensemble Distillation in Federated Learning ICML 2023

Discrete Continuous Optimization Framework for Simultaneous Clustering and Training in Mixture Models ICML 2023

Linkless Link Prediction via Relational Distillation ICML 2023

Less is More: Task-aware Layer-wise Distillation for Language Model Compression ICML 2023

Understanding the Distillation Process from Deep Generative Models to Tractable Probabilistic Circuits ICML 2023

UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers ICML 2023

Quantifying the Knowledge in GNNs for Reliable Distillation into MLPs ICML 2023

Prototype-Sample Relation Distillation: Towards Replay-Free Continual Learning ICML 2023

Margin-based Neural Network Watermarking ICML 2023