← Application Areas
Machine Learning › Application Areas ›

Knowledge Distillation

2907 directly classified papers

Papers per year

Papers

Incremental pre-training from smaller language models ACL 2024
Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation AAAI 2024
Knowledge boosting during low-latency inference INTERSPEECH 2024
Self-Distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach AAAI 2024
FedTAD: Topology-aware Data-free Knowledge Distillation for Subgraph Federated Learning IJCAI 2024
Data Distribution Distilled Generative Model for Generalized Zero-Shot Recognition AAAI 2024
No Head Left Behind – Multi-Head Alignment Distillation for Transformers AAAI 2024
Domain Adaptation for Large-Vocabulary Object Detectors NIPS 2024
Expediting Contrastive Language-Image Pretraining via Self-Distilled Encoders AAAI 2024
Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models IJCAI 2024
A Small and Fast BERT for Chinese Medical Punctuation Restoration INTERSPEECH 2024
Impartial Adversarial Distillation: Addressing Biased Data-Free Knowledge Distillation via Adaptive Constrained Optimization AAAI 2024
Online Knowledge Distillation of Decoder-Only Large Language Models for Efficient Speech Recognition INTERSPEECH 2024
Knowledge Transfer via Compact Model in Federated Learning (Student Abstract) AAAI 2024
RaD-Net 2: A causal two-stage repairing and denoising speech enhancement network with knowledge distillation and complex axial self-attention INTERSPEECH 2024
Omnipotent Distillation with LLMs for Weakly-Supervised Natural Language Video Localization: When Divergence Meets Consistency AAAI 2024
Rethinking Attention: Exploring Shallow Feed-Forward Neural Networks as an Alternative to Attention Layers in Transformers (Student Abstract) AAAI 2024
Hear You Say You: An Efficient Framework for Marine Mammal Sounds’ Classification AAAI 2024
Knowledge Distillation for Tiny Speech Enhancement with Latent Feature Augmentation INTERSPEECH 2024
Complementary Knowledge Distillation for Robust and Privacy-Preserving Model Serving in Vertical Federated Learning AAAI 2024
ShareBERT: Embeddings Are Capable of Learning Hidden Layers AAAI 2024
Apprenticeship-Inspired Elegance: Synergistic Knowledge Distillation Empowers Spiking Neural Networks for Efficient Single-Eye Emotion Recognition IJCAI 2024
CSL: Class-Agnostic Structure-Constrained Learning for Segmentation Including the Unseen AAAI 2024
LLM vs Small Model? Large Language Model Based Text Augmentation Enhanced Personality Detection Model AAAI 2024
Discrepancy and Uncertainty Aware Denoising Knowledge Distillation for Zero-Shot Cross-Lingual Named Entity Recognition AAAI 2024