← Keywords

model compression

3283 papers

Explore in graph

Also known as

MC

Co-occurring keywords

knowledge distillation (3680) large language model (12755) neural network (6616) efficient computing (779) neural network optimization (1293) transfer learning (5442) convolutional neural network (4216) neural network pruning (265) language model (4573) parameter efficiency (415)

Papers

FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search ICCV 2021
Self-Born Wiring for Neural Trees ICCV 2021
RankDistil: Knowledge Distillation for Ranking AISTATS 2021
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks JMLR 2021
Distilling Optimal Neural Networks: Rapid Search in Diverse Spaces ICCV 2021
Tokens-to-Token ViT: Training Vision Transformers From Scratch on ImageNet ICCV 2021
BERT might be Overkill: A Tiny but Effective Biomedical Entity Linker based on Residual Convolutional Neural Networks EMNLP 2021
A Greedy Algorithm for Quantizing Neural Networks JMLR 2021
I-BERT: Integer-only BERT Quantization ICML 2021
NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application EMNLP 2021
Online Multi-Granularity Distillation for GAN Compression ICCV 2021
MSD: Saliency-aware Knowledge Distillation for Multimodal Understanding EMNLP 2021
EfficientBERT: Progressively Searching Multilayer Perceptron via Warm-up Knowledge Distillation EMNLP 2021
How to Select One Among All ? An Empirical Study Towards the Robustness of Knowledge Distillation in Natural Language Understanding EMNLP 2021
Beyond Preserved Accuracy: Evaluating Loyalty and Robustness of BERT Compression EMNLP 2021
BASE Layers: Simplifying Training of Large, Sparse Models ICML 2021
Meta-KD: A Meta Knowledge Distillation Framework for Language Model Compression across Domains ACL 2021
Multi-stage Pre-training over Simplified Multimodal Pre-training Models ACL 2021
Elastic Consistency: A Practical Consistency Model for Distributed Stochastic Gradient Descent AAAI 2021
Interpretable Embedding Procedure Knowledge Transfer via Stacked Principal Component Analysis and Graph Neural Network AAAI 2021
Collaborative Group Learning AAAI 2021
ActNN: Reducing Training Memory Footprint via 2-Bit Activation Compressed Training ICML 2021
LassoNet: A Neural Network with Feature Sparsity JMLR 2021
Beyond Distillation: Task-level Mixture-of-Experts for Efficient Inference EMNLP 2021
Towards Zero-Shot Knowledge Distillation for Natural Language Processing EMNLP 2021