conftrace_
← Keywords

bert compression

13 papers

Explore in graph

Co-occurring keywords

model compression (3283) knowledge distillation (3680) neural network optimization (1293) model quantization (279) student model (106) transfer learning (5442) model distillation (105) token selection (25) transformer efficiency (12) neural architecture search (665)

Papers

Maximizing the Effectiveness of Larger BERT Models for Compression ACL 2025
How to Trade Off the Quantity and Capacity of Teacher Ensemble: Learning Categorical Distribution to Stochastically Employ a Teacher for Distillation AAAI 2024
Weight-Inherited Distillation for Task-Agnostic BERT Compression NAACL 2024
Adaptive Contrastive Knowledge Distillation for BERT Compression ACL 2023
Efficient Two-Stage Progressive Quantization of BERT EMNLP 2022
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022
Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection ACL 2022
BinaryBERT: Pushing the Limit of BERT Quantization IJCNLP 2021
ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques AAAI 2021
Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation IJCNLP 2021
AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search IJCAI 2020
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT AAAI 2020
Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation AACL 2020