bert compression

13 papers

Explore in graph

Co-occurring keywords

model compression (3283) knowledge distillation (3680) neural network optimization (1293) model quantization (279) student model (106) transfer learning (5442) model distillation (105) token selection (25) transformer efficiency (12) neural architecture search (665)

Papers

Maximizing the Effectiveness of Larger BERT Models for Compression ACL 2025

How to Trade Off the Quantity and Capacity of Teacher Ensemble: Learning Categorical Distribution to Stochastically Employ a Teacher for Distillation AAAI 2024

Weight-Inherited Distillation for Task-Agnostic BERT Compression NAACL 2024

Adaptive Contrastive Knowledge Distillation for BERT Compression ACL 2023

Efficient Two-Stage Progressive Quantization of BERT EMNLP 2022

The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models EMNLP 2022

Pyramid-BERT: Reducing Complexity via Successive Core-set based Token Selection ACL 2022

BinaryBERT: Pushing the Limit of BERT Quantization IJCNLP 2021

ROSITA: Refined BERT cOmpreSsion with InTegrAted techniques AAAI 2021

Marginal Utility Diminishes: Exploring the Minimum Knowledge for BERT Knowledge Distillation IJCNLP 2021

AdaBERT: Task-Adaptive BERT Compression with Differentiable Neural Architecture Search IJCAI 2020

Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT AAAI 2020

Towards Non-task-specific Distillation of BERT via Sentence Representation Approximation AACL 2020