Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion

Binchi Zhang; Zaiyi Zheng; Zhengzhang Chen; Jundong Li

2025 ICML ICML 2025

Beyond the Permutation Symmetry of Transformers: The Role of Rotation for Model Fusion

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Binchi Zhang, Zaiyi Zheng, Zhengzhang Chen, Jundong Li

Topics

Machine Learning > Optimization & Theory > Optimization Deep Learning > Architectures > Transformers Artificial Intelligence > Core AI > Large Language Models

Keywords

model fusion parameter space pre-trained transformer self-attention layer rotation symmetry equivalence set

Download PDF