REB-former: RWKV-enhanced E-branchformer for Speech Recognition

Transformer-based architectures have achieved significant success in automatic speech recognition (ASR). However, the quadratic complexity of their self-attention mechanisms limits processing efficiency for speech sequences. To address this issue, this paper proposes the Receptance Weighted Key Value (RWKV)-enhanced E-Branchformer (REB-former). Specifically, the REB-former interleaves the E-Branchformer and RWKV layers, combining different attention mechanisms to reduce computational complexity and enhance speech modeling. To overcome RWKV's unidirectional limitation, we introduce the GroupBiRWKV module for efficient contextual feature capture. The results show that the REB-former outperforms the E-Branchformer in terms of computational efficiency and inference speed, achieving a relative reduction of up to 7.1% in the word error rate (WER). On the LibriSpeech 100h dataset, our model achieves WER of 6.0%/15.8% on test-clean/test-other, setting a new state-of-the-art performance.