UniFast-HGR: Scalable and Efficient Maximal Correlation for Multimodal Models
Abstract
This paper presents UniFast-HGR, a scalable surrogate for Hirschfeld–Gebelein–Rényi (HGR) maximal correlation in high-dimensional multimodal learning. The method replaces explicit covariance whitening with centered and $\ell_2$-normalized cosine alignment, uses the covariance-to-Gram trace identity to construct a local-batch structural surrogate, and removes invariant diagonal self-correlation through Trivial Spectrum Suppression (TSS). The resulting objective retains paired dependence maximization and the covariance-control role of Soft-HGR while replacing its finite-sample covariance estimator with a differentiable local-batch objective whose dominant structural cost is $O(m^2K)$ for local batch size $m$ and feature dimension $K$. OptFast-HGR further reduces the practical memory burden by estimating the off-diagonal structural term through stochastic projection. Experiments across retrieval, image classification, remote sensing segmentation, and multimodal emotion recognition show consistent gains over covariance-based HGR/CCA variants and contrastive or neural MI-estimator objectives on strong multimodal backbones, while microbenchmarks confirm stable behavior at extreme feature dimensions.