Skip to content
conftrace_
2026 ICML ICML 2026

(Sparse) Attention to the Details: Preserving Spectral Fidelity in ML-based Weather Forecasting Models

We introduce Mosaic, a probabilistic weather forecasting model that addresses two sources of spectral degradation in ML-based weather prediction: training to predict the ensemble mean deterministically and compressive encoding creating an information bottleneck. Mosaic combines learned functional perturbations for ensemble forecasting with block-sparse attention, a hardware-aligned formulation that shares keys and values across spatially adjacent queries, enabling each block to dynamically attend to the most relevant regions. By capturing arbitrarily long-range dependencies at linear cost, Mosaic processes high-resolution weather data without compression. On IFS HRES data, Mosaic at 1.5$^\circ$ resolution matches or outperforms models trained on 0.25$^\circ$ data, with individual ensemble members exhibiting near-perfect spectral alignment across all resolved frequencies.

More from ICML 2026

All ICML 2026 papers →