Survival analysis of histopathological image based on a pretrained hypergraph model of spatial transcriptomics data

Survival analysis is critical for clinical decision-making and prognosis in breast cancer treatment. Recent multimodal approaches leverage histopathology images and bulk RNA-seq to improve survival prediction performance, but these approaches fail to explore spatial distribution at the cellular level. In this work, we present a multimodal hypergraph neural network for survival analysis (MHNN-surv) that introduces a pre-trained model for spatial transcriptomic prediction. The method is characterized by making full use of histopathological images to reveal both morphological and genetic information, thus improving the interpretation of heterogeneity. Specifically, MHNN-surv first slices Whole-Slide Imaging (WSI) into patch images, followed by extracting image features and predicting spatial transcriptomic, respectively. Subsequently, an image-based hypergraph is constructed based on three-dimensional nearest-neighbor relationships, while a gene-based hypergraph is formed based on gene expression similarity. By fusing the dual hypergraphs, MHNN-surv performs an in-depth survival analysis on breast cancer using the Cox proportional hazards model. The experimental results demonstrate that MHNN-surv outperforms the state-of-the-art multimodal models in survival analysis.