| ||||
| ||||
![]() Title:Modality-Embedded Set Transformer Pooling for Multimodal Prostate Cancer Survival Prediction Conference:IEEE CBMS 2026 Tags:biochemical recurrence, cross-attention, multimodal learning and prostate cancer Abstract: Biochemical recurrence (BCR) after curative-intent therapy for prostate cancer is assessed from heterogeneous evidence spanning histopathology, multiparametric MRI (mpMRI), and structured clinical variables. Learning robust multimodal predictors remains challenging due to small cohort sizes, modality-specific noise, and missingness. In this work, we study intermediate fusion for BCR risk prediction on the MICCAI CHIMERA benchmark under a controlled setup. We train attention-based multiple instance learning (ABMIL) aggregators to obtain patient-level embeddings for whole-slide histology patches and for sequence-wise MRI-CORE slice embeddings (ADC/HBV/T2w), and then compare two fusion operators with matched modality projections, modality-type embeddings, and an identical prediction head: (i) a modality-token Transformer encoder with CLS readout (MEFT) and (ii) seeded cross-attention pooling with learned fusion tokens (MEST). All models are trained with the Cox objective and evaluated using a fixed 5-fold cross-validation with out-of-fold embedding generation to avoid leakage. Both fusion approaches achieve comparable performance (C-index ≈ 0.85), with MEST showing slightly lower fold-to-fold variance. Ablations indicate that histology and clinical variables dominate performance, while mpMRI provides complementary gains when fused despite weaker monomodal performance. These results suggest that, on CHIMERA, the fusion operator has a modest effect under matched embeddings, and that preserving sequence-wise mpMRI representations can improve multimodal risk stratification. Modality-Embedded Set Transformer Pooling for Multimodal Prostate Cancer Survival Prediction ![]() Modality-Embedded Set Transformer Pooling for Multimodal Prostate Cancer Survival Prediction | ||||
| Copyright © 2002 – 2026 EasyChair |
