ProtoSurv-X: Explainable Brain Tumor Survival Prediction
Abstract
Accurate survival prediction for patients with high-grade glioma is important for prognostic assessment and personalized treatment planning; however, substantial intratumoral heterogeneity, complex multimodal MRI patterns, and limited interpretability challenge existing deep learning approaches. This study aims to develop ProtoSurv-X, an explainable and uncertainty-aware framework for MRI-based glioma survival prediction that integrates probabilistic tumor habitat modeling, prototype-guided learning, evidential prediction, and evidence-grounded clinical explanations. A unified cohort was constructed from the BraTS 2019 and BraTS 2020 datasets by removing duplicate subjects, resulting in 369 unique subjects, including 118 gross total resection patients with complete survival annotations. The proposed framework uses diffusion-enhanced SwinUNETR for tumor segmentation, probabilistic habitat construction, and fusion of radiomic, deep imaging, habitat, and age-related features. Prototype learning and survival-aware contrastive learning generate prognostic representations, while Evidential Deep Learning estimates risk and predictive uncertainty alongside continuous survival regression. For explanation generation, seven open-source large language models were benchmarked using structured model evidence, and a Meta-Llama-3.1-8B-Instruct model was fine-tuned using QLoRA. In five-fold cross-validation on the unified survival cohort, ProtoSurv-X achieved a mean absolute error of 145.2 ± 6.5 days, an RMSE of 189.4 ± 8.3 days, a C-index of 0.745 ± 0.008, and an integrated Brier score of 0.124. The segmentation module achieved Dice scores of 91.24%, 87.38%, and 81.76% for whole tumor, tumor core, and enhancing tumor, respectively, on BraTS 2019. The fine-tuned explanation model obtained a BERTScore of 0.915 and a hallucination rate of 2.6%. These findings indicate that ProtoSurv-X offers a unified computational framework for accurate, uncertainty-aware, and evidence-grounded glioma prognostic modeling, while further clinical and multicenter validation remains necessary
Downloads
References
R. Cao, X. Hu, L. Xiao, G. Qu, H. Huo, V. D. Calhoun, Y.-P. Wang, and X. Sun, “Cooperative multiplex GNN for high-grade glioma survival prediction from preoperative multi-modal radiomics-based brain networks,” IEEE Trans. Med. Imaging, vol. 45, no. 7, pp. 3464–3476, 2026. https://doi.org/10.1109/TMI.2026.3677065
S. Bakas, M. Reyes, A. Jakab, S. Bauer, M. Rempfler, A. Crimi, R. T. Shinohara et al., “Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge,” in BrainLesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, 2019, pp. 374–387. https://doi.org/10.17863/CAM.38755
A. H. Song, R. J. Chen, G. Jaume, A. J. Vaidya, A. S. Baras, and F. Mahmood, “Multimodal prototyping for cancer survival prediction,” in Proc. 41st Int. Conf. Mach. Learn. (ICML), 2024, pp. 46050–46073. https://dl.acm.org/doi/10.5555/3780338.3781154
Y. Xing, L. Huang, J. Ma, R. Hong, J. Qiu, P. Liu, K. He, H. Fu, and M. Feng, “DPSurv: Dual-prototype evidential fusion for uncertainty-aware and interpretable whole slide image survival prediction,” in Proc. Int. Conf. Mach. Learn. (ICML), 2026. https://icml.cc/virtual/2026/papers.html
M. O. Ronneberger, P. Fischer, and T. Brox, “U-Net: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Cham, Switzerland: Springer, 2015, pp. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
F. Milletari, N. Navab, and S.-A. Ahmadi, “V-Net: Fully convolutional neural networks for volumetric medical image segmentation,” in Proc. 4th Int. Conf. 3D Vision (3DV), Stanford, CA, USA, 2016, pp. 565–571. https://doi.org/10.1109/3DV.2016.79
F. Isensee, P. F. Jaeger, S. A. A. Kohl, J. Petersen, and K. H. Maier-Hein, “nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation,” Nat. Methods, vol. 18, no. 2, pp. 203–211, 2021. https://doi.org/10.1038/s41592-020-01008-z
A. Hatamizadeh, V. Nath, Y. Tang, D. Yang, H. R. Roth, and D. Xu, “Swin UNETR: Swin transformers for semantic segmentation of brain tumors in MRI images,” in Proc. Int. MICCAI BrainLes Workshop, Cham, Switzerland: Springer, 2021, pp. 272–284. https://doi.org/10.1007/978-3-031-08999-2_22
H. Liu, D. Wei, D. Lu, J. Sun, L. Wang, and Y. Zheng, “M3AE: Multimodal representation learning for brain tumor segmentation with missing modalities,” in Proc. AAAI Conf. Artif. Intell., vol. 37, no. 2, pp. 1657–1665, 2023. https://doi.org/10.1609/aaai.v37i2.25253
G. Liang, Q. Zhou, Z. Wang, J. Chen, L. Gu, C. Yao, S. Wu, B. Huang, and K. Chen, “Semantic-guided masked mutual learning for multi-modal brain tumor segmentation with arbitrary missing modalities,” in Proc. AAAI Conf. Artif. Intell., vol. 39, no. 5, pp. 5137–5145, 2025. https://doi.org/10.1609/aaai.v39i5.32545
Y. Zhao, C. Chen, Q. Y. Pang, Y. Fu, Q. Li, C. Tang, B. T. Ang, and Y. Jin, “Tackling dual-stage missing modalities in brain tumor segmentation via robust modality reconstruction and prompt-guided modality adaptation,” in Proc. AAAI Conf. Artif. Intell., vol. 40, no. 16, pp. 13314–13322, 2026. https://doi.org/10.1609/aaai.v40i16.38334
T. Liu, H. Jiang, and K. Huang, “KMD: Koopman multi-modality decomposition for generalized brain tumor segmentation under incomplete modalities,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 15663–15671, 2025. https://doi.org/10.1109/CVPR52734.2025.01460
H. Yang, J. Sun, and Z. Xu, “Learning unified hyper-network for multi-modal MR image synthesis and tumor segmentation with missing modalities,” IEEE Transactions on Medical Imaging, vol. 42, no. 12, pp. 3678–3689, 2023. https://doi.org/10.1109/TMI.2023.3301934
I. Mazumdar and J. Mukhopadhyay, “Multi-scale transformer-CNN network for brain tumor segmentation and survival prediction,” in Proc. 15th Indian Conf. Comput. Vis. Graph. Image Process. (ICVGIP), 2024, pp. 1–9. https://doi.org/10.1145/3702250.3702257
A. Datta, S. Sarkar, A. L. Sharma, and P. Ghosal, “Multimodal exponential moving average-guided representation for integrated tumor-segmentation and survival prediction,” Eng. Appl. Artif. Intell., vol. 167, Art. no. 113868, 2026. https://doi.org/10.1016/j.engappai.2026.113868
L. Qu, J. Xiao, X. Liu, C. Sun, H. Cui, Y. Fang, R. Su, Q. Jin, and L. Wei, “MDCS-MoAME: Multi-directional composite scanning with mixture of attention and Mamba experts for cancer survival prediction,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2026, pp. 14461–14470. https://openaccess.thecvf.com/content/CVPR2026/html/Qu_MDCS-MoAME_Multi-directional_Composite_Scanning_with_Mixture_of_Attention_and_Mamba_CVPR_2026_paper.html
A. H. Song, R. J. Chen, T. Ding, D. F. K. Williamson, G. Jaume, and F. Mahmood, “Morphological prototyping for unsupervised slide representation learning in computational pathology,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 11566–11578. https://doi.org/10.1109/CVPR52733.2024.01099
J. Wu, M. Chen, X. Ke, T. Xun, X. Jiang, H. Zhou, L. Shao, and Y. Kong, “Learning heterogeneous tissues with mixture of experts for gigapixel whole slide images,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2025, pp. 5144–5153. https://doi.org/10.1109/CVPR52734.2025.00485
C. Li, C. Wong, S. Zhang, N. Usuyama, H. Liu, J. Yang, T. Naumann, H. Poon, and J. Gao, “LLaVA-Med: Training a large language-and-vision assistant for biomedicine in one day,” in Advances in Neural Information Processing Systems, vol. 36, pp. 28541–28564, 2023. https://doi.org/10.52202/075280-1240
C. Wu, X. Zhang, Y. Zhang, H. Hui, Y. Wang, and W. Xie, “Towards generalist foundation model for radiology by leveraging web-scale 2D and 3D medical data,” Nat. Commun., vol. 16, no. 1, Art. no. 7866, 2025. https://doi.org/10.1038/s41467-025-62385-7
B. Kandala and R. K. G. V. S. Raj Kumar, “Trustworthiness Index SHapley Additive exPlanations-based skin cancer diagnosis via hybrid deep concept bottleneck learning and LLM-driven reasoning,” Biomed. Signal Process. Control, vol. 122, Art. no. 110309, 2026. https://doi.org/10.1016/j.bspc.2026.110309
B. W. Patterson et al., “Evaluating clinical AI summaries with large language models as judges,” npj Digital Medicine, vol. 8, Art. no. 640, 2025. https://doi.org/10.1038/s41746-025-02005-2
T. Henry, A. Carré, M. Lerousseau, T. Estienne, C. Robert, N. Paragios, and E. Deutsch, “Brain tumor segmentation with self-ensembled, deeply-supervised 3D U-Net neural networks: A BraTS 2020 challenge solution,” in BrainLesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries, Cham, Switzerland: Springer, 2021, pp. 327–339. https://doi.org/10.1007/978-3-030-72084-1_30
Samo, H., Ali, K., Memon, M., Abbasi, F.A., Koondhar, M.Y. and Dahri, K., 2024. Fine-tuning mistral 7b large language model for python query response and code generation: A parameter efficient approach. VAWKUM Transactions on Computer Sciences, 12(1), pp.205-217. https://doi.org/10.21015/vtcs.v12i1.1885
Prucker, P., Bressem, K.K., Kim, S.H., Weller, D., Kader, A., Dorfner, F.J., Ziegelmayer, S., Graf, M.M., Lemke, T., Gassert, F. and Can, E., 2026. Privacy-preserving generation of structured lymphoma progression reports from cross-sectional imaging: A comparative analysis of Llama 3.3 and Llama 4. Journal of Imaging Informatics in Medicine, 39(2), pp.1868-1878. https://doi.org/10.1007/s10278-025-01618-z
M. Abdin, J. Aneja, H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, J. R. Lee, Y. T. Lee, Y. Li, W. Liu, C. C. T. Mendes, A. Nguyen, E. Price, G. de Rosa, O. Saarikivi, A. Salim, S. Shah, X. Wang, R. Ward, Y. Wu, D. Yu, C. Zhang, and Y. Zhang, “Phi-4 Technical Report,” Microsoft Research, MSR-TR-2024-57, 2024. https://www.microsoft.com/en-us/research/publication/phi-4-technical-report/
T. Lieberum, S. Rajamanoharan, A. Conmy, L. Smith, N. Sonnerat, V. Varma, J. Kramár, A. Dragan, R. Shah, and N. Nanda, “Gemma Scope: Open sparse autoencoders everywhere all at once on Gemma 2,” in Proc. 7th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, 2024, pp. 278–300. https://doi.org/10.18653/v1/2024.blackboxnlp-1.19
K. A. Huang, S. V. Prakash, D. Samvelian, N. S. Prakash, and S. Prakash, “Implementation of a locally deployed Qwen2.5-7B-Instruct pipeline for structured classification of elbow MRI reports,” Cureus, vol. 18, no. 8, 2026. https://doi.org/10.7759/cureus.114918.
D. Guo et al., “DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning,” Nature, vol. 645, pp. 633–638, 2025. https://doi.org/10.1038/s41586-025-09422-z
T. Dettmers, A. Pagnoni, A. Holtzman, and L. Zettlemoyer, “QLoRA: Efficient finetuning of quantized LLMs,” in Advances in Neural Information Processing Systems, vol. 36, pp. 10088–10115, 2023. https://doi.org/10.52202/075280-0441
Z. Li, W. Chen, H. Zhong, and C. Liang, “PCLSurv: A prototypical contrastive learning-based multi-omics data integration model for cancer survival prediction,” Briefings in Bioinformatics, vol. 26, no. 2, Art. no. bbaf124, 2025. https://doi.org/10.1093/bib/bbaf124.
W. Wang, C. Chen, M. Ding, H. Yu, S. Zha, and J. Li, “TransBTS: Multimodal brain tumor segmentation using transformer,” in Proc. Int. Conf. Med. Image Comput. Comput.-Assist. Intervent. (MICCAI), Cham, Switzerland: Springer, 2021, pp. 109–119. https://doi: 10.1007/978-3-030-87193-2_11.
T. Chen, S. Kornblith, M. Norouzi, and G. Hinton, “A simple framework for contrastive learning of visual representations,” in Proc. 37th Int. Conf. Mach. Learn. (ICML), vol. 119, 2020, pp. 1597–1607. https://proceedings.mlr.press/v119/chen20j.html
P. Khosla, P. Teterwak, C. Wang, A. Sarna, Y. Tian, P. Isola, A. Maschinot, C. Liu, and D. Krishnan, “Supervised contrastive learning,” in Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 33 2020, pp. 18661–18673. https://proceedings.neurips.cc/paper_files/paper/2020/hash/d89a66c7c80a29b1bdbab0f2a1a94af8-Abstract.html
J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems, vol. 33, pp. 6840–6851, 2020. https://doi.org/10.5555/3495724.3496298
A. Q. Nichol and P. Dhariwal, “Improved denoising diffusion probabilistic models,” in Proc. 38th Int. Conf. Mach. Learn. (ICML), vol. 139, 2021, pp. 8162–8171. Available: https://proceedings.mlr.press/v139/nichol21a.html
D. R. Cox, “Regression models and life-tables,” J. Roy. Stat. Soc. Ser. B (Methodological), vol. 34, no. 2, pp. 187–202, 1972. https://doi.org/10.1111/j.2517-6161.1972.tb00899.x
H. Ishwaran, U. B. Kogalur, E. H. Blackstone, and M. S. Lauer, “Random survival forests,” Ann. Appl. Stat., vol. 2, no. 3, pp. 841–860, 2008. https://doi.org/10.1214/08-AOAS169
J. L. Katzman, U. Shaham, A. Cloninger, J. Bates, T. Jiang, and Y. Kluger, “DeepSurv: Personalized treatment recommender system using a Cox proportional hazards deep neural network,” BMC Med. Res. Methodol., vol. 18, no. 1, Art. no. 24, 2018. https://doi.org/10.1186/s12874-018-0482-1
C. Lee, W. R. Zame, J. Yoon, and M. van der Schaar, “DeepHit: A deep learning approach to survival analysis with competing risks,” in Proc. 32nd AAAI Conf. Artif. Intell. (AAAI), New Orleans, LA, USA, 2018, pp. 2314–2321. https://doi.org/10.1609/aaai.v32i1.11842
M. Sensoy, L. Kaplan, and M. Kandemir, “Evidential deep learning to quantify classification uncertainty,” in Advances in Neural Information Processing Systems, vol. 31, pp. 3179–3189, 2018. https://doi.org/10.5555/3327144.3327239
C.-Y. Hsieh, C.-L. Li, C.-K. Yeh, H. Nakhost, Y. Fujii, A. Ratner, R. Krishna, C.-Y. Lee, and T. Pfister, “Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes,” in Findings of the Association for Computational Linguistics: ACL 2023, pp. 8003–8017, 2023. https://doi.org/10.18653/v1/2023.findings-acl.507
S.-A. Qi, N. Kumar, M. Farrokh, W. Sun, L.-H. Kuan, R. Ranganath, R. Henao, and R. Greiner, “An Effective Meaningful Way to Evaluate Survival Models,” in Proc. 40th Int. Conf. Mach. Learn. (ICML), vol. 202, 2023, pp. 28244–28276. https://proceedings.mlr.press/v202/qi23b.html
Copyright (c) 2026 Neeshu Chaudhary, Saurabh Shah, Chintan Thacker
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlikel 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
.png)
.png)
.png)
.png)
.png)
