A Review of Deep Learning Methods for Brain Tumor Segmentation with Missing Modalities

Yamin Wang; Beibei Hou

doi:10.54097/nn608663

Authors

Yamin Wang
Beibei Hou

DOI:

https://doi.org/10.54097/nn608663

Keywords:

Missing modality, Brain tumor segmentation, Magnetic Resonance Imaging (MRI)

Abstract

Multi-modality imaging significantly enhances the accuracy and reliability of brain tumor segmentation by providing complementary biological information. However, in clinical practice, obtaining a complete set of Magnetic Resonance Imaging (MRI) modalities is often hindered by equipment, time, and cost constraints. The challenge of missing modalities thus becomes a major obstacle to achieving high-performance segmentation. This paper systematically reviews emerging methods addressing this issue, with a focus on their network architectures. The main strategies include data synthesis for generating missing scans, hetero-modal segmentation utilizing flexible architectures for variable inputs, and Knowledge Distillation (KD), which transfers knowledge from models trained on complete datasets. Building on these foundations, we analyze the novelty, strengths, and limitations of each method. To provide context, we also introduce commonly used MRI datasets. Ultimately, this review aims to deliver a comprehensive performance evaluation of compensation techniques and outline promising future directions for overcoming this persistent clinical challenge.

Downloads

Download data is not yet available.

References

[1] Mohammad Havaei,Axel Davy,David Warde{-}Farley, et al. Brain Tumor Segmentation with Deep Neural Networks[J]. Corr, 2015, abs/1505.03540.

[2] Fabian Isensee,Paul F. Jaeger,Simon A. A. Kohl, et al. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation[J]. Nature Methods, 2021, 18(2): 203-211.

[3] Bjoern H. Menze,Andras Jakab,Stefan Bauer, et al. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)[J]. Ieee Transactions on Medical Imaging, 2015, 34(10): 1993-2024.

[4] Liyue Shen,Wentao Zhu,Xiaosong Wang, et al. Multi-Domain Image Completion for Random Missing Input Data[J]. Corr, 2020, abs/2007.05534.

[5] Dingwen Zhang,Guohai Huang,Qiang Zhang, et al. Exploring Task Structure for Brain Tumor Segmentation From Multi-Modality MR Images[J]. Ieee Transactions on Image Processing, 2020, 29: 9032-9043.

[6] Shuai Wang,Zipei Yan,Daoan Zhang, et al. Prototype Knowledge Distillation for Medical Segmentation with Missing Modality[Z], 2023.

[7] Minhao Hu,Matthis Maillard,Ya Zhang, et al. Knowledge distillation from multi-modal to mono-modal segmentation networks[J]. Corr, 2021, abs/2106.09564.

[8] Yoonseok Choi,Mohammed A. Al-masni,Kyu-Jin Jung, et al. A single stage knowledge distillation network for brain tumor segmentation on limited MR image modalities[J]. Computer Methods and Programs in Biomedicine, 2023, 240: 107644.

[9] Mohammad Havaei,Nicolas Guizard,Nicolas Chapados, et al. HeMIS: Hetero-Modal Image Segmentation[Z], 2016.

[10] Jianghao Wu,Dong Guo,Lu Wang, et al. TISS-net: Brain tumor image synthesis and segmentation using cascaded dual-task networks and error-prediction consistency[J]. Neurocomputing, 2023, 544: 126295.

[11] Tongxue Zhou,Su Ruan,Haigen Hu. A literature survey of MR-based brain tumor segmentation with missing modalities[J]. Computerized Medical Imaging and Graphics, 2023, 104: 102167.

[12] Heran Yang,Jian Sun,Zongben Xu. Learning Unified Hyper-Network for Multi-Modal MR Image Synthesis and Tumor Segmentation With Missing Modalities[J]. Ieee Transactions on Medical Imaging, 2023, 42(12): 3678-3689.

[13] Weide Liu,Jingwen Hou,Xiaoyang Zhong, et al. Improving multi-modal brain tumor segmentation via pre-training and knowledge distillation based post-training[J]. Neurocomputing, 2025, 640: 130318.

[14] Junjie Wang,Huanlan Kang,Tao Liu. The Teacher–Assistant–Student Collaborative and Competitive Network for Brain Tumor Segmentation with Missing Modalities[J]. Diagnostics, 2025, 15(12): 1552.

[15] Feng Xiong,Chuyun Shen,Xiangfeng Wang. Generalized Knowledge Distillation for Unimodal Glioma Segmentation from Multimodal Models[J]. Electronics, 2023, 12(7): 1516.

[16] Geoffrey Hinton,Oriol Vinyals,Jeff Dean. Distilling the Knowledge in a Neural Network[Z], 2015.

[17] Dianlong An,Panpan Liu,Yan Feng, et al. Dynamic weighted knowledge distillation for brain tumor segmentation[J]. Pattern Recognition, 2024, 155: 110731.

[18] Yuan Qi,Wenyin Zhang,Xing Wang, et al. Efficient Knowledge Distillation for Brain Tumor Segmentation[J]. Applied Sciences-basel, 2022, 12(23): 11980.

[19] Jiawei Su,Zhiming Luo,Chengji Wang, et al. Reconstruct incomplete relation for incomplete modality brain tumor segmentation[J]. Neural Networks, 2024, 180: 106657.

[20] Jiayi Li,Lei Zhang,Ke Zhong, et al. A discrepancy-aware self-distillation method for multi-modal glioma grading[J]. Knowledge-based Systems, 2024, 295: 111858.

[21] Reuben Dorent,Thomas Booth,Wenqi Li, et al. Learning joint segmentation of tissues and brain lesions from task-specific hetero-modal domain-shifted datasets[J]. Medical Image Analysis, 2021, 67: 101862.

[22] Pengyu Wang,Huaqi Zhang,Meilu Zhu, et al. MGIML: Cancer Grading With Incomplete Radiology-Pathology Data via Memory Learning and Gradient Homogenization[J]. Ieee Transactions on Medical Imaging, 2024, 43(6): 2113-2124.

[23] Hsienchih Ting,Manhua Liu. Multimodal Transformer of Incomplete MRI Data for Brain Tumor Segmentation[J]. Ieee Journal of Biomedical and Health Informatics, 2024, 28(1): 89-99.

[24] Hengyi Yang,Tao Zhou,Yi Zhou, et al. Flexible Fusion Network for Multi-Modal Brain Tumor Segmentation[J]. Ieee Journal of Biomedical and Health Informatics, 2023, 27(7): 3349-3359.

[25] Mobeen Ur Rehman,SeungBin Cho,Jeehong Kim, et al. BrainSeg-Net: Brain Tumor MR Image Segmentation via Enhanced Encoder–Decoder Network[J]. Diagnostics, 2021, 11(2): 169.

[26] Xiaoliang Lei,Xiaosheng Yu,Jianning Chi, et al. Brain tumor segmentation in MR images using a sparse constrained level set algorithm[J]. Expert Systems with Applications, 2021, 168: 114262.

[27] Fethi Ghazouani,Pierre Vera,Su Ruan. Efficient brain tumor segmentation using Swin transformer and enhanced local self-attention[J]. International Journal of Computer Assisted Radiology and Surgery, 2023, 19: 273-281.

[28] V. Nehru,V. Prabhu. Automated Multimodal Brain Tumor Segmentation and Localization in MRI Images Using Hybrid Res2-UNeXt[J]. Journal of Electrical Engineering & Technology, 2024, 19(5): 3485-3497.

[29] Jianwei Lin,Jiatai Lin,Cheng Lu, et al. CKD-TransBTS: Clinical Knowledge-Driven Hybrid Transformer With Modality-Correlated Cross-Attention for Brain Tumor Segmentation[J]. Ieee Transactions on Medical Imaging, 2023, 42(8): 2451-2461.

[30] Annachiara Cariola,Elena Sibilano,Antonio Brunetti, et al. Enhanced Segmentation of Glioma Subregions via Modality-Aware Encoding and Channel-Wise Attention in Multimodal MRI[J]. Applied Sciences-basel, 2025, 15(14): 8061.

[31] Meghana Karri,Chandra Sekhara Rao Annavarapu,U. Rajendra Acharya. Explainable multi-module semantic guided attention based network for medical image segmentation[J]. Computers in Biology and Medicine, 2022, 151: 106231.

[32] Yiming Yao,Peisheng Qian,Ziyuan Zhao, et al. Residual Channel Attention Network for Brain Glioma Segmentation[C]//Annual International Conference of the Ieee Engineering in Medicine and Biology Society. Ieee Engineering in Medicine and Biology Society. Annual International Conference: Ieee, 2022: 2132-2135.

[33] Yu Wang,Juan Xu,Yucheng Guan, et al. MSegNet: A Multi-View Coupled Cross-Modal Attention Model for Enhanced MRI Brain Tumor Segmentation[J]. International Journal of Computational Intelligence Systems, 2025, 18(1).

[34] Yulan Yan,Yinwei Zhan,Huiyao He. Adaptive Fusion and Edge‐Oriented Enhancement for Brain Tumor Segmentation With Missing Modalities[J]. International Journal of Imaging Systems and Technology, 2025, 35(1).

[35] Shoukun Xu,Rui Tang,Jialu Chen, et al. CMIT-Net: a cross-modal information transfer network for multi-modal brain tumor segmentation[J]. Signal Image and Video Processing, 2025, 19(3).

[36] Junjie Shi,Li Yu,Qimin Cheng, et al. MFTrans: Modality-Masked Fusion Transformer for Incomplete Multi-Modality Brain Tumor Segmentation[J]. Ieee Journal of Biomedical and Health Informatics, 2024, 28(1): 379-390.

[37] Zechen Zhao,Heran Yang,Jian Sun. Modality-Adaptive Feature Interaction for Brain Tumor Segmentation with Missing Modalities[C]: Springer Nature Switzerland, 2022: 183-192.

[38] Chenggang Lyu,Hai Shu. A Two-Stage Cascade Model with Variational Autoencoders and Attention Gates for MRI Brain Tumor Segmentation[M]: Springer International Publishing, 2020: 435-447.

[39] Linyu Xing,Mengxi Chen,Jiangchao Yao, et al. Pre-Post Interaction Learning for Brain Tumor Segmentation with Missing MRI Modalities[C]//Icassp 2024 - 2024 Ieee International Conference on Acoustics, Speech and Signal Processing (icassp): Ieee, 2024: 1711-1715.

[40] Satyanarayana Murthy Teki,Mohan Krishna Varma,Anjana K. Yadav. Brain Tumour Segmentation Using U-net Based Adversarial Networks[J]. Traitement Du Signal, 2019, 36(4): 353-359.

[41] Walter H. L. Pinaya,Petru-Daniel Tudosiu,Robert Gray, et al. Unsupervised brain imaging 3D anomaly detection and segmentation with transformers[J]. Medical Image Analysis, 2022, 79: 102475.

[42] Yuhang Ding,Xin Yu,Yi Yang. RFNet: Region-aware Fusion Network for Incomplete Multi-modal Brain Tumor Segmentation[C]//2021 Ieee/cvf International Conference on Computer Vision (iccv): Ieee, 2021: 3955-3964.

[43] Cheng Chen,Qi Dou,Yueming Jin, et al. Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion[Z], 2020.