TY - JOUR
T1 - Multi-teacher cross-modal distillation with cooperative deep supervision fusion learning for unimodal segmentation
AU - Ahmad, Saeed
AU - Ullah, Zahid
AU - Gwak, Jeonghwan
N1 - Publisher Copyright:
© 2024 Elsevier B.V.
PY - 2024/8/3
Y1 - 2024/8/3
N2 - Accurate brain tumor segmentation is a labor-intensive and time-consuming task that requires automation to enhance its efficacy. Recent advanced techniques have shown promising results in segmenting brain tumors; however, their dependency on extensive multimodal magnetic resonance imaging (MRI) data limits their practicality in clinical environments where such data may not be readily available. To address this, we propose a novel multi-teacher cross-modal knowledge distillation framework, which utilizes the privileged multimodal data during training while relying solely on unimodal data for inference. Our framework is tailored to the unimodal segmentation of the T1ce MRI sequence, which is prevalently available in clinical practice and structurally akin to the T1 modality, providing ample information for the segmentation task. Our framework introduces two learning strategies for knowledge distillation (KD): (1) performance-aware response-based KD and (2) cooperative deep supervision fusion learning (CDSFL). The first strategy involves dynamically assigning confidence weights to each teacher model based on its performance, ensuring that the KD is performance-driven, and the CDSFL module augments the learning capabilities of the multi-teacher models by fostering mutual learning. Moreover, the fused information is distilled into the student model to improve its deep supervision. Extensive experiments on BraTS datasets demonstrate that our framework achieves promising unimodal segmentation results on the T1ce and T1 modalities and outperforms previous state-of-the-art methods. Code is available at https://github.com/ami-lab-knut/mtcm_kd.
AB - Accurate brain tumor segmentation is a labor-intensive and time-consuming task that requires automation to enhance its efficacy. Recent advanced techniques have shown promising results in segmenting brain tumors; however, their dependency on extensive multimodal magnetic resonance imaging (MRI) data limits their practicality in clinical environments where such data may not be readily available. To address this, we propose a novel multi-teacher cross-modal knowledge distillation framework, which utilizes the privileged multimodal data during training while relying solely on unimodal data for inference. Our framework is tailored to the unimodal segmentation of the T1ce MRI sequence, which is prevalently available in clinical practice and structurally akin to the T1 modality, providing ample information for the segmentation task. Our framework introduces two learning strategies for knowledge distillation (KD): (1) performance-aware response-based KD and (2) cooperative deep supervision fusion learning (CDSFL). The first strategy involves dynamically assigning confidence weights to each teacher model based on its performance, ensuring that the KD is performance-driven, and the CDSFL module augments the learning capabilities of the multi-teacher models by fostering mutual learning. Moreover, the fused information is distilled into the student model to improve its deep supervision. Extensive experiments on BraTS datasets demonstrate that our framework achieves promising unimodal segmentation results on the T1ce and T1 modalities and outperforms previous state-of-the-art methods. Code is available at https://github.com/ami-lab-knut/mtcm_kd.
KW - Brain tumor segmentation
KW - Cooperative learning
KW - Feature fusion
KW - Knowledge distillation
KW - Multi-teacher framework
UR - http://www.scopus.com/inward/record.url?scp=85194147990&partnerID=8YFLogxK
U2 - 10.1016/j.knosys.2024.111854
DO - 10.1016/j.knosys.2024.111854
M3 - Article
AN - SCOPUS:85194147990
SN - 0950-7051
VL - 297
JO - Knowledge-Based Systems
JF - Knowledge-Based Systems
M1 - 111854
ER -