Improvement of CNN model using integration of multimodal MRI sequences and multilevel fusion to enhance performance for brain tumor classification

Abstract

The prevalence of brain tumors has been increasing annually, and headaches, a common initial symptom, represent the most common manifestation. However, there is a paucity of research on effective methods of assessing brain tumors. This study proposes a novel approach by introducing various modality fusion techniques based on their fusion levels, which are then categorized into four groups: single-modal, data-level fusion, feature-level fusion, and multilevel fusion. A total of 51 combinations are designed to evaluate the efficacy of these fusion techniques and modality configurations. The experiments used a BraTS2021, which comprises four magnetic resonance imaging (MRI) sequences (flair, t1, t1ce and t2). Initially, the image was pre-processed, encompassing data selection, conversion, and normalization. Subsequently, it was input into a 13-layer CNN architecture for feature extraction. Classification was facilitated by a soft voting method in ensemble learning, incorporating support vector machine (SVM), k-nearest neighbor (KNN), logistic regression, random forest, and decision tree algorithms. The predictive efficacy of the model was rigorously assessed through a comprehensive suite of metrics, prominently featuring accuracy, AUCROC, AUCPR, Cohen's Kappa, and MCC. The results indicate that multilevel fusion exhibits optimal performance, with an average accuracy of 95.84%, followed by feature-level fusion and data-level fusion, at 95.12% and 94.77%, respectively. The optimal fusion technique was identified as the combination with the FF configuration (1,2),3,4), producing an accuracy of 96.62%. The best-model combination proposed exhibited an accuracy difference of nearly 6% from the baseline model, underscoring the efficacy of the proposed approach. These empirical results establish a robust baseline for future investigations into sophisticated fusion architectures across hierarchical integration levels.