(UIN Sunan Kalijaga Yogyakarta, Indonesia)(2) Shofwatul Uyun
(UIN Sunan Kalijaga Yogyakarta, Indonesia)*corresponding author
AbstractThe prevalence of brain tumors has been increasing annually, and headaches, a common initial symptom, represent the most common manifestation. However, there is a paucity of research on effective methods of assessing brain tumors. This study proposes a novel approach by introducing various modality fusion techniques based on their fusion levels, which are then categorized into four groups: single-modal, data-level fusion, feature-level fusion, and multilevel fusion. A total of 51 combinations are designed to evaluate the efficacy of these fusion techniques and modality configurations. The experiments used a BraTS2021, which comprises four magnetic resonance imaging (MRI) sequences (flair, t1, t1ce and t2). Initially, the image was pre-processed, encompassing data selection, conversion, and normalization. Subsequently, it was input into a 13-layer CNN architecture for feature extraction. Classification was facilitated by a soft voting method in ensemble learning, incorporating support vector machine (SVM), k-nearest neighbor (KNN), logistic regression, random forest, and decision tree algorithms. The predictive efficacy of the model was rigorously assessed through a comprehensive suite of metrics, prominently featuring accuracy, AUCROC, AUCPR, Cohen's Kappa, and MCC. The results indicate that multilevel fusion exhibits optimal performance, with an average accuracy of 95.84%, followed by feature-level fusion and data-level fusion, at 95.12% and 94.77%, respectively. The optimal fusion technique was identified as the combination with the FF configuration (1,2),3,4), producing an accuracy of 96.62%. The best-model combination proposed exhibited an accuracy difference of nearly 6% from the baseline model, underscoring the efficacy of the proposed approach. These empirical results establish a robust baseline for future investigations into sophisticated fusion architectures across hierarchical integration levels.
KeywordsBrain tumor; Multilevel fusion; Feature-level fusion; Data-level fusion; Ensemble learning
|
DOIhttps://doi.org/10.26555/ijain.v12i2.1975 |
Article metricsAbstract views : 267 | PDF views : 9 |
Cite |
Full Text Download
|
References
[1] K. D. Miller et al., “Brain and other central nervous system tumor statistics, 2021,” CA. Cancer J. Clin., vol. 71, no. 5, pp. 381–406, Sep. 2021, doi: 10.3322/caac.21693.
[2] R. L. Siegel, K. D. Miller, N. S. Wagle, and A. Jemal, “Cancer statistics, 2023,” CA. Cancer J. Clin., vol. 73, no. 1, pp. 17–48, Jan. 2023, doi: 10.3322/caac.21763.
[3] L. R. Schaff and I. K. Mellinghoff, “Glioblastoma and Other Primary Brain Malignancies in Adults: A Review,” JAMA, vol. 329, no. 7, pp. 574–587, Feb. 2023, doi: 10.1001/jama.2023.0023.
[4] M. Arabahmadi, R. Farahbakhsh, and J. Rezazadeh, “Deep Learning for Smart Healthcare, A Survey on Brain Tumor Detection from Medical Imaging,” Sensors, vol. 22, no. 5, p. 1960, Mar. 2022, doi: 10.3390/s22051960.
[5] B. S. Negi, R. Chauhan, R. Rawat, Y. Chanti, and G. Kaur, “Brain Tumor Detection Using integrated approach of FCM & CNN,” in 2024 IEEE 9th International Conference for Convergence in Technology (I2CT), Apr. 2024, pp. 1–5, doi: 10.1109/I2CT61223.2024.10543848.
[6] B. Abhisheka, S. K. Biswas, B. Purkayastha, D. Das, and A. Escargueil, “Recent trend in medical imaging modalities and their applications in disease diagnosis: a review,” Multimed. Tools Appl., vol. 83, no. 14, pp. 43035–43070, Oct. 2023, doi: 10.1007/s11042-023-17326-1.
[7] M. C. Florkow, K. Willemsen, V. V Mascarenhas, E. H. G. Oei, M. van Stralen, and P. R. Seevinck, “Magnetic resonance imaging versus computed tomography for Three‐Dimensional bone imaging of musculoskeletal pathologies: a review,” J. Magn. Reson. Imaging, vol. 56, no. 1, pp. 11–34, 2022, doi: 10.1002/jmri.28067.
[8] M. C. Florkow, K. Willemsen, V. V. Mascarenhas, E. H. G. Oei, M. van Stralen, and P. R. Seevinck, “Magnetic Resonance Imaging Versus Computed Tomography for Three‐Dimensional Bone Imaging of Musculoskeletal Pathologies: A Review,” J. Magn. Reson. Imaging, vol. 56, no. 1, pp. 11–34, Jul. 2022, doi: 10.1002/jmri.28067.
[9] A. M. Gab Allah, A. M. Sarhan, and N. M. Elshennawy, “Edge U-Net: Brain tumor segmentation using MRI based on deep U-Net model with boundary information,” Expert Syst. Appl., vol. 213, p. 118833, Mar. 2023, doi: 10.1016/j.eswa.2022.118833.
[10] H. Sadr, M. Nazari, S. Yousefzadeh-Chabok, H. Emami, R. Rabiei, and A. Ashraf, “Enhancing brain tumor classification in MRI images: A deep learning-based approach for accurate diagnosis,” Image Vis. Comput., vol. 159, p. 105555, Jun. 2025, doi: 10.1016/j.imavis.2025.105555.
[11] S. Labus et al., “A concurrent, deep learning–based computer-aided detection system for prostate multiparametric MRI: a performance study involving experienced and less-experienced radiologists,” Eur. Radiol., vol. 33, no. 1, pp. 64–76, Jul. 2022, doi: 10.1007/s00330-022-08978-y.
[12] M. S. I. Khan et al., “Accurate brain tumor detection using deep convolutional neural network,” Comput. Struct. Biotechnol. J., vol. 20, pp. 4733–4745, Jan. 2022, doi: 10.1016/j.csbj.2022.08.039.
[13] P. Xue et al., “Unassisted Clinicians Versus Deep Learning–Assisted Clinicians in Image-Based Cancer Diagnostics: Systematic Review With Meta-analysis,” J. Med. Internet Res., vol. 25, p. e43832, Mar. 2023, doi: 10.2196/43832.
[14] S. R. Stahlschmidt, B. Ulfenborg, and J. Synnergren, “Multimodal deep learning for biomedical data fusion: a review,” Brief. Bioinform., vol. 23, no. 2, p. bbab569, Mar. 2022, doi: 10.1093/bib/bbab569.
[15] X. Xu et al., “A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis,” Bioengineering, vol. 11, no. 3, p. 219, Feb. 2024, doi: 10.3390/bioengineering11030219.
[16] S. Lee et al., “Ensemble learning-based radiomics with multi-sequence magnetic resonance imaging for benign and malignant soft tissue tumor differentiation,” PLoS One, vol. 18, no. 5, p. e0286417, May 2023, doi: 10.1371/journal.pone.0286417.
[17] E. Mahmoud et al., “MU-Glioma Post: A comprehensive dataset of automated MR multi-sequence segmentation and clinical features,” Sci. Data, vol. 12, no. 1, p. 1847, Nov. 2025, doi: 10.1038/s41597-025-06011-7.
[18] E. C. Yilmaz et al., “Evaluating deep learning and radiologist performance in volumetric prostate cancer analysis with biparametric MRI and histopathologically mapped slides,” Abdom. Radiol., vol. 50, no. 6, pp. 2732–2744, Dec. 2024, doi: 10.1007/s00261-024-04734-6.
[19] A. Hekmat, Z. Zhang, S. Ur Rehman Khan, I. Shad, and O. Bilal, “An attention-fused architecture for brain tumor diagnosis,” Biomed. Signal Process. Control, vol. 101, p. 107221, 2025, doi: 10.1016/j.bspc.2024.107221.
[20] N. F. Aurna, M. A. Yousuf, K. A. Taher, A. K. M. Azad, and M. A. Moni, “A classification of MRI brain tumor based on two stage feature level ensemble of deep CNN models,” Comput. Biol. Med., vol. 146, p. 105539, 2022, doi: 10.1016/j.compbiomed.2022.105539.
[21] A. Srinivas and J. P. Mosiganti, “A brain stroke detection model using soft voting based ensemble machine learning classifier,” Meas. Sensors, vol. 29, p. 100871, 2023, doi: 10.1016/j.measen.2023.100871.
[22] S. ‘Uyun, L. Choridah, S. Riyadi, and A. U. Ramadhan, “Convolutional Layer-Based Feature Extraction in an Ensemble Machine Learning Model for Breast Cancer Classification,” International Journal of Advanced Computer Science and Applications, vol. 15, no. 12. pp. 418–425, 2024, doi: 10.14569/IJACSA.2024.0151244.
[23] Asmita and P. Mittal, “Brain Tumor Radiogenomic Classification using multilevel contrastive information fusion,” Procedia Comput. Sci., vol. 259, pp. 260–268, 2025, doi: 10.1016/j.procs.2025.03.327.
[24] H. Liu, J. Huang, Q. Li, X. Guan, and M. Tseng, “A deep convolutional neural network for the automatic segmentation of glioblastoma brain tumor: Joint spatial pyramid module and attention mechanism network,” Artif. Intell. Med., vol. 148, p. 102776, Feb. 2024, doi: 10.1016/j.artmed.2024.102776.
[25] G. Zheng et al., “An attention-based multi-modal MRI fusion model for major depressive disorder diagnosis,” J. Neural Eng., vol. 20, no. 6, p. 066005, Dec. 2023, doi: 10.1088/1741-2552/ad038c.
[26] E. K. Jadoon, F. G. Khan, S. Shah, A. Khan, and M. ElAffendi, “Deep Learning-Based Multi-Modal Ensemble Classification Approach for Human Breast Cancer Prognosis,” IEEE Access, vol. 11, pp. 85760–85769, 2023, doi: 10.1109/ACCESS.2023.3304242.
[27] M. M. M, M. T. R, V. K. V, and S. Guluwadi, “Enhancing brain tumor detection in MRI images through explainable AI using Grad-CAM with Resnet 50,” BMC Med. Imaging, vol. 24, no. 1, p. 107, May 2024, doi: 10.1186/s12880-024-01292-7.
[28] V. Nehru and V. Prabhu, “Automated Multimodal Brain Tumor Segmentation and Localization in MRI Images Using Hybrid Res2-UNeXt,” J. Electr. Eng. Technol., vol. 19, no. 5, pp. 3485–3497, Jul. 2024, doi: 10.1007/s42835-023-01779-3.
[29] Y. Peng and J. Sun, “The multimodal MRI brain tumor segmentation based on AD-Net,” Biomed. Signal Process. Control, vol. 80, p. 104336, 2023, doi: 10.1016/j.bspc.2022.104336.
[30] M. P. Usha, G. Kannan, and M. Ramamoorthy, “Multimodal Brain Tumor Classification Using Convolutional Tumnet Architecture,” Behav. Neurol., vol. 2024, no. 1, p. 4678554, Jan. 2024, doi: 10.1155/2024/4678554.
[31] M. S. Ullah, M. A. Khan, N. A. Almujally, M. Alhaisoni, T. Akram, and M. Shabaz, “BrainNet: a fusion assisted novel optimal framework of residual blocks and stacked autoencoders for multimodal brain tumor classification,” Sci. Rep., vol. 14, no. 1, p. 5895, Mar. 2024, doi: 10.1038/s41598-024-56657-3.
[32] S. Maqsood, R. Damaševičius, and R. Maskeliūnas, “Multi-Modal Brain Tumor Detection Using Deep Neural Network and Multiclass SVM,” Medicina (B. Aires)., vol. 58, no. 8, p. 1090, Aug. 2022, doi: 10.3390/medicina58081090.
[33] S. Srivastava, M. Arfat, S. Pachar, R. Krishna Yellapragada, and P. Jyotiyana, “Pre-Processing Investigation For Brain Abnormality Detection And Analysis Through MRI of Brain,” in 2023 1st International Conference on Innovations in High Speed Communication and Signal Processing (IHCSP), Mar. 2023, pp. 418–422, doi: 10.1109/IHCSP56702.2023.10127204.
[34] S. De Sutter, J. Wuts, W. Geens, A.-M. Vanbinst, J. Duerinck, and J. Vandemeulebroucke, “Modality redundancy for MRI-based glioblastoma segmentation,” Int. J. Comput. Assist. Radiol. Surg., vol. 19, no. 10, pp. 2101–2109, 2024, doi: 10.1007/s11548-024-03238-4.
[35] B. C. Mohanty, P. K. Subudhi, R. Dash, and B. Mohanty, “Feature-enhanced deep learning technique with soft attention for MRI-based brain tumor classification,” Int. J. Inf. Technol., vol. 16, no. 3, pp. 1617–1626, Mar. 2024, doi: 10.1007/s41870-023-01701-0.
[36] I. Pacal, O. Celik, B. Bayram, and A. Cunha, “Enhancing EfficientNetv2 with global and efficient channel attention mechanisms for accurate MRI-Based brain tumor classification,” Cluster Comput., vol. 27, no. 8, pp. 11187–11212, Nov. 2024, doi: 10.1007/s10586-024-04532-1.
[37] S. Aburass, O. Dorgham, J. Al Shaqsi, M. Abu Rumman, and O. Al-Kadi, “Vision Transformers in Medical Imaging: a Comprehensive Review of Advancements and Applications Across Multiple Diseases,” J. Imaging Informatics Med., vol. 38, no. 6, pp. 3928–3971, Mar. 2025, doi: 10.1007/s10278-025-01481-y.
[38] S. Tabatabaei, K. Rezaee, and M. Zhu, “Attention transformer mechanism and fusion-based deep learning architecture for MRI brain tumor classification system,” Biomed. Signal Process. Control, vol. 86, p. 105119, Sep. 2023, doi: 10.1016/j.bspc.2023.105119.
[39] V. Durairaj and P. Uthirapathy, “Interactive Multi-scale Fusion: Advancing Brain Tumor Detection Through Trans-IMSM Model,” J. Imaging Informatics Med., vol. 38, no. 2, pp. 757–774, Aug. 2024, doi: 10.1007/s10278-024-01222-7.
[40] G. Kong, C. Wu, Z. Zhang, C. Yin, and D. Qin, “M3: using mask-attention and multi-scale for multi-modal brain MRI classification,” Front. Neuroinform., vol. 18, p. 1403732, Jul. 2024, doi: 10.3389/fninf.2024.1403732.
[41] L. Liu and K. Xia, “BTIS-Net: Efficient 3D U-Net for Brain Tumor Image Segmentation,” IEEE Access, vol. 12, pp. 133392–133405, 2024, doi: 10.1109/ACCESS.2024.3460797.
[42] L. M. Pereira, A. Salazar, and L. Vergara, “A Comparative Analysis of Early and Late Fusion for the Multimodal Two-Class Problem,” IEEE Access, vol. 11, pp. 84283–84300, 2023, doi: 10.1109/ACCESS.2023.3296098.
[43] A. He, T. Li, Y. Wu, K. Zou, and H. Fu, “FRCNet: Frequency and Region Consistency for Semi-supervised Medical Image Segmentation,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, 2024, pp. 305–315, doi: 10.1007/978-3-031-72111-3_29.
[44] Y. Jiang et al., “Anatomical Invariance Modeling and Semantic Alignment for Self-supervised Learning in 3D Medical Image Analysis,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Oct. 2023, pp. 15813–15823, doi: 10.1109/ICCV51070.2023.01453.
[45] S. Li, Y. Dai, J. Chen, F. Yan, and Y. Yang, “MRI-based habitat imaging in cancer treatment: current technology, applications, and challenges,” Cancer Imaging, vol. 24, no. 1, p. 107, Aug. 2024, doi: 10.1186/s40644-024-00758-9.
[46] M.-T. Tran, H.-J. Yang, S.-H. Kim, and G.-S. Lee, “Prediction of Survival of Glioblastoma Patients Using Local Spatial Relationships and Global Structure Awareness in FLAIR MRI Brain Images,” IEEE Access, vol. 11, pp. 37437–37449, 2023, doi: 10.1109/ACCESS.2023.3266771.
[47] Y. Su, J. Cheng, C. Zhong, C. Jiang, J. Ye, and J. He, “Accurate polyp segmentation through enhancing feature fusion and boosting boundary performance,” Neurocomputing, vol. 545, p. 126233, Aug. 2023, doi: 10.1016/j.neucom.2023.126233.
[48] Z. Zhu, M. Sun, G. Qi, Y. Li, X. Gao, and Y. Liu, “Sparse Dynamic Volume TransUNet with multi-level edge fusion for brain tumor segmentation,” Comput. Biol. Med., vol. 172, p. 108284, Apr. 2024, doi: 10.1016/j.compbiomed.2024.108284.
[49] Z. Xiong, “ResSAXU-Net for multimodal brain tumor segmentation from brain MRI,” Sci. Rep., vol. 15, no. 1, p. 24179, Jul. 2025, doi: 10.1038/s41598-025-09539-1.
[50] Z. Li, W. Silamu, Y. Ma, and Y. Li, “DualTrans: A Novel Glioma Segmentation Framework Based on a Dual-Path Encoder Network and Multi-View Dynamic Fusion Model,” Appl. Sci., vol. 14, no. 11, p. 4834, Jun. 2024, doi: 10.3390/app14114834.
[51] I. A. Abbasi, M. Alshehri, and Y. AlQahtani, “Tumor-specific PET tracer imaging and contrast-enhanced Mri based tumor volume differences inspection of glioblastoma patients,” Sci. Rep., vol. 15, no. 1, p. 30011, Aug. 2025, doi: 10.1038/s41598-025-15185-4.
[52] P. L. Y. Tang, E. A. H. Warnert, and M. Smits, “Conventional and Advanced MRI in Neuro-Oncology,” in Advanced Imaging and Therapy in Neuro-Oncology, Cham: Springer Nature Switzerland, 2024, pp. 9–30, doi: 10.1007/978-3-031-59341-3_2.
[53] F. Raab, W. Malloni, S. Wein, M. W. Greenlee, and E. W. Lang, “Investigation of an efficient multi-modal convolutional neural network for multiple sclerosis lesion detection,” Sci. Rep., vol. 13, no. 1, p. 21154, Nov. 2023, doi: 10.1038/s41598-023-48578-4.
[54] S. Fanton and W. H. Thompson, “NetPlotBrain : A Python package for visualizing networks and brains,” Netw. Neurosci., vol. 7, no. 2, pp. 461–477, Jun. 2023, doi: 10.1162/netn_a_00313.
[55] L. Huang, J. Qin, Y. Zhou, F. Zhu, L. Liu, and L. Shao, “Normalization Techniques in Training DNNs: Methodology, Analysis and Application,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 8, pp. 10173–10196, Aug. 2023, doi: 10.1109/TPAMI.2023.3250241.
[56] D. Chicco, N. Tötsch, and G. Jurman, “The Matthews correlation coefficient (MCC) is more reliable than balanced accuracy, bookmaker informedness, and markedness in two-class confusion matrix evaluation,” BioData Min., vol. 14, no. 1, p. 13, Feb. 2021, doi: 10.1186/s13040-021-00244-z.
[57] D. Chicco and G. Jurman, “The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification,” BioData Min., vol. 16, no. 1, p. 4, Feb. 2023, doi: 10.1186/s13040-023-00322-4.
[58] D. Chicco, M. J. Warrens, and G. Jurman, “The Matthews Correlation Coefficient (MCC) is More Informative Than Cohen’s Kappa and Brier Score in Binary Classification Assessment,” IEEE Access, vol. 9, pp. 78368–78381, 2021, doi: 10.1109/ACCESS.2021.3084050.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
___________________________________________________________
International Journal of Advances in Intelligent Informatics
ISSN 2442-6571 (print) | 2548-3161 (online)
Organized by UAD and ASCEE Computer Society
Published by Universitas Ahmad Dahlan
W: http://ijain.org
E: info@ijain.org (paper handling issues)
andri.pranolo.id@ieee.org (publication issues)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0