Multi-Modal Hybrid Breast Cancer Classification Using Efficientnet-B3, Reliability-Weighted Decision Fusion, And Stacking Ensemble

Abstract

Early diagnosis of breast cancer is essential because of its high prevalence rate. This research focuses on enhancing the Multi-Modal Breast Cancer Classification System with the use of mammogram, ultrasound, and histopathology. Improving the accuracy level is the primary goal. For our study, EfficientNet-B3 network architecture was used for feature extraction from mammography and ultrasonography instead of shallow Convolutional Neural Network. Attention mechanism was used to integrate features from different modalities through assigning the learned attention weight. In addition, elastic deformation is employed as an enhancement technique. A stacking ensemble approach with logistic regression as a meta learner was utilized. The proposed model was evaluated using six publicly available datasets, five-fold stratified cross validation, and Synthetic Minority Over-Sampling Technique to address class imbalance issue. The results indicate remarkable performance across various modalities. The accuracy rate of the model when applied on mammography, ultrasonography, and histopathology was 91.23%, 98.16%, and 98.57%, respectively. Furthermore, the fusion of all three resulted in 98.57%. In summary, the proposed framework has shown improvement over the previously developed model, namely the MMHBC-Net, as a result of increased accuracy, improved robustness, and enhanced multi-modal fusion.