Centralized Multi Modal Deep Learning for Breast Cancer Diagnosis: A Physics Aware Approach

Breast cancer remains a leading cause of cancer-related deaths in women, necessitating accurate diagnosis that leverages multiple imaging modalities. This work proposes a centralized, physics-aware deep learning framework that integrates Ultrasound, MRI, and Mammography to improve automated breast cancer diagnosis. Modality-specific preprocessing pipelines are designed to handle the distinct physical degradation characteristics of each imaging type, including speckle reduction for Ultrasound, contrast enhancement for Mammography, and N4 bias field correction plus Otsu thresholding on key 2D slices for MRI. By converting 3D MRI volumes into informative 2D slices through Key Slice Extraction, the method enables efficient use of pre-trained 2D CNNs, avoiding the high computational cost and data demands of 3D models. Transfer learning with ResNet50 for Ultrasound and Mammography and DenseNet121 for MRI is employed within a centralized multi-modal architecture. The proposed models achieve state-of-the-art accuracies of 92.50% (Ultrasound), 90.63% (Mammography), and 92.00% (MRI), demonstrating that a physics-aware, centralized multi-modal approach can substantially enhance diagnostic precision.