An End‐to‐End Explainable Vision‐Language Framework for Breast Cancer Diagnosis From Mammograms

ABSTRACT

The medical imaging field has been transformed by deep learning as it offers unparalleled performance with efficiency in diagnosis and detection of complex medical diseases, including breast cancer. This study introduces a novel end‐to‐end deep learning explainable framework for efficient and accurate diagnosis of breast cancer using mammograms. For breast cancer classification, the EfficientFormer‐V2 model was used to support visual clinical interpretation. Grad‐CAM was employed for heatmap generation, and a transformer‐based architecture for textual report generation. The proposed architecture was extensively evaluated on two datasets: CDD‐CESM and INbreast, achieving accuracies of 93.04% and 96.34%, respectively. To further validate our quantitative results, both visual and textual explanations for each decision were generated as heatmaps and textual reports. The textual explanations were evaluated using conventional NLP metrics such as BLEU, ROUGE, METEOR, and CIDEr, indicating the similarity between the generated reports and their corresponding reference reports across both datasets. The proposed model has achieved BLEU‐1 of 0.58 and 0.42, ROUGE of 0.66 and 0.58, METEOR of 0.62 and 0.52, and CIDEr of 0.58 and 0.52, on CDD‐CESM and INbreast datasets, respectively. This end‐to‐end framework combines breast cancer classification with complementary visual and textual explanations, providing a unified representation of the model's diagnostic output.