Explainable hybrid vision transformer-convolutional neural network with cross-attention fusion for multi-institutional breast cancer receptor status prediction from H&E histopathology images

The integration of artificial intelligence (AI) into digital pathology has revolutionised breast cancer diagnostics, offering transformative opportunities to automate and refine molecular profiling, which is essential to precision oncology. Accurate determination of receptor status—estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2)—remains pivotal in guiding personalised treatment planning and prognostic assessment. However, traditional immunohistochemistry (IHC) and molecular assays are limited by resource intensity, lengthy turnaround times, and inter-observer variability, underscoring the need for scalable computational alternatives. This study proposes a novel integrative AI-driven framework for predicting breast cancer receptor profiles directly from hematoxylin and eosin (H&E)-stained histopathology images. The framework employs a hybrid vision transformer-convolutional neural network (ViT-CNN) architecture with a cross-attention fusion mechanism to capture both global contextual information and fine-grained local features. Experiments show AUCs of 0.967, 0.952, and 0.941 for ER, PR, and HER2 prediction, respectively, representing relative improvements of 12.4%, 9.8%, and 10.8% over the best-performing prior method (DCS-ST), re-evaluated under our identical protocol (absolute AUC differences of 0.107, 0.085 and 0.092). Furthermore, the framework enables automated molecular subtyping into luminal A, luminal B, HER2-enriched, and triple-negative categories with a macro-averaged F1-score of 95.7%. A comprehensive multi-institutional validation was performed on 2,847 external whole-slide images from three partner institutions (not including the primary BINO Hospital test set), showing consistent performance (a mean AUC reduction of 0.83 percentage points relative to the internal test set) across the staining protocols and scanners represented. Integrated explainability modules, including attention visualisations and Grad-CAM, enhance interpretability, supporting pathologist review. In a single-institution retrospective workflow audit, AI assistance was associated with an estimated 72.7% reduction in diagnostic turnaround time (from a conventional mean of 72.4 hours to 19.8 hours; Section 6.10, where approximately 65% of the AI-assisted time is un-itemised queueing and logistics) and, in a 120-WSI reader study, with higher inter-observer agreement (kappa 0.847 to 0.967). These clinical-impact estimates are hypothesis-generating and require prospective validation. The findings position the framework as a promising, explainable approach to AI-assisted digital pathology that warrants prospective, multi-centre clinical evaluation.