Benchmarking Open-Source Pathology Foundation Models for Breast Cancer Biomarker Prediction from H&E Whole-Slide Images

Background/Objectives: Breast cancer biomarker detection through immunohistochemistry (IHC) is essential for treatment planning but faces challenges including turnaround time, variability, and laboratory resource constraints. Large open-source vision-language foundation models offer a potential avenue for inferring biomarker status directly from hematoxylin-and-eosin (H&E)-stained whole-slide images (WSIs). Methods: We evaluated two open-source pathology foundation models—TITAN (Transformer-based Pathology Image and Text Alignment Network, approximately 48.5 M parameters) and CHIEF (Clinical Histopathology Imaging Evaluation Foundation Model, approximately 1.2 M parameters)—for predicting estrogen receptor (ER), progesterone receptor (PR), and human epidermal growth factor receptor 2 (HER2) status from H&E-stained breast cancer WSIs. WSI data were obtained from The Cancer Genome Atlas Breast Invasive Carcinoma collection (TCGA-BRCA) via the NCI Imaging Data Commons, with biomarker labels from the NCI Genomic Data Commons. In total, 937 cases (995 WSIs; 78.3% ER-positive) were evaluated for ER, 934 cases (992 WSIs; 68.4% PR-positive) for PR, and 646 cases (691 WSIs; 21.1% HER2-positive) for HER2. All evaluation was performed under a strict patient-level 50/25/25 split with 10 independent random partitions; metrics are reported as the mean across partitions with percentile-based 95% confidence intervals. Performance was assessed using area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), sensitivity, specificity, and positive predictive value (PPV). Results: TITAN and CHIEF achieved comparable performance for ER (TITAN AUROC: 0.885 [95% CI: 0.848, 0.921], AUPRC: 0.954 [0.940, 0.964]; CHIEF AUROC: 0.877 [0.831, 0.914], AUPRC: 0.955 [0.938, 0.969]) and PR (TITAN AUROC: 0.799, AUPRC: 0.868; CHIEF AUROC: 0.791, AUPRC: 0.864). At the default 0.5 operating point, ER PPV was 0.90 and PR PPV was 0.79–0.81. For HER2, both models achieved AUROC values of 0.71–0.74 and AUPRC values of 0.41–0.45—well above the prevalence-based random baseline (approximately 0.211)—but default-threshold sensitivity was very low (approximately 0.07–0.08), reflecting class imbalance and the use of an uncalibrated default threshold rather than a categorical absence of morphologic signal. Conclusions: Under retrospective evaluation, both models demonstrate strong discriminative performance for ER and moderate performance for PR; HER2 prediction at the default operating point is limited and motivates threshold-calibration and multimodal extensions before any clinical use. AUPRC summarizes precision−recall behavior across thresholds and is distinct from threshold-specific precision (PPV); the two should be reported together for clinical-utility assessment in pathology AI. The findings are hypothesis-generating and motivate prospective external validation across independent institutional cohorts before any clinical deployment is considered.