AI-Assisted Risk Prioritization of Surgical Stapler Adverse-Event Reports for Post-Market Surveillance: A Cross-Database Machine-Learning Study

Background: Post-market surveillance databases support medical-device safety management, but heterogeneous reporting structures and narrative styles may limit cross-database prediction. Objective: This study developed and externally validated an AI-assisted framework for prioritizing serious surgical stapler adverse-event reports and tested whether LLM-derived mechanism variables improved target-domain performance. Methods: This retrospective study included 52,250 reports from the FDA Manufacturer and User Facility Device Experience database (MAUDE) and 5,593 reports from Jiangsu, China. Logistic regression, random forest and XGBoost were evaluated using five harmonized shared fields, shared fields plus existing IMDRF code indicators, and shared fields plus 24 predefined narrative-derived mechanism variables. Mechanism models excluded 281 MAUDE reports with empty narratives. Model development used MAUDE data exclusively, with Jiangsu reserved for external validation. Discrimination, calibration and feature attribution were assessed. Results: Shared-field XGBoost achieved an internal ROC AUC of 0.8345 and average precision of 0.7349, which decreased to 0.5089 and 0.4057 in Jiangsu. Adding 24 mechanism variables increased its external ROC AUC to 0.6477 (95% CI, 0.6335–0.6625) and average precision to 0.5633 (95% CI, 0.5463–0.5820). External ROC AUCs for the corresponding logistic regression and random forest models were 0.7070 and 0.6828, respectively. Conclusion: Structured-field models showed substantial performance degradation across databases. Narrative-derived mechanism variables improved external discrimination, supporting further evaluation for retrospective report prioritization. Outcome-related information within these variables requires careful interpretation, and prospective validation is needed before operational use.