A Two-Stage NLP Pipeline for Automated Surveillance of Surgical Site Infections
C Ghisolfi, C Barbati, L Viviani, F Pescol, L Prisciano, L Sacchi, L Gentile, R Bellazzi, A OdoneAbstract
Background
SSIs are among the leading causes of post-operative complications, with significant impact on patient health, length of stay, and healthcare costs. Surveillance still relies on manual record review, a time-consuming and resource-intensive process. The PRESID-IA project aims to develop a predictive system to estimate individual SSI risk in the immediate post-operative period, enabling early intervention.
Methods
As supervised model training requires labelling of surgical cases according to infectious outcome, NLP techniques were applied to post-operative outpatient reports to automate this process. Several LLMs were tested using prompt engineering to extract SSI clinical criteria by the ECDC protocol for SSI surveillance and classify reports via rule-based logic applied to the extracted criteria. Preliminary results were validated by clinical specialists on 100 reports selected from 15,778 available records.
Results
Compared to a direct approach in which the model classifies the case from the report and criteria, the two-stage pipeline (precision=0.50, recall=0.71, F1=0.58) substantially outperformed direct classification (precision=0.07, recall=1.00, F1=0.13). Clinical validation revealed agreement between model outputs and specialist judgement. Analysis of discordance highlighted that manual review is subject to errors, underscoring the limitations of traditional surveillance.
Conclusions
The proposed approach demonstrates how LLM-based NLP pipelines can automate SSI surveillance, improving efficiency and transparency. Future work will focus on developing the predictive component to estimate individual SSI risk and support clinical decision-making.
Key messages
• NLP-based automation can support SSI surveillance, reducing the burden of manual record review.
• A two-stage pipeline combining structured extraction and rule-based classification outperforms direct LLM-based classification for automated SSI case identification.