Locally deployed large language model for real‐world lecanemab eligibility pre‐screening
Carolin Miklitz, Maya Shrestha, Wegner Philipp, Nils Henk, Alois Martin Sprinkart, Wolfgang Block, Julian Alexander Luetkens, Alexander Radbruch, Nils Christian Lehnen, Sebastian NowakAbstract
INTRODUCTION
The introduction of disease‐modifying Alzheimer's therapies requires complex, labor‐intensive patient screening. Cloud‐based large language models (LLMs) could support this task but are often unsuitable for routine care due to data protection constraints.
METHODS
We evaluated an on‐premises, open‐weights LLM (gpt‐oss‐120b) for automated extraction of therapy‐relevant variables for lecanemab eligibility from German memory clinic reports. In a two‐stage design, LLM‐based extraction prompts, a deterministic rule‐based extractor, and a shared downstream rule‐based classifier were optimized on a development set ( n = 97) and evaluated on an independent hold‐out set ( n = 99), with expert consensus as ground truth.
RESULTS
The LLM‐based pipeline achieved 94% accuracy and a Cohen's kappa of 0.90 on the hold‐out set, significantly surpassing the rule‐based comparator (80% accuracy) and demonstrating performance comparable to human experts.
DISCUSSION
A locally deployed, on‐premises LLM may assist eligibility screening as a triage support tool, potentially facilitating access to novel therapies without compromising patient data privacy.