942 Comparing Large Language Models in Predicting Multi-MDisciplinary Team Meeting Outcomes
T Badenoch, C Oliver-Blaney, E MasseyAbstract
Aim
Multidisciplinary team meetings (MDT) are an essential part of modern cancer care, involving multiple specialists discussing patient diagnosis and management. Increased workloads and complexity of patients and treatment put pressure on these meetings, worsened by other clinical commitment pressures. Large language models (LLMs) can understand large amounts of information and respond to specific queries and prompts and are beginning to be used as adjuncts to clinical care. We aim to assess the difference between two LLMS, Claude Opus 3 and Gemini, in predicting MDT outcomes.
Method
A prospective diagnostic concordance and validation study, assessing the LLM’s ability to interpret clinical information and provide guideline-based management recommendations. We will provide the LLMs with the same clinical information that will be presented at MDT and prompt them to create treatment plans based Association of Breast Surgery and National Institute of Health and Care Excellence guidelines. The treatments plans will be reviewed and given a concordance score against the actual MDT outcome,
Results
Initial results demonstrate that both LLMs produce treatment plans with structure and justification according to the guidelines. We would predict that there is good concordance with the MDT outcome, but that Claude is able to provide more reasoning around further decision making.
Conclusions
We predict that Claude Opus 3, being a more advanced LLM that is trained with more professional data including medical data, would have better concordance with actual MDT outcomes. WE predict it will also be able to understand more complex prompts including around the need for further investigation and imaging.