Abstract 15632: Warfarin Dose Management Using Offline Deep Reinforcement Learning

Introduction: Warfarin is a commonly prescribed anticoagulant for treating atrial fibrillation, mechanical valves, and venous thromboembolism. Warfarin dose management remains challenging due to dosing variability between patients and warfarin’s narrow therapeutic window. Time in therapeutic range (TTR) is critical to warfarin’s safety and efficacy, but TTR typically remains low (40-50%) in community practices. Specialized anticoagulation clinics and protocolized approaches can increase TTR but have great administrative burdens and health care costs.

Aim: To develop standardized optimal warfarin dose decision support using a machine learning model based on time series anticoagulation data and patient demographic characteristics

Methods: The dataset included 12,497 warfarin patients monitored in the anticoagulation tracker of electronic medical records across the Mayo Clinic Enterprise. The dataset contained time series anticoagulation data (warfarin dose, INR, INR target range) and patient demographic characteristics. We implemented an offline deep reinforcement learning model (DRL) to predict the cumulative warfarin dose for the days until the next INR test based on a patient’s historical anticoagulation data. Prior approaches utilized traditional supervised learning methods such as regression or Long Short Term Memory (LSTM) neural networks to approximate a function mimicking the behavior of the training data. On the other hand, DRL learned an optimal dosing policy through continuous interaction and feedback from the training data. The key advantage of DRL is the model can learn to behave differently (and potentially better) for suboptimal clinical states in the data such as overdosing or underdosing. To evaluate the DRL model we compared the predicted warfarin doses with the physician-prescribed doses

Results: DRL model’s prediction accuracy was 96.96%, outperforming our implementation of a baseline LSTM model with a prediction accuracy of 70.58%. Further evaluation of the DRL model indicated that the model correctly adjusted the warfarin dose at time steps when patients had out-of-range INRs.

Conclusions: Offline deep reinforcement learning demonstrates potential in supporting warfarin dose management to maximize TTR.