The Ethical Dimension between Innovation and Responsibility - Artifical Intelligence in Medicine and Digital Health

Case presentation: Patient E. suffers severe iatrogenic injury during a sinuplasty due to an AI-assisted navigation system incorrectly mapping the position of the instruments to the paranasal sinus region during surgery, even though the instruments were actually located in the immediate vicinity of the carotid artery. Uncritical reliance of the surgeon on the output of an AI-assisted system lead to a carotid artery lesion and a subsequent ischemic stroke with long-term neurological deficits. This raises ethical and legal questions regarding medical responsibility, the duty to inform, and the allocation of liability. Methodology: Using the described clinical case study as a point of departure, the ethical and legal implications of AI-assisted medical decision-making within the frameworks of biomedical ethics and health regulations and laws in the EU and US were critically qualitatively analyzed. Particular attention was given to questions of autonomy, beneficence, professional responsibility, informed consent, and liability, with the aim of assessing the requirements for the responsible and ethically justifiable use of AI in healthcare. Results: Ethically, the focus is particularly on autonomy, non-maleficence, beneficence, and justice. A potential violation of autonomy occurs when patients are not informed about the use of AI and its risks. The principles of non-maleficence and beneficence are violated when medical plausibility checks are omitted and an “automation bias” leads to blind trust in the AI-system. Legally, AI remains a medical device and is classified as a high-risk system under the EU AI Act, thereby subject to requirements regarding safety, robustness, and human oversight. The physician’s duty of care (“lege artis”) is not diminished by AI but continues to require critical oversight. Additionally, liability issues arise that affect physicians, healthcare facilities, and manufacturers alike. Conclusion: AI can be medically beneficial and improve patient safety when used responsibly. However, the present case demonstrates that a lack of human oversight, as well as insufficient reflection and control can lead to serious harm. It is therefore essential to provide transparent patient information, establish clear oversight mechanisms, offer training, and clearly define legal accountability.