Nature Medicine ( IF 58.7 ) Pub Date : 2024-10-18 , DOI: 10.1038/s41591-024-03310-1 Yilin Ning, Xiaoxuan Liu, Gary S. Collins, Karel G. M. Moons, Melissa McCradden, Daniel Shu Wei Ting, Jasmine Chiat Ling Ong, Benjamin Alan Goldstein, Siegfried K. Wagner, Pearse A. Keane, Eric J. Topol, Nan Liu
The deployment of artificial intelligence (AI)-powered prediction models in healthcare can lead to ethical concerns about their implementation and upscaling. For example, AI prediction models can hinder clinical decision-making if they advise different diagnoses or treatments by sex and gender or by race and ethnicity without clear justification. Recent guidance (such as the WHO guidance on ethics and governance of AI for health and the Dutch guideline on AI for healthcare) and legislation (such as the European Union AI Act and the White House Executive Order on Safe, Secure, and Trustworthy Development and Use of AI in United States) have outlined important principles for the implementation of AI, including ethical considerations1,2. Health systems have responded by establishing governance committees and processes to ensure the safe and equitable implementation of AI tools3. However, there is currently no assessment tool that can identify and mitigate ethical issues during the implementation of AI prediction models in healthcare practice, including for public health.
The development and validation of AI prediction models has benefited from detailed reporting and risk-of-bias tools, such as TRIPOD+AI4 and PROBAST (with its forthcoming AI extension) for fairness and bias control and CLAIM5 for data privacy, security and interpretability of AI imaging studies. However, when planning the implementation of a rigorously developed and well-performing AI prediction model in healthcare practice, existing recommendations and guidance on ethics are sparse and lack operational detail. For example, the DECIDE-AI reporting guideline6 contains a small number of ethics-related recommendations for early clinical evaluation of AI concerning equity, safety and human-AI interaction, and FUTURE-AI7 provides recommendations based on six principles (fairness, universality, traceability, usability, robustness and explainability) in model design, development, validation and deployment. A bioethics-centric delivery science toolkit for responsible AI implementation in healthcare is needed8.
中文翻译:
人工智能在医疗保健领域实施的伦理评估工具:CARE-AI
在医疗保健领域部署人工智能 (AI) 驱动的预测模型可能会导致对其实施和扩大规模的道德担忧。例如,如果 AI 预测模型在没有明确理由的情况下按性别和性别或种族和民族建议不同的诊断或治疗,则可能会阻碍临床决策。最近的指南(例如 WHO 关于健康 AI 伦理和治理的指南和荷兰医疗保健 AI 指南)和立法(例如欧盟 AI 法案和白宫关于在美国安全、可靠和可信地开发和使用 AI 的行政命令)概述了实施 AI 的重要原则。 包括道德考虑1,2.卫生系统通过建立治理委员会和流程来应对,以确保安全和公平地实施人工智能工具3.然而,目前还没有评估工具可以识别和减轻在医疗保健实践(包括公共卫生)中实施 AI 预测模型期间的道德问题。
AI 预测模型的开发和验证受益于详细的报告和偏差风险工具,例如用于公平性和偏差控制的 TRIPOD+AI4 和 PROBAST(及其即将推出的 AI 扩展),以及用于 AI 成像研究的数据隐私、安全性和可解释性的 CLAIM5。然而,在规划在医疗保健实践中实施经过严格开发且性能良好的 AI 预测模型时,现有的道德建议和指导很少,并且缺乏操作细节。例如,DECIDE-AI 报告指南6 包含少量与伦理相关的建议,用于 AI 的早期临床评估,涉及公平性、安全性和人机交互,FUTURE-AI7 在模型设计、开发、验证和部署方面基于六项原则(公平性、普遍性、可追溯性、可用性、稳健性和可解释性)提供建议。需要一个以生物伦理学为中心的交付科学工具包,用于在医疗保健领域负责任地实施AI 8.