Transferring and replicating predictive algorithms across healthcare systems constitutes a unique yet crucial challenge that needs to be addressed to enable the widespread adoption of machine learning in healthcare. In this study, we explored the impact of important differences across healthcare systems and the associated Electronic Health Records (EHRs) on machine-learning algorithms to predict mental health crises, up to 28 days in advance. We evaluated both the transferability and replicability of such machine learning models, and for this purpose, we trained six models using features and methods developed on EHR data from the Birmingham and Solihull Mental Health NHS Foundation Trust in the UK. These machine learning models were then used to predict the mental health crises of 2907 patients seen at the Rush University System for Health in the US between 2018 and 2020. The best one was trained on a combination of US-specific structured features and frequency features from anonymized patient notes and achieved an AUROC of 0.837. A model with comparable performance, originally trained using UK structured data, was transferred and then tuned using US data, achieving an AUROC of 0.826. Our findings establish the feasibility of transferring and replicating machine learning models to predict mental health crises across diverse hospital systems.

在医疗保健系统中传输和复制预测算法构成了一个独特而关键的挑战，需要解决这一挑战，以实现机器学习在医疗保健领域的广泛采用。在这项研究中，我们探讨了医疗保健系统和相关电子健康记录 (EHR) 之间的重要差异对机器学习算法的影响，以提前 28 天预测心理健康危机。我们评估了此类机器学习模型的可转移性和可复制性，为此，我们使用根据英国伯明翰和索利哈尔心理健康 NHS 基金会信托基金的 EHR 数据开发的功能和方法训练了 6 个模型。然后，这些机器学习模型被用来预测 2018 年至 2020 年间在美国拉什大学健康系统就诊的 2907 名患者的心理健康危机。最好的模型接受了美国特有的结构特征和频率特征的组合训练。匿名患者笔记并实现 AUROC 为 0.837。最初使用英国结构化数据训练的性能相当的模型被转移，然后使用美国数据进行调整，实现了 0.826 的 AUROC。我们的研究结果确立了转移和复制机器学习模型以预测不同医院系统中心理健康危机的可行性。