In recent years, there has been a growing interest in the prediction of individualized treatment effects. While there is a rapidly growing literature on the development of such models, there is little literature on the evaluation of their performance. In this paper, we aim to facilitate the validation of prediction models for individualized treatment effects. The estimands of interest are defined based on the potential outcomes framework, which facilitates a comparison of existing and novel measures. In particular, we examine existing measures of discrimination for benefit (variations of the c‐for‐benefit), and propose model‐based extensions to the treatment effect setting for discrimination and calibration metrics that have a strong basis in outcome risk prediction. The main focus is on randomized trial data with binary endpoints and on models that provide individualized treatment effect predictions and potential outcome predictions. We use simulated data to provide insight into the characteristics of the examined discrimination and calibration statistics under consideration, and further illustrate all methods in a trial of acute ischemic stroke treatment. The results show that the proposed model‐based statistics had the best characteristics in terms of bias and accuracy. While resampling methods adjusted for the optimism of performance estimates in the development data, they had a high variance across replications that limited their accuracy. Therefore, individualized treatment effect models are best validated in independent data. To aid implementation, a software implementation of the proposed methods was made available in R.

中文翻译：

---

评估个体化治疗效果预测：基于模型的歧视和校准评估视角


近年来，人们对个体化治疗效果的预测越来越感兴趣。尽管有关此类模型开发的文献数量迅速增加，但对其性能评估的文献却很少。在本文中，我们的目标是促进个体化治疗效果预测模型的验证。兴趣估计值是根据潜在结果框架定义的，这有助于对现有措施和新措施进行比较。特别是，我们检查了现有的福利歧视衡量标准（c-for-benefit 的变体），并提出了基于模型的歧视治疗效果设置的扩展和在结果风险预测中具有坚实基础的校准指标。主要重点是具有二元终点的随机试验数据以及提供个体化治疗效果预测和潜在结果预测的模型。我们使用模拟数据来深入了解所考虑的检查歧视和校准统计数据的特征，并进一步说明急性缺血性中风治疗试验中的所有方法。结果表明，所提出的基于模型的统计在偏差和准确性方面具有最佳特征。虽然重采样方法根据开发数据中性能估计的乐观情况进行了调整，但它们在重复过程中存在很大差异，限制了其准确性。因此，个体化治疗效果模型最好在独立数据中得到验证。为了帮助实施，所提出方法的软件实施可在右。