Conformal prediction has seen many applications in pharmaceutical science, being able to calibrate outputs of machine learning models and producing valid prediction intervals. We here present the open source software CPSign that is a complete implementation of conformal prediction for cheminformatics modeling. CPSign implements inductive and transductive conformal prediction for classification and regression, and probabilistic prediction with the Venn-ABERS methodology. The main chemical representation is signatures but other types of descriptors are also supported. The main modeling methodology is support vector machines (SVMs), but additional modeling methods are supported via an extension mechanism, e.g. DeepLearning4J models. We also describe features for visualizing results from conformal models including calibration and efficiency plots, as well as features to publish predictive models as REST services. We compare CPSign against other common cheminformatics modeling approaches including random forest, and a directed message-passing neural network. The results show that CPSign produces robust predictive performance with comparative predictive efficiency, with superior runtime and lower hardware requirements compared to neural network based models. CPSign has been used in several studies and is in production-use in multiple organizations. The ability to work directly with chemical input files, perform descriptor calculation and modeling with SVM in the conformal prediction framework, with a single software package having a low footprint and fast execution time makes CPSign a convenient and yet flexible package for training, deploying, and predicting on chemical data. CPSign can be downloaded from GitHub at https://github.com/arosbio/cpsign . Scientific contribution CPSign provides a single software that allows users to perform data preprocessing, modeling and make predictions directly on chemical structures, using conformal and probabilistic prediction. Building and evaluating new models can be achieved at a high abstraction level, without sacrificing flexibility and predictive performance—showcased with a method evaluation against contemporary modeling approaches, where CPSign performs on par with a state-of-the-art deep learning based model.

中文翻译：

---

CPSign：化学信息学建模的保形预测


保形预测在制药科学中得到了许多应用，能够校准机器学习模型的输出并产生有效的预测区间。我们在此介绍开源软件 CPSign，它是化学信息学建模的共形预测的完整实现。 CPSign 使用 Venn-ABERS 方法实现分类和回归的归纳和转导共形预测以及概率预测。主要的化学表示是签名，但也支持其他类型的描述符。主要建模方法是支持向量机 (SVM)，但通过扩展机制支持其他建模方法，例如 DeepLearning4J 模型。我们还描述了用于可视化共形模型结果的功能，包括校准和效率图，以及将预测模型发布为 REST 服务的功能。我们将 CPSign 与其他常见的化学信息学建模方法（包括随机森林和定向消息传递神经网络）进行比较。结果表明，与基于神经网络的模型相比，CPSign 具有稳健的预测性能和相对预测效率，具有卓越的运行时间和更低的硬件要求。 CPSign 已用于多项研究，并在多个组织中投入生产使用。能够直接处理化学输入文件，在共形预测框架中使用 SVM 执行描述符计算和建模，单个软件包占用空间小且执行时间快，这使得 CPSign 成为一个方便而灵活的软件包，用于培训、部署和部署。预测化学数据。 CPSign 可以从 GitHub 下载：https://github.com/arosbio/cpsign。 科学贡献 CPSign 提供了一个单一软件，允许用户使用保角和概率预测直接对化学结构进行数据预处理、建模和预测。可以在高抽象级别上实现构建和评估新模型，而无需牺牲灵活性和预测性能——通过针对当代建模方法的方法评估进行展示，其中 CPSign 的性能与最先进的基于深度学习的模型相当。