The accurate identification of protein–ligand binding sites is of critical importance in understanding and modulating protein function. Accordingly, ligand binding site prediction has remained a research focus for over three decades with over 50 methods developed and a change of paradigm from geometry-based to machine learning. In this work, we collate 13 ligand binding site predictors, spanning 30 years, focusing on the latest machine learning-based methods such as VN-EGNN, IF-SitePred, GrASP, PUResNet, and DeepPocket and compare them to the established P2Rank, PRANK and fpocket and earlier methods like PocketFinder, Ligsite and Surfnet. We benchmark the methods against the human subset of our new curated reference dataset, LIGYSIS. LIGYSIS is a comprehensive protein–ligand complex dataset comprising 30,000 proteins with bound ligands which aggregates biologically relevant unique protein–ligand interfaces across biological units of multiple structures from the same protein. LIGYSIS is an improvement for testing methods over earlier datasets like sc-PDB, PDBbind, binding MOAD, COACH420 and HOLO4K which either include 1:1 protein–ligand complexes or consider asymmetric units. Re-scoring of fpocket predictions by PRANK and DeepPocket display the highest recall (60%) whilst IF-SitePred presents the lowest recall (39%). We demonstrate the detrimental effect that redundant prediction of binding sites has on performance as well as the beneficial impact of stronger pocket scoring schemes, with improvements up to 14% in recall (IF-SitePred) and 30% in precision (Surfnet). Finally, we propose top-N+2 recall as the universal benchmark metric for ligand binding site prediction and urge authors to share not only the source code of their methods, but also of their benchmark. Scientific contributions This study conducts the largest benchmark of ligand binding site prediction methods to date, comparing 13 original methods and 15 variants using 10 informative metrics. The LIGYSIS dataset is introduced, which aggregates biologically relevant protein–ligand interfaces across multiple structures of the same protein. The study highlights the detrimental effect of redundant binding site prediction and demonstrates significant improvement in recall and precision through stronger scoring schemes. Finally, top-N+2 recall is proposed as a universal benchmark metric for ligand binding site prediction, with a recommendation for open-source sharing of both methods and benchmarks.

中文翻译：

---

蛋白质-配体结合位点预测方法的比较评价


准确鉴定蛋白质-配体结合位点对于理解和调节蛋白质功能至关重要。因此，配体结合位点预测三十多年来一直是研究重点，开发了 50 多种方法，并将范式从基于几何的范式转变为机器学习。在这项工作中，我们整理了 13 个配体结合位点预测因子，时间跨度为 30 年，专注于最新的基于机器学习的方法，如 VN-EGNN、IF-SitePred、GrASP、PUResNet 和 DeepPocket，并将它们与已建立的 P2Rank、PRANK 和 fpocket 以及早期的方法进行比较，如 PocketFinder、Ligsite 和 Surfnet。我们将这些方法与我们新的精选参考数据集 LIGYSIS 的人类子集进行基准测试。LIGYSIS 是一个全面的蛋白质-配体复合物数据集，由 30,000 种具有结合配体的蛋白质组成，它聚集了来自同一蛋白质的多个结构的生物单元中具有生物学相关性的独特蛋白质-配体界面。LIGYSIS 是对早期数据集（如 sc-PDB、PDBbind、结合 MOAD、COACH420 和 HOLO4K）测试方法的改进，这些数据集包括 1：1 蛋白质-配体复合物或考虑不对称单位。PRANK 和 DeepPocket 对 fpocket 预测的重新评分显示召回率最高 （60%），而 IF-SitePred 的召回率最低 （39%）。我们证明了结合位点的冗余预测对性能的不利影响以及更强的口袋评分方案的有益影响，召回率提高了 14% （IF-SitePred），精确率提高了 30% （Surfnet）。最后，我们建议将 top-N+2 召回率作为配体结合位点预测的通用基准指标，并敦促作者不仅分享他们方法的源代码，而且分享他们的基准。 科学贡献 本研究进行了迄今为止最大的配体结合位点预测方法基准测试，使用 10 个信息指标比较了 13 种原始方法和 15 种变体。介绍了 LIGYSIS 数据集，该数据集聚合了同一蛋白质的多个结构中具有生物学相关性的蛋白质-配体界面。该研究强调了冗余结合位点预测的有害影响，并证明了通过更强大的评分方案可以提高召回率和精确率。最后，top-N+2 召回率被提议作为配体结合位点预测的通用基准指标，并建议开源共享方法和基准。