Designing compounds with a range of desirable properties is a fundamental challenge in drug discovery. In pre-clinical early drug discovery, novel compounds are often designed based on an already existing promising starting compound through structural modifications for further property optimization. Recently, transformer-based deep learning models have been explored for the task of molecular optimization by training on pairs of similar molecules. This provides a starting point for generating similar molecules to a given input molecule, but has limited flexibility regarding user-defined property profiles. Here, we evaluate the effect of reinforcement learning on transformer-based molecular generative models. The generative model can be considered as a pre-trained model with knowledge of the chemical space close to an input compound, while reinforcement learning can be viewed as a tuning phase, steering the model towards chemical space with user-specific desirable properties. The evaluation of two distinct tasks—molecular optimization and scaffold discovery—suggest that reinforcement learning could guide the transformer-based generative model towards the generation of more compounds of interest. Additionally, the impact of pre-trained models, learning steps and learning rates are investigated. Scientific contribution Our study investigates the effect of reinforcement learning on a transformer-based generative model initially trained for generating molecules similar to starting molecules. The reinforcement learning framework is applied to facilitate multiparameter optimisation of starting molecules. This approach allows for more flexibility for optimizing user-specific property profiles and helps finding more ideas of interest.

中文翻译：

---

基于 Transformer 的分子设计中强化学习的评估


设计具有一系列理想特性的化合物是药物发现中的基本挑战。在临床前的早期药物发现中，新化合物通常是基于现有的有前景的起始化合物，通过结构修饰来设计，以进一步优化性能。最近，通过对相似分子对进行训练，探索了基于 Transformer 的深度学习模型来完成分子优化任务。这提供了生成与给定输入分子相似的分子的起点，但对于用户定义的属性配置文件的灵活性有限。在这里，我们评估强化学习对基于 Transformer 的分子生成模型的影响。生成模型可以被视为具有接近输入化合物的化学空间知识的预训练模型，而强化学习可以被视为调整阶段，将模型引导至具有用户特定所需属性的化学空间。对分子优化和支架发现这两个不同任务的评估表明，强化学习可以指导基于 Transformer 的生成模型生成更多感兴趣的化合物。此外，还研究了预训练模型、学习步骤和学习率的影响。科学贡献我们的研究调查了强化学习对基于变压器的生成模型的影响，该模型最初经过训练用于生成与起始分子类似的分子。应用强化学习框架来促进起始分子的多参数优化。这种方法可以更灵活地优化用户特定的属性配置文件，并有助于发现更多感兴趣的想法。