Computational methods for generating molecules with specific physiochemical properties or biolog- ical activity can greatly assist drug discovery efforts. Deep learning generative models constitute a significant step towards that direction. In this work, we introduce a novel approach that utilizes a Reinforcement Learning paradigm, called proximal policy optimization (PPO), for optimizing chemical molecules in the latent space of a pre-trained deep learning generative model. Working in the latent space of a generative model allows us to bypass the need for explicitly defining chemical rules when computationally designing molecules. The generation of molecules with desired properties is achieved through navigating the latent space for identifying regions that correspond to molecules with desired properties. Proximal policy optimization is a state-of-the-art policy gradient algorithm capable of operating in continuous high dimensional spaces in a sample-efficient manner. We have paired our optimization framework with the latent spaces of two autoencoder models, a variational autoencoder and an autoencoder trained with mutual information machine loss respectively, showing that the method is agnostic to the underlying architecture. We present results on commonly used benchmarks for molecule optimization that demonstrate that our method has comparable or even superior performance to state-of-the-art approaches. We additionally show how our method can generate molecules that contain a pre-specified substructure while simultaneously optimizing for molecular properties, a task highly relevant to real drug discovery scenarios.

中文翻译：

---

使用潜在强化学习进行靶向分子生成


生成具有特定理化性质或生物活性的分子的计算方法可以极大地帮助药物发现工作。深度学习生成模型是朝着这个方向迈出的重要一步。在这项工作中，我们介绍了一种利用强化学习范式的新方法，称为近端策略优化 （PPO），用于优化预先训练的深度学习生成模型潜在空间中的化学分子。在生成模型的潜在空间中工作使我们能够在计算设计分子时绕过明确定义化学规则的需要。具有所需特性的分子的生成是通过导航潜在空间来识别与具有所需特性的分子相对应的区域来实现的。近端策略优化是一种最先进的策略梯度算法，能够以样本高效的方式在连续的高维空间中运行。我们将优化框架与两个自动编码器模型的潜在空间配对，一个变分自动编码器和一个分别用互信息机损失训练的自动编码器，表明该方法与底层架构无关。我们展示了分子优化常用基准的结果，这些结果表明我们的方法具有与最先进的方法相当甚至更好的性能。我们还展示了我们的方法如何生成包含预先指定的子结构的分子，同时优化分子特性，这是一项与实际药物发现场景高度相关的任务。