Machine learning is becoming a preferred method for the virtual screening of organic materials due to its cost-effectiveness over traditional computationally demanding techniques. However, the scarcity of labeled data for organic materials poses a significant challenge for training advanced machine learning models. This study showcases the potential of utilizing databases of drug-like small molecules and chemical reactions to pretrain the BERT model, enhancing its performance in the virtual screening of organic materials. By fine-tuning the BERT models with data from five virtual screening tasks, the version pretrained with the USPTO–SMILES dataset achieved R2 scores exceeding 0.94 for three tasks and over 0.81 for two others. This performance surpasses that of models pretrained on the small molecule or organic materials databases and outperforms three traditional machine learning models trained directly on virtual screening data. The success of the USPTO–SMILES pretrained BERT model can be attributed to the diverse array of organic building blocks in the USPTO database, offering a broader exploration of the chemical space. The study further suggests that accessing a reaction database with a wider range of reactions than the USPTO could further enhance model performance. Overall, this research validates the feasibility of applying transfer learning across different chemical domains for the efficient virtual screening of organic materials. Scientific contribution This study verifies the feasibility of applying transfer learning to large language models in different chemical fields to help organic materials perform virtual screening. Through the comparison of transfer learning from different chemical fields to a variety of organic material molecules, the high precision virtual screening of organic materials is realized.

中文翻译：

---

跨不同化学领域的迁移学习：利用小分子和化学反应数据预训练的深度学习模型对有机材料进行虚拟筛选


由于机器学习比传统的计算要求高的技术具有成本效益，因此正在成为有机材料虚拟筛选的首选方法。然而，有机材料标记数据的稀缺给训练高级机器学习模型带来了重大挑战。这项研究展示了利用类药物小分子和化学反应数据库来预训练 BERT 模型的潜力，从而增强其在有机材料虚拟筛选中的性能。通过使用来自五个虚拟筛选任务的数据对 BERT 模型进行微调，使用 USPTO-SMILES 数据集预训练的版本在三个任务中实现了超过 0.94 的 R2 分数，在另外两个任务中实现了超过 0.81 的 R2 分数。这种性能超越了在小分子或有机材料数据库上预训练的模型，并且优于直接在虚拟筛选数据上训练的三种传统机器学习模型。 USPTO-SMILES 预训练 BERT 模型的成功可归因于 USPTO 数据库中多样化的有机构建模块，为化学空间提供了更广泛的探索。该研究进一步表明，访问反应范围比美国专利商标局更广泛的反应数据库可以进一步提高模型性能。总的来说，这项研究验证了跨不同化学领域应用迁移学习来有效虚拟筛选有机材料的可行性。科学贡献 这项研究验证了将迁移学习应用于不同化学领域的大语言模型以帮助有机材料进行虚拟筛选的可行性。 通过不同化学领域对多种有机材料分子的迁移学习对比，实现有机材料的高精度虚拟筛选。