Pretrained Graph Neural Networks have been widely adopted for various molecular property prediction tasks. Despite their ability to encode structural and relational features of molecules, traditional fine-tuning of such pretrained GNNs on the target task can lead to poor generalization. To address this, we explore the adaptation of pretrained GNNs to the target task by jointly training them with multiple auxiliary tasks. This could enable the GNNs to learn both general and task-specific features, which may benefit the target task. However, a major challenge is to determine the relatedness of auxiliary tasks with the target task. To address this, we investigate multiple strategies to measure the relevance of auxiliary tasks and integrate such tasks by adaptively combining task gradients or by learning task weights via bi-level optimization. Additionally, we propose a novel gradient surgery-based approach, Rotation of Conflicting Gradients ( $$\mathop {\texttt{RCGrad}}\limits$$ ), that learns to align conflicting auxiliary task gradients through rotation. Our experiments with state-of-the-art pretrained GNNs demonstrate the efficacy of our proposed methods, with improvements of up to 7.7% over fine-tuning. This suggests that incorporating auxiliary tasks along with target task fine-tuning can be an effective way to improve the generalizability of pretrained GNNs for molecular property prediction. Scientific contribution We introduce a novel framework for adapting pretrained GNNs to molecular tasks using auxiliary learning to address the critical issue of negative transfer. Leveraging novel gradient surgery techniques such as $$\mathop {\texttt{RCGrad}}\limits$$ , the proposed adaptation framework represents a significant departure from the dominant pretraining fine-tuning approach for molecular GNNs. Our contributions are significant for drug discovery research, especially for tasks with limited data, filling a notable gap in the efficient adaptation of pretrained models for molecular GNNs.

中文翻译：

---

通过辅助学习和特定任务适应增强分子特性预测


预训练的图神经网络已广泛应用于各种分子特性预测任务。尽管它们能够编码分子的结构和关系特征，但此类预训练 GNN 在目标任务上的传统微调可能会导致泛化能力较差。为了解决这个问题，我们通过将预训练的 GNN 与多个辅助任务联合训练来探索预训练的 GNN 对目标任务的适应。这可以使 GNN 学习一般特征和特定于任务的特征，这可能有利于目标任务。然而，一个主要的挑战是确定辅助任务与目标任务的相关性。为了解决这个问题，我们研究了多种策略来衡量辅助任务的相关性，并通过自适应地组合任务梯度或通过双层优化学习任务权重来整合这些任务。此外，我们提出了一种基于梯度手术的新颖方法，冲突梯度旋转（ $$\mathop {\texttt{RCGrad}}\limits$$ ），它学习通过旋转来对齐冲突的辅助任务梯度。我们使用最先进的预训练 GNN 进行的实验证明了我们提出的方法的有效性，与微调相比提高了高达 7.7%。这表明将辅助任务与目标任务微调结合起来可以是提高预训练 GNN 分子属性预测泛化能力的有效方法。科学贡献 我们引入了一种新颖的框架，使用辅助学习使预训练的 GNN 适应分子任务，以解决负迁移的关键问题。 利用新颖的梯度手术技术，例如 $$\mathop {\texttt{RCGrad}}\limits$$ ，所提出的适应框架代表了与分子 GNN 的主流预训练微调方法的显着背离。我们的贡献对于药物发现研究具有重要意义，特别是对于数据有限的任务，填补了分子 GNN 预训练模型有效适应方面的显着空白。