Physics-Informed Neural Networks (PINN) are deep learning algorithms that leverage physical laws by including partial differential equations together with a respective set of boundary and initial conditions as penalty terms in their loss function. In this work, we observe the significant role of correctly weighting the combination of multiple competitive loss functions for training PINNs effectively. To this end, we implement and evaluate different methods aiming at balancing the contributions of multiple terms of the PINN’s loss function and their gradients. After reviewing three existing loss scaling approaches (Learning Rate Annealing, GradNorm and SoftAdapt), we propose a novel self-adaptive loss balancing scheme for PINNs named ReLoBRaLo (Relative Loss Balancing with Random Lookback). We extensively evaluate the performance of the aforementioned balancing schemes by solving both forward as well as inverse problems on three benchmark PDEs for PINNs: Burgers’ equation, Kirchhoff’s plate bending equation, Helmholtz’s equation and over 20 PDEs from the ”PINNacle” collection. The results show that ReLoBRaLo is able to consistently outperform the baseline of existing scaling methods in terms of accuracy while also inducing significantly less computational overhead for a variety of PDE classes.

中文翻译：

---

用于物理信息深度学习的多目标损失平衡


物理信息神经网络 （PINN） 是一种深度学习算法，它利用物理定律，将偏微分方程以及相应的边界和初始条件集作为其损失函数中的惩罚项。在这项工作中，我们观察到正确加权多个竞争性损失函数的组合对于有效训练 PINN 的重要作用。为此，我们实施并评估了不同的方法，旨在平衡 PINN 损失函数的多项及其梯度的贡献。在回顾了三种现有的损失缩放方法（学习率退火、GradNorm 和 SoftAdapt）之后，我们提出了一种新颖的 PINN 自适应损失平衡方案，名为 ReLoBRaLo（随机回溯的相对损失平衡）。我们通过求解 PINN 的三个基准 PDE 的正向和逆向问题来广泛评估上述平衡方案的性能：Burgers 方程、Kirchhoff 板弯曲方程、亥姆霍兹方程和“PINNacle”集合中的 20 多个 PDE。结果表明，ReLoBRaLo 能够在准确性方面始终优于现有缩放方法的基线，同时还显著减少了各种 PDE 类别的计算开销。