Nature Machine Intelligence ( IF 18.8 ) Pub Date : 2024-09-25 , DOI: 10.1038/s42256-024-00897-5 Nick McGreivy, Ammar Hakim
One of the most promising applications of machine learning in computational physics is to accelerate the solution of partial differential equations (PDEs). The key objective of machine-learning-based PDE solvers is to output a sufficiently accurate solution faster than standard numerical methods, which are used as a baseline comparison. We first perform a systematic review of the ML-for-PDE-solving literature. Out of all of the articles that report using ML to solve a fluid-related PDE and claim to outperform a standard numerical method, we determine that 79% (60/76) make a comparison with a weak baseline. Second, we find evidence that reporting biases are widespread, especially outcome reporting and publication biases. We conclude that ML-for-PDE-solving research is overoptimistic: weak baselines lead to overly positive results, while reporting biases lead to under-reporting of negative results. To a large extent, these issues seem to be caused by factors similar to those of past reproducibility crises: researcher degrees of freedom and a bias towards positive results. We call for bottom-up cultural changes to minimize biased reporting and so top-down structural reforms to reduce perverse incentives for doing so.
中文翻译:
基线薄弱和报告偏差导致对流体相关偏微分方程的机器学习过度乐观
机器学习在计算物理中最有前途的应用之一是加速偏微分方程(PDE)的求解。基于机器学习的偏微分方程求解器的主要目标是比标准数值方法更快地输出足够准确的解,用作基线比较。我们首先对偏微分方程求解的机器学习文献进行系统回顾。在所有报告使用 ML 求解流体相关偏微分方程并声称优于标准数值方法的文章中,我们确定 79% (60/76) 与弱基线进行了比较。其次,我们发现有证据表明报告偏见普遍存在,尤其是结果报告和出版偏见。我们的结论是,偏微分方程求解的机器学习研究过于乐观:基线薄弱会导致过于积极的结果,而报告偏差会导致消极结果的报告不足。在很大程度上,这些问题似乎是由与过去的可重复性危机类似的因素引起的:研究人员的自由度和对积极结果的偏见。我们呼吁进行自下而上的文化变革,以最大限度地减少有偏见的报道,并呼吁进行自上而下的结构性改革,以减少这样做的不正当动机。