Nonlinear operators with long-distance spatiotemporal dependencies are fundamental in modelling complex systems across sciences; yet, learning these non-local operators remains challenging in machine learning. Integral equations, which model such non-local systems, have wide-ranging applications in physics, chemistry, biology and engineering. We introduce the neural integral equation, a method for learning unknown integral operators from data using an integral equation solver. To improve scalability and model capacity, we also present the attentional neural integral equation, which replaces the integral with self-attention. Both models are grounded in the theory of second-kind integral equations, where the indeterminate appears both inside and outside the integral operator. We provide a theoretical analysis showing how self-attention can approximate integral operators under mild regularity assumptions, further deepening previously reported connections between transformers and integration, as well as deriving corresponding approximation results for integral operators. Through numerical benchmarks on synthetic and real-world data, including Lotka–Volterra, Navier–Stokes and Burgers’ equations, as well as brain dynamics and integral equations, we showcase the models’ capabilities and their ability to derive interpretable dynamics embeddings. Our experiments demonstrate that attentional neural integral equations outperform existing methods, especially for longer time intervals and higher-dimensional problems. Our work addresses a critical gap in machine learning for non-local operators and offers a powerful tool for studying unknown complex systems with long-range dependencies.

具有长距离时空依赖性的非线性算子是跨科学复杂系统建模的基础；然而，学习这些非本地运算符在机器学习中仍然具有挑战性。对此类非局部系统进行建模的积分方程在物理、化学、生物学和工程领域具有广泛的应用。我们介绍神经积分方程，这是一种使用积分方程求解器从数据中学习未知积分算子的方法。为了提高可扩展性和模型容量，我们还提出了注意力神经积分方程，用自注意力代替积分。这两个模型都基于第二类积分方程理论，其中不定式出现在积分算子的内部和外部。我们提供了理论分析，展示了自注意力如何在温和的规律性假设下逼近积分算子，进一步加深了先前报道的变压器和积分之间的联系，并导出了积分算子的相应逼近结果。通过合成数据和真实世界数据的数值基准，包括 Lotka–Volterra、Navier–Stokes 和 Burgers 方程，以及大脑动力学和积分方程，我们展示了模型的功能及其导出可解释的动力学嵌入的能力。我们的实验表明，注意力神经积分方程优于现有方法，特别是对于较长时间间隔和高维问题。我们的工作解决了非本地操作员在机器学习方面的一个关键差距，并为研究具有远程依赖性的未知复杂系统提供了强大的工具。