SIAM Review, Volume 65, Issue 3, Page 735-773, August 2023.
We analyze neural ordinary differential equations (NODEs) from a control theoretical perspective to address some of the main properties and paradigms of deep learning (DL), in particular, data classification and universal approximation. These objectives are tackled and achieved from the perspective of the simultaneous control of systems of NODEs. For instance, in the context of classification, each item to be classified corresponds to a different initial datum for the control problem of the NODE, to be classified, all of them by the same common control, to the location (a subdomain of the Euclidean space) associated to each label. Our proofs are genuinely nonlinear and constructive, allowing us to estimate the complexity of the control strategies we develop. The nonlinear nature of the activation functions governing the dynamics of NODEs under consideration plays a key role in our proofs, since it allows deforming half of the phase space while the other half remains invariant, a property that classical models in mechanics do not fulfill. This very property allows us to build elementary controls inducing specific dynamics and transformations whose concatenation, along with properly chosen hyperplanes, allows us to achieve our goals in finitely many steps. The nonlinearity of the dynamics is assumed to be Lipschitz. Therefore, our results apply also in the particular case of the ReLU activation function. We also present the counterparts in the context of the control of neural transport equations, establishing a link between optimal transport and deep neural networks.

中文翻译：

---

用于分类、逼近和传输的神经 ODE 控制

《SIAM 评论》，第 65 卷，第 3 期，第 735-773 页，2023 年 8 月。
我们从控制理论的角度分析神经常微分方程（NODE），以解决深度学习（DL）的一些主要属性和范式，特别是数据分类和通用逼近。这些目标是从节点系统同步控制的角度来解决和实现的。例如，在分类的上下文中，每个要分类的项目对应于要分类的节点的控制问题的不同初始数据，所有这些数据都由相同的公共控制，到位置（欧几里得的子域）空间）与每个标签相关联。我们的证明是真正的非线性和建设性的，使我们能够估计我们开发的控制策略的复杂性。控制所考虑的节点动力学的激活函数的非线性性质在我们的证明中起着关键作用，因为它允许一半相空间变形，而另一半保持不变，这是力学中的经典模型无法满足的属性。这一特性使我们能够构建基本控制，从而引发特定的动态和变换，这些动态和变换的串联以及正确选择的超平面使我们能够在有限的多个步骤中实现我们的目标。动力学的非线性假设为 Lipschitz。因此，我们的结果也适用于 ReLU 激活函数的特定情况。我们还提出了神经传输方程控制背景下的对应方法，建立了最优传输和深度神经网络之间的联系。