The approximation properties of infinitely wide shallow neural networks heavily depend on the choice of the activation function. To understand this influence, we study embeddings between Barron spaces with different activation functions. These embeddings are proven by providing push-forward maps on the measures used to represent functions . An activation function of particular interest is the rectified power unit (RePU) given by . For many commonly used activation functions, the well-known Taylor remainder theorem can be used to construct a push-forward map, which allows us to prove the embedding of the associated Barron space into a Barron space with a RePU as activation function. Moreover, the Barron spaces associated with the have a hierarchical structure similar to the Sobolev spaces .

中文翻译：

---

具有高阶激活函数的巴伦空间之间的嵌入


无限宽浅层神经网络的逼近特性在很大程度上取决于激活函数的选择。为了理解这种影响，我们研究了具有不同激活函数的巴伦空间之间的嵌入。这些嵌入通过提供用于表示函数的度量的前推图来证明。特别令人感兴趣的激活函数是由 给出的整流功率单元 (RePU)。对于许多常用的激活函数，可以使用著名的泰勒余数定理来构造前推映射，这使我们能够证明将关联的巴伦空间嵌入到以 RePU 作为激活函数的巴伦空间中。此外，与 相关的 Barron 空间具有与 Sobolev 空间类似的层次结构。