When can the input of a ReLU neural network be inferred from its output? In other words, when is the network injective? We consider a single layer, x↦ReLU(Wx), with a random Gaussian m×n matrix W, in a high-dimensional setting where n,m→∞. Recent work connects this problem to spherical integral geometry giving rise to a conjectured sharp injectivity threshold for α=m/n by studying the expected Euler characteristic of a certain random set. We adopt a different perspective and show that injectivity is equivalent to a property of the ground state of the spherical perceptron, an important spin glass model in statistical physics. By leveraging the (non-rigorous) replica symmetry-breaking theory, we derive analytical equations for the threshold whose solution is at odds with that from the Euler characteristic. Furthermore, we use Gordon's min–max theorem to prove that a replica-symmetric upper bound refutes the Euler characteristic prediction. Along the way we aim to give a tutorial-style introduction to key ideas from statistical physics in an effort to make the exposition accessible to a broad audience. Our analysis establishes a connection between spin glasses and integral geometry but leaves open the problem of explaining the discrepancies.

中文翻译：

---

ReLU 网络的注入性：统计物理学的观点


何时可以从 ReLU 神经网络的输出推断出 ReLU 神经网络的输入？换句话说，网络何时是单射的？我们考虑一个单层 x↦ReLU（Wx），具有一个随机高斯 m×n 矩阵 W，在高维设置中，其中 n，m→∞。最近的工作将这个问题与球面积分几何联系起来，通过研究某个随机集的预期欧拉特性，产生了 α=m/n 的猜想尖锐注射率阈值。我们采用不同的观点，表明注射率相当于球形感知器的基态特性，球形感知器是统计物理学中重要的自旋玻璃模型。通过利用（非严格的）复制对称性打破理论，我们推导出了阈值的解析方程，其解与欧拉特性的解不一致。此外，我们使用 Gordon 的 min-max 定理来证明复制对称上限反驳了 Euler 特征预测。在此过程中，我们的目标是以教程的形式介绍统计物理学的关键思想，以努力使广大读者能够理解本综述。我们的分析在旋转玻璃和整体几何之间建立了联系，但留下了解释差异的悬而未决的问题。