It is frequently observed that overparameterized neural networks generalize well. Regarding such phenomena, existing theoretical work mainly devotes to linear settings or fully-connected neural networks. This paper studies the learning ability of an important family of deep neural networks, deep convolutional neural networks (DCNNs), under both underparameterized and overparameterized settings. We establish the first learning rates of underparameterized DCNNs without parameter or function variable structure restrictions presented in the literature. We also show that by adding well-defined layers to a non-interpolating DCNN, we can obtain some interpolating DCNNs that maintain the good learning rates of the non-interpolating DCNN. This result is achieved by a novel network deepening scheme designed for DCNNs. Our work provides theoretical verification of how overfitted DCNNs generalize well.

人们经常观察到，过度参数化的神经网络具有良好的泛化能力。对于此类现象，现有的理论工作主要致力于线性设置或全连接神经网络。本文研究了深度神经网络的一个重要家族——深度卷积神经网络（DCNN）在参数不足和参数过高的设置下的学习能力。我们建立了第一个参数化 DCNN 的学习率，没有文献中提出的参数或函数变量结构限制。我们还表明，通过向非插值 DCNN 添加明确定义的层，我们可以获得一些保持非插值 DCNN 良好学习率的插值 DCNN。这一结果是通过为 DCNN 设计的新颖网络深化方案实现的。