Artificial neural networks, deep-learning methods and the backpropagation algorithm¹ form the foundation of modern machine learning and artificial intelligence. These methods are almost always used in two phases, one in which the weights of the network are updated and one in which the weights are held constant while the network is used or evaluated. This contrasts with natural learning and many applications, which require continual learning. It has been unclear whether or not deep learning methods work in continual learning settings. Here we show that they do not—that standard deep-learning methods gradually lose plasticity in continual-learning settings until they learn no better than a shallow network. We show such loss of plasticity using the classic ImageNet dataset and reinforcement-learning problems across a wide range of variations in the network and the learning algorithm. Plasticity is maintained indefinitely only by algorithms that continually inject diversity into the network, such as our continual backpropagation algorithm, a variation of backpropagation in which a small fraction of less-used units are continually and randomly reinitialized. Our results indicate that methods based on gradient descent are not enough—that sustained deep learning requires a random, non-gradient component to maintain variability and plasticity.

人工神经网络、深度学习方法和反向传播算法¹构成了现代机器学习和人工智能的基础。这些方法几乎总是在两个阶段中使用，第一阶段更新网络的权重，第二阶段在使用或评估网络时保持权重恒定。这与自然学习和许多需要持续学习的应用程序形成鲜明对比。目前尚不清楚深度学习方法是否适用于持续学习环境。在这里，我们证明了事实并非如此——标准的深度学习方法在持续学习环境中逐渐失去可塑性，直到它们的学习效果不比浅层网络好。我们使用经典的 ImageNet 数据集以及网络和学习算法的各种变化中的强化学习问题来展示这种可塑性的损失。只有通过不断向网络注入多样性的算法才能无限期地维持可塑性，例如我们的连续反向传播算法，这是反向传播的一种变体，其中一小部分较少使用的单元被连续随机地重新初始化。我们的结果表明，基于梯度下降的方法是不够的——持续的深度学习需要随机的、非梯度的成分来保持可变性和可塑性。