Animals are born with extensive innate behavioral capabilities, which arise from neural circuits encoded in the genome. However, the information capacity of the genome is orders of magnitude smaller than that needed to specify the connectivity of an arbitrary brain circuit, indicating that the rules encoding circuit formation must fit through a “genomic bottleneck” as they pass from one generation to the next. Here, we formulate the problem of innate behavioral capacity in the context of artificial neural networks in terms of lossy compression of the weight matrix. We find that several standard network architectures can be compressed by several orders of magnitude, yielding pretraining performance that can approach that of the fully trained network. Interestingly, for complex but not for simple test problems, the genomic bottleneck algorithm also captures essential features of the circuit, leading to enhanced transfer learning to novel tasks and datasets. Our results suggest that compressing a neural circuit through the genomic bottleneck serves as a regularizer, enabling evolution to select simple circuits that can be readily adapted to important real-world tasks. The genomic bottleneck also suggests how innate priors can complement conventional approaches to learning in designing algorithms for AI.

中文翻译：

---

通过基因组瓶颈编码先天能力


动物生来就具有广泛的先天行为能力，这些能力源自基因组中编码的神经回路。然而，基因组的信息容量比指定任意大脑回路连接所需的信息容量要小几个数量级，这表明编码回路形成的规则在从一代传递到下一代时必须适应“基因组瓶颈” 。在这里，我们根据权重矩阵的有损压缩来阐述人工神经网络背景下的先天行为能力问题。我们发现几种标准网络架构可以压缩几个数量级，从而产生可以接近完全训练网络的预训练性能。有趣的是，对于复杂但不简单的测试问题，基因组瓶颈算法还捕获了电路的基本特征，从而增强了对新任务和数据集的迁移学习。我们的结果表明，通过基因组瓶颈压缩神经回路可以作为正则化器，使进化能够选择可以轻松适应重要的现实世界任务的简单回路。基因组瓶颈还表明先天先验如何补充人工智能算法设计中的传统学习方法。