Natural visual scenes are dominated by spatiotemporal image dynamics, but how the visual system integrates “movie” information over time is unclear. We characterized optic tectal neuronal receptive fields using sparse noise stimuli and reverse correlation analysis. Neurons recognized movies of ~200-600 ms durations with defined start and stop stimuli. Movie durations from start to stop responses were tuned by sensory experience though a hierarchical algorithm. Neurons encoded families of image sequences following trigonometric functions. Spike sequence and information flow suggest that repetitive circuit motifs underlie movie detection. Principles of frog topographic retinotectal plasticity and cortical simple cells are employed in machine learning networks for static image recognition, suggesting that discoveries of principles of movie encoding in the brain, such as how image sequences and duration are encoded, may benefit movie recognition technology. We built and trained a machine learning network that mimicked neural principles of visual system movie encoders. The network, named MovieNet, outperformed current machine learning image recognition networks in classifying natural movie scenes, while reducing data size and steps to complete the classification task. This study reveals how movie sequences and time are encoded in the brain and demonstrates that brain-based movie processing principles enable efficient machine learning.

中文翻译：

---

识别电影编码神经元支持电影识别 AI


自然视觉场景由时空图像动态主导，但视觉系统如何随着时间的推移整合“电影”信息尚不清楚。我们使用稀疏噪声刺激和逆相关分析表征了视盖神经元感受野。神经元识别持续时间为 ~200-600 毫秒的电影，具有定义的开始和停止刺激。电影从开始到停止的持续时间响应通过分层算法通过感官体验进行调整。神经元按照三角函数编码图像序列系列。尖峰序列和信息流表明重复的电路基序是电影检测的基础。在机器学习网络中采用青蛙地形视网膜可塑性和皮质简单细胞的原理进行静态图像识别，这表明在大脑中发现电影编码原理，例如图像序列和持续时间的编码方式，可能有利于电影识别技术。我们构建并训练了一个机器学习网络，该网络模拟了视觉系统电影编码器的神经原理。该网络名为 MovieNet，在对自然电影场景进行分类方面优于当前的机器学习图像识别网络，同时减少了数据大小和完成分类任务的步骤。这项研究揭示了电影序列和时间在大脑中的编码方式，并证明了基于大脑的电影处理原理可以实现高效的机器学习。