Memristors with photonic sensory capabilities can be used as elements in machine vision systems but face challenges in terms of encoding and processing optical data. This has led to different neural network architectures being developed for specific vision tasks, which limits progress towards more versatile in-sensor vision computing platforms. Here we describe a multi-phototransistor and one-memristor array that is based on niobium oxide memristors. It has reconfigurable dynamics and is compatible with both machine learning (analogue) and bioinspired (spiking) neural network architectures. The array can sense and process optical images and synchronize spatio-temporal data across different encoding formats. When the array is coupled with a classifier network using a one-transistor and one-memristor non-volatile memory array, it supports a variety of optical neural networks (including optical convolutional neural networks, recurrent neural networks and spiking neural networks). The resulting system can perform various computing vision tasks, such as recognizing static, motion and colour images.

具有光子传感能力的忆阻器可以用作机器视觉系统中的元件，但在编码和处理光学数据方面面临挑战。这导致针对特定视觉任务开发了不同的神经网络架构，这限制了向更通用的传感器内视觉计算平台迈进的进展。在这里，我们描述了一个基于氧化铌忆阻器的多光电晶体管和单忆阻器阵列。它具有可重构的动力学功能，并与机器学习（模拟）和生物启发（尖峰）神经网络架构兼容。该阵列可以感知和处理光学图像，并跨不同编码格式同步时空数据。当该阵列与使用单晶体管和单忆阻器非易失性存储器阵列的分类器网络耦合时，它支持多种光学神经网络（包括光学卷积神经网络、递归神经网络和脉冲神经网络）。由此产生的系统可以执行各种计算视觉任务，例如识别静态、运动和彩色图像。