In-sensor computing, which integrates sensing, memory and processing functions, has shown substantial potential in artificial vision systems. However, large-scale monolithic integration of in-sensor computing based on emerging devices with complementary metal–oxide–semiconductor (CMOS) circuits remains challenging, lacking functional demonstrations at the hardware level. Here we report a fully integrated 1-kb array with 128 × 8 one-transistor one-optoelectronic memristor (OEM) cells and silicon CMOS circuits, which features configurable multi-mode functionality encompassing three different modes of electronic memristor, dynamic OEM and non-volatile OEM (NV-OEM). These modes are configured by modulating the charge density within the oxygen vacancies via synergistic optical and electrical operations, as confirmed by differential phase-contrast scanning transmission electron microscopy. Using this OEM system, three visual processing tasks are demonstrated: image sensory pre-processing with a recognition accuracy enhanced from 85.7% to 96.1% by the NV-OEM mode, more advanced object tracking with 96.1% accuracy using both dynamic OEM and NV-OEM modes and human motion recognition with a fully OEM-based in-sensor reservoir computing system achieving 91.2% accuracy. A system-level benchmark further shows that it consumes over 20 times less energy than graphics processing units. By monolithically integrating the multi-functional OEMs with Si CMOS, this work provides a cost-effective platform for diverse in-sensor computing applications.

集成传感、存储和处理功能的传感器内计算在人工视觉系统中显示出巨大的潜力。然而，基于新兴器件的传感器内计算与互补金属氧化物半导体 （CMOS） 电路的大规模单片集成仍然具有挑战性，缺乏硬件级别的功能演示。本文介绍了一个完全集成的 1 KB 阵列，具有 128 × 8 个单晶体管单光电忆阻器 （OEM） 单元和硅 CMOS 电路，它具有可配置的多模式功能，包括三种不同模式的电子忆阻器、动态 OEM 和非易失性 OEM （NV-OEM）。这些模式是通过协同光学和电操作调制氧空位内的电荷密度来配置的，差分相差扫描透射电子显微镜证实了这一点。使用该 OEM 系统，演示了三个视觉处理任务：NV-OEM 模式识别准确率从 85.7% 提高到 96.1% 的图像传感预处理，使用动态 OEM 和 NV-OEM 模式的更高级目标跟踪，准确率达到 96.1%，以及完全基于 OEM 的传感器内储层计算系统的人体运动识别，准确率达到 91.2%。系统级基准测试进一步表明，它消耗的能源比图形处理单元少 20 倍以上。通过将多功能 OEM 与 Si CMOS 单片集成，这项工作为各种传感器内计算应用提供了一个经济高效的平台。