Huge advancements have been made over the years in terms of modern image-sensing hardware and visual computing algorithms (e.g., computer vision, image processing, and computational photography). However, to this day, there still exists a current gap between the hardware and software design in an imaging system, which silos one research domain from another. Bridging this gap is the key to unlocking new visual computing capabilities for end applications in commercial photography, industrial inspection, and robotics. In this survey, we explore existing works in the literature that can be leveraged to replace conventional hardware components in an imaging system with software for enhanced reconfigurability. As a result, the user can program the image sensor in a way best suited to the end application. We refer to this as software-defined imaging (SDI), where image sensor behavior can be altered by the system software depending on the user’s needs. The scope of our survey covers imaging systems for single-image capture, multi-image, and burst photography, as well as video. We review works related to the sensor primitives, image signal processor (ISP) pipeline, computer architecture, and operating system elements of the SDI stack. Finally, we outline the infrastructure and resources for SDI systems, and we also discuss possible future research directions for the field.

中文翻译：

---

软件定义成像：调查


多年来，现代图像传感硬件和视觉计算算法（例如计算机视觉、图像处理和计算摄影）取得了巨大进步。然而，直到今天，成像系统的硬件和软件设计之间仍然存在差距，这使得一个研究领域与另一个研究领域相互隔离。弥合这一差距是为商业摄影、工业检测和机器人技术等终端应用解锁新视觉计算能力的关键。在本次调查中，我们探索了文献中的现有作品，这些作品可用于用软件替换成像系统中的传统硬件组件，以增强可重构性。因此，用户可以以最适合最终应用的方式对图像传感器进行编程。我们将其称为软件定义成像 (SDI)，其中图像传感器的行为可以由系统软件根据用户的需求进行更改。我们的调查范围涵盖用于单图像捕捉、多图像、连拍摄影以及视频的成像系统。我们回顾了与 SDI 堆栈的传感器基元、图像信号处理器 (ISP) 管道、计算机架构和操作系统元素相关的工作。最后，我们概述了 SDI 系统的基础设施和资源，并讨论了该领域未来可能的研究方向。