Creating an image focal stack requires multiple shots, which captures images at different depths within the same scene. Such methods are not suitable for scenes undergoing continuous changes. Achieving an all-in-focus image from a single shot poses significant challenges, due to the highly ill-posed nature of rectifying defocus and deblurring from a single image. In this paper, to restore an all-in-focus image, we introduce the neuromorphic focal stack, which is defined as neuromorphic signal streams captured by an event/ a spike camera during a continuous focal sweep, aiming to restore an all-in-focus image. Given an RGB image focused at any distance, we harness the high temporal resolution of neuromorphic signal streams. From neuromorphic signal streams, we automatically select refocusing timestamps and reconstruct corresponding refocused images to form a focal stack. Guided by the neuromorphic signal around the selected timestamps, we can merge the focal stack using proper weights and restore a sharp all-in-focus image. We test our method on two distinct neuromorphic cameras. Experimental results from both synthetic and real datasets demonstrate a marked improvement over existing state-of-the-art methods.

中文翻译：

---

来自神经形态焦点堆栈的混合全焦点成像


创建图像焦点堆栈需要多次拍摄，即在同一场景中捕获不同深度的图像。此类方法不适合连续变化的场景。由于从单张图像中校正散焦和去模糊的性质非常不适定，因此从单次拍摄中获得全焦图像面临着巨大的挑战。在本文中，为了恢复全焦点图像，我们引入了神经形态焦点堆栈，它被定义为在连续焦点扫描期间由事件/尖峰相机捕获的神经形态信号流，旨在恢复全焦点图像。焦点图像。给定聚焦在任意距离的 RGB 图像，我们利用神经形态信号流的高时间分辨率。从神经形态信号流中，我们自动选择重新聚焦时间戳并重建相应的重新聚焦图像以形成焦点堆栈。在选定时间戳周围的神经形态信号的指导下，我们可以使用适当的权重合并焦点堆栈并恢复清晰的全焦点图像。我们在两个不同的神经形态相机上测试我们的方法。合成数据集和真实数据集的实验结果表明，比现有最先进的方法有显着改进。