Multi-spectral imaging is a fundamental tool characterizing the constituent energy of scene radiation. However, current multi-spectral video cameras cannot scale up beyond megapixel resolution due to optical constraints and the complexity of the reconstruction algorithms. To circumvent the above issues, we propose a tens-of-megapixel handheld multi-spectral videography approach (THETA), with a proof-of-concept camera achieving 65-megapixel videography of 12 wavebands within visible light range. The high performance is brought by multiple designs: We propose an imaging scheme to fabricate a thin mask for encoding spatio-spectral data using a conventional film camera. Afterwards, a fiber optic plate is introduced for building a compact prototype supporting pixel-wise encoding with a large space-bandwidth product. Finally, a deep-network-based algorithm is adopted for large-scale multi-spectral data decoding, with the coding pattern specially designed to facilitate efficient coarse-to-fine model training. Experimentally, we demonstrate THETA’s advantageous and wide applications in outdoor imaging of large macroscopic scenes.

多光谱成像是表征场景辐射组成能量的基本工具。然而，由于光学限制和重建算法的复杂性，当前的多光谱摄像机无法扩展到超过百万像素的分辨率。为了解决上述问题，我们提出了一种数十兆像素的手持式多光谱摄像方法（THETA），其概念验证相机可在可见光范围内实现 12 个波段的 65 兆像素摄像。高性能是由多种设计带来的：我们提出了一种成像方案来制造薄掩模，以使用传统的胶片相机对空间光谱数据进行编码。随后，引入光纤板来构建紧凑的原型，支持具有大空间带宽乘积的像素级编码。最后，采用基于深度网络的算法进行大规模多光谱数据解码，其编码模式经过专门设计，有利于高效的从粗到精的模型训练。通过实验，我们展示了THETA在大型宏观场景户外成像中的优势和广泛应用。