To realize the exquisite interaction and precise manipulation for the robot, in this article, we propose a multispectral multimodal visuotactile sensor named M $^{3}$ Tac, which combines visible, near-infrared, and mid-infrared imaging technologies for the first time and can exceed the sensing ability of human skin in terms of resolution (719 pixels/cm $^{2}$ ), temperature sensing range (−20–130 $^\text{o}$ C), etc. The M $^{3}$ Tac cannot only realize high-quality sensing of deformation, texture, force, stickiness, and temperature comparable to human skin but also can realize proximity sensing that is lacking for human skin. To achieve this, we not only design a multispectral imaging system with an elastic film whose light penetrability can be regulated by the brightness of the light, but also develop corresponding algorithms, including the pixel-level force sensing with finite element method (accuracy: $\pm$ 0.023N), the proximity perception (accuracy: $\pm$ 3.8 mm), the 3-D reconstruction (accuracy: 0.33 mm), the super-resolution temperature sensing (accuracy: $\pm 0.3^\text{o}$ C), the multimodal fusion classification (accuracy: 98%), and the stickiness recognition (accuracy: 98%). Finally, we conduct experiments to verify the effectiveness and application potential of our research.

中文翻译：

---

M$^{3}$Tac：具有超越人类感知能力的多光谱多模态视觉触觉传感器


为了实现机器人的精致交互和精准操控，在本文中，我们提出了一种名为 M $^{3}$ Tac 的多光谱多模态视觉触觉传感器，它首次结合了可见光、近红外和中红外成像技术，在分辨率（719 像素/cm $^{2}$ ）、温度感应范围（-20–130 $^\text{o}$ C）、 等。M $^{3} Tac 不仅可以实现与人体皮肤相媲美的变形、质地、力、粘性和温度的高质量传感，还可以实现人类皮肤所缺乏的接近感应。为了实现这一目标，我们不仅设计了一个具有弹性膜的多光谱成像系统，其光穿透性可以通过光的亮度来调节，而且还开发了相应的算法，包括有限元法的像素级力传感（精度：$\pm$ 0.023N）、接近感知（精度：$\pm$ 3.8 mm）、三维重建（精度： 0.33 mm）、超分辨率温度传感（准确率：$\pm 0.3^\text{o}$ C）、多模态融合分类（准确率：98%）和粘性识别（准确率：98%）。最后，我们进行实验以验证我们研究的有效性和应用潜力。