Augmented reality (AR) mirrors are novel displays that have great potential for commercial applications such as virtual apparel try-on. Typically the camera is placed beside the display, leading to distorted perspectives during user interaction. In this paper, we present a novel approach to address this problem by placing the camera behind a transparent display, thereby providing users with a perspective-aligned experience. Simply placing the camera behind the display can compromise image quality due to optical effects. We meticulously analyze the image formation process, and present an image restoration algorithm that benefits from physics-based data synthesis and network design. Our method significantly improves image quality and outperforms existing methods especially on the underexplored wire and backscatter artifacts. We then carefully design a full AR mirror system including display and camera selection, real-time processing pipeline, and mechanical design. Our user study demonstrates that the system is exceptionally well-received by users, highlighting its advantages over existing camera configurations not only as an AR mirror, but also for video conferencing. Our work represents a step forward in the development of AR mirrors, with potential applications in retail, cosmetics, fashion, etc. The image restoration dataset and code are available at https://perspective-armirror.github.io/.

中文翻译：

---

带屏下摄像头的透视对齐 AR 镜子


增强现实 （AR） 镜子是一种新型显示器，在虚拟服装试穿等商业应用中具有巨大潜力。通常，摄像头放置在显示屏旁边，导致用户交互期间视角失真。在本文中，我们提出了一种解决此问题的新方法，通过将摄像头放置在透明显示屏后面，从而为用户提供透视对齐的体验。由于光学效应，简单地将摄像头放在显示器后面可能会影响图像质量。我们仔细分析了图像形成过程，并提出了一种受益于基于物理的数据合成和网络设计的图像恢复算法。我们的方法显著提高了图像质量，并优于现有方法，尤其是在未充分开发的有线和反向散射伪影上。然后，我们仔细设计了一个完整的 AR 镜像系统，包括显示器和摄像头选择、实时处理管道和机械设计。我们的用户研究表明，该系统非常受用户欢迎，凸显了它相对于现有摄像头配置的优势，不仅可以作为 AR 镜子，还可以用于视频会议。我们的工作代表了 AR 镜子发展向前迈出的一步，在零售、化妆品、时尚等领域具有潜在应用。图像恢复数据集和代码可在 https://perspective-armirror.github.io/ 获取。