The majority of existing face inpainting methods primarily focus on generating a single result that visually resembles the original image. The generation of diverse and plausible results has emerged as a new branch in image restoration, often referred to as “Pluralistic Image Completion”. However, most diversity methods simply use random latent vectors to generate multiple results, leading to uncontrollable outcomes. To overcome these limitations, we introduce a novel architecture known as the Reference-Guided Directional Diverse Face Inpainting Network. In this paper, instead of using a background image as reference, which is typically used in image restoration, we have used a face image, which can have many different characteristics from the original image, including but not limited to gender and age, to serve as a reference face style. Our network firstly infers the semantic information of the masked face, i.e., the face parsing map, based on the partial image and its mask, which subsequently guides and constrains directional diverse generator network. The network will learn the distribution of face images from different domains in a low-dimensional manifold space. To validate our method, we conducted extensive experiments on the CelebAMask-HQ dataset. Our method not only produces high-quality oriented diverse results but also complements the images with the style of the reference face image. Additionally, our diverse results maintain correct facial feature distribution and sizes, rather than being random. Our network has achieved SOTA results in face diverse inpainting when writing. Code will is available at https://github.com/nothingwithyou/RDFINet.

大多数现有的面部修复方法主要侧重于生成视觉上类似于原始图像的单个结果。生成多样化且可信的结果已成为图像恢复的一个新分支，通常称为“多元图像补全”。然而，大多数多样性方法只是使用随机潜在向量来生成多个结果，从而导致不可控的结果。为了克服这些限制，我们引入了一种称为参考引导定向多样化面部修复网络的新颖架构。在本文中，我们没有使用图像恢复中通常使用的背景图像作为参考，而是使用人脸图像来服务，该人脸图像可以具有与原始图像许多不同的特征，包括但不限于性别和年龄。作为参考脸型。我们的网络首先根据部分图像及其掩模推断蒙面人脸的语义信息，即人脸解析图，随后引导和约束定向多样化生成器网络。该网络将学习低维流形空间中不同域的人脸图像的分布。为了验证我们的方法，我们在 CelebAMask-HQ 数据集上进行了广泛的实验。我们的方法不仅产生高质量的多样化结果，而且还用参考人脸图像的风格补充图像。此外，我们的多样化结果保持了正确的面部特征分布和大小，而不是随机的。我们的网络在写作时的面部多样化修复方面取得了 SOTA 的结果。代码可在 https://github.com/nothingwithyou/RDFINet 获取。