With the surge in emerging technologies such as Metaverse, spatial computing, and generative AI, the application of facial style transfer has gained much interest from researchers and startups enthusiasts alike. StyleGAN methods have paved the way for transfer-learning strategies that could reduce the dependency on the vast data available for the training process. However, StyleGAN methods tend to need to be more balanced, resulting in the introduction of artifacts in the facial images. Studies such as DualStyleGAN proposed multipath networks but required the networks to be trained for a specific style rather than simultaneously generating a fusion of facial styles. In this paper, we propose a Fusion of STyles (FIST) network for facial images that leverages pretrained multipath style transfer networks to eliminate the problem associated with the lack of enormous data volume in the training phase and the fusion of multiple styles at the output. We leverage pretrained styleGAN networks with an external style pass that uses a residual modulation block instead of a transform coding block. The method also preserves facial structure, identity, and details via the gated mapping unit introduced in this study. The aforementioned components enable us to train the network with minimal data while generating high-quality stylized images, opening up new possibilities for facial style transfer in emerging technologies. Our training process adapts curriculum learning strategy to perform efficient, flexible style, and model fusion in the generative space. We perform extensive experiments to show the superiority of the proposed FISTNet compared to existing state-of-the-art methods.

中文翻译：

---

FISTNet：用于面部风格迁移的风格路径生成网络的融合


随着元宇宙、空间计算和生成人工智能等新兴技术的兴起，面部风格迁移的应用引起了研究人员和初创公司爱好者的极大兴趣。 StyleGAN 方法为迁移学习策略铺平了道路，可以减少对训练过程中可用的大量数据的依赖。然而，StyleGAN 方法往往需要更加平衡，从而导致在面部图像中引入伪影。 DualStyleGAN 等研究提出了多路径网络，但要求网络针对特定风格进行训练，而不是同时生成面部风格的融合。在本文中，我们提出了一种用于面部图像的风格融合（FIST）网络，它利用预训练的多路径风格传输网络来消除与训练阶段缺乏大量数据以及输出时多种风格融合相关的问题。我们利用带有外部样式传递的预训练 styleGAN 网络，该传递使用残差调制块而不是变换编码块。该方法还通过本研究中引入的门控映射单元保留面部结构、身份和细节。上述组件使我们能够用最少的数据训练网络，同时生成高质量的风格化图像，为新兴技术中的面部风格迁移开辟了新的可能性。我们的培训过程采用课程学习策略，在生成空间中执行高效、灵活的风格和模型融合。我们进行了大量的实验，以证明所提出的 FISTNet 与现有最先进方法相比的优越性。