Unpaired Artistic Portrait Style Transfer via Asymmetric Double-Stream GAN
Date
2023
Authors
Kong, F.
Pu, Y.
Lee, I.
Nie, R.
Zhao, Z.
Xu, D.
Qian, W.
Liang, H.
Editors
Advisors
Journal Title
Journal ISSN
Volume Title
Type:
Journal article
Citation
IEEE Transactions on Neural Networks and Learning Systems, 2023; 34(9):5427-5439
Statement of Responsibility
Conference Name
Abstract
With the development of image style transfer technologies, portrait style transfer has attracted growing attention in this research community. In this article, we present an asymmetric double-stream generative adversarial network (ADS-GAN) to solve the problems that caused by cartoonization and other style transfer techniques when they are applied to portrait photos, such as facial deformation, contours missing, and stiff lines. By observing the characteristics between source and target images, we propose an edge contour retention (ECR) regularized loss to constrain the local and global contours of generated portrait images to avoid the portrait deformation. In addition, a content-style feature fusion module is introduced for further learning of the target image style, which uses a style attention mechanism to integrate features and embeds style features into content features of portrait photos according to the attention weights. Finally, a guided filter is introduced in content encoder to smooth the textures and specific details of source image, thereby eliminating its negative impact on style transfer. We conducted overall unified optimization training on all components and got an ADS-GAN for unpaired artistic portrait style transfer. Qualitative comparisons and quantitative analyses demonstrate that the proposed method generates superior results than benchmark work in preserving the overall structure and contours of portrait; ablation and parameter study demonstrate the effectiveness of each component in our framework.
School/Discipline
Dissertation Note
Provenance
Description
Data source: Supplementary information, https://doi.org/10.1109/TNNLS.2023.3263846
Access Status
Rights
Copyright 2023 IEEE.