SN-CLPGAN：基于谱归一化的中国传统山水画风格迁移方法

胡琦瑶; 刘乾龙; 彭先霖; 张翔; 彭盛霖; 范建平

doi:10.16152/j.cnki.xdxbzr.2025-01-005

您当前的位置：

首页 >

文章列表页 >

SN-CLPGAN：基于谱归一化的中国传统山水画风格迁移方法

文化遗产数字化保护 | 更新时间：2025-02-08

- SN-CLPGAN：基于谱归一化的中国传统山水画风格迁移方法
- SN-CLPGAN: A method of style transfer in Chinese traditional landscape painting based on spectral normalization
- 西北大学学报（自然科学版） 2025年55卷第1期页码：63-74
- 作者机构：
  
  1.西北大学　信息科学与技术学院，陕西　西安　 710127
  2.生成式人工智能与混合现实陕西省高等学校重点实验室，陕西　西安　 710127
  3.西北大学　网络与数据中心，陕西　西安　 710127
  4.西北大学　艺术学院，陕西　西安　 710127
  5.西北大学　新型网络智能信息服务国家地方联合工程研究中心，陕西　西安　 710127
- 作者简介：
  
  胡琦瑶，女，讲师，从事人工智能艺术、AIGC研究，huqiyao@nwu.edu.cn。
  彭先霖，男，副教授，从事文化遗产数字化保护研究，pxl@nwu.edu.cn。
- 基金信息：
  
  国家自然科学基金(62471390;62306237);陕西省重点研发计划(2024GX-YBXM-149);西北大学研究生创新项目(CX2024204;CX2024206)
- DOI：10.16152/j.cnki.xdxbzr.2025-01-005
  中图分类号： TP391.41
- 纸质出版日期：2025-02-25，
  
  收稿日期：2024-05-20，
- 稿件说明：
移动端阅览
胡琦瑶, 刘乾龙, 彭先霖, 等. SN-CLPGAN：基于谱归一化的中国传统山水画风格迁移方法[J]. 西北大学学报（自然科学版）, 2025,55(1):63-74.

HU QIYAO, LIU QIANLONG, PENG XIANLIN, et al. SN-CLPGAN: A method of style transfer in Chinese traditional landscape painting based on spectral normalization. [J]. Journal of northwest university (natural science edition), 2025, 55(1): 63-74.
胡琦瑶, 刘乾龙, 彭先霖, 等. SN-CLPGAN：基于谱归一化的中国传统山水画风格迁移方法[J]. 西北大学学报（自然科学版）, 2025,55(1):63-74. DOI： 10.16152/j.cnki.xdxbzr.2025-01-005.

HU QIYAO, LIU QIANLONG, PENG XIANLIN, et al. SN-CLPGAN: A method of style transfer in Chinese traditional landscape painting based on spectral normalization. [J]. Journal of northwest university (natural science edition), 2025, 55(1): 63-74. DOI： 10.16152/j.cnki.xdxbzr.2025-01-005.

摘要

中国传统山水画的风格迁移为文化遗产数字化保护与传承提供了新的路径，近年来，深度学习技术已实现了不同图像间的风格迁移，并展现出栩栩如生的效果。中国传统山水画的风格迁移旨在继承中国古代画家独特的绘画技巧，但存在3个缺陷：①缺乏高质量的中国传统山水画图像数据集；②忽略了中国传统山水画特有的技法和笔墨细节；③风格迁移效果与真实山水画有所差距。为了弥补上述缺陷，首先，创建了一个基于风格迁移的中国传统山水画数据集STCLP，包含4 281幅高质量的中国山水画以及自然景观图像，并提出了一种基于谱归一化的中国山水画风格迁移方法SN-CLPGAN。其次，提出了在生成器中使用融合反射填充层的残差稠密块（residual-in-residual dense block，RRDB）学习中国山水画独特的笔触和技法。接着，引入了多尺度结构相似性指数测量（multi-scale structural similarity index measure，MS-SSIM）损失以减少2幅图像之间的像素差异，使生成图像更接近传统绘画的色彩和颜料。最后，采用了融合谱归一化（spectral normalization，SN）的U-Net判别器增强图像纹理细节，并确保了模型训练过程的稳定性。大量实验验证了提出的方法在中国传统山水画风格迁移任务中的有效性和先进性。

Abstract

The style tranfer of Chinese landscape paintings offering new avenues for the digital preservation and inheritance of cultural heritage. In recent years

deep learning technologies have enabled style transfer between different images

achieving lifelike effects. Style transfer in Chinese landscape paintings aims to preserve the unique paintings skills of ancient Chinese painters

but faces three main challenges: ① The lack of high-quality datasets of traditional Chinese landscape paintings. ② The oversight of the unique techniques and ink details specific to traditional Chinese landscape paintings. ③ The gap between the style transfer outcomes and real landscape paintings. To address these deficiencies

this paper first introduces a Chinese landscape paintings dataset for style transfer

STCLP

which contains 4281 high-quality images of Chinese landscape paintings and natural landscapes. A generative adversarial network of style transfer in Chinese landscape painting based on spectral normalization is proposed

termed SN-CLPGAN. Additionally

it introduces the use of residual-in-residual dense blocks (RRDB) with reflect padding layers in the generator to learn the distinctive brushstrokes and techniques of Chinese landscape paintings. Furthermore

it employs the multi-scale structural similarity index measure (MS-SSIM) loss to minimize pixel-level differences between images

thereby producing images closer to traditional paintings in terms of color and pigmentation. Finally

the U-Net discriminator fused with SN is utilized to enhance the textural details of images

ensuring the stability of the model training process. Extensive experiments validate the effectiveness and advancement of the proposed method in the task of style transfer for Chinese landscape paintings.

关键词

风格迁移人工智能艺术中国传统山水画生成对抗网络

Keywords

style transferartificial intelligence artChinese landscape paintingsgenerative adversarial network

references

彭进业, 余喆, 屈书毅, 等. 基于深度学习的图像修复方法研究综述[J]. 西北大学学报(自然科学版), 2023, 53(6): 943-963.

PENG J Y, YU Z, QU S Y, et al. A survey of image inpainting methods based on deep learning[J]. Journal of Northwest University (Natural Science Edition), 2023, 53(6): 943-963.

胡琦瑶, 杨皓文, 王佳欣, 等. 基于弱监督深度学习的图像检索技术研究[J]. 西北大学学报(自然科学版), 2020, 50(5): 793-801.

HU Q Y, YANG H W, WANG J X, et al. Research on image retrieval based on weakly-supervised deep learning[J]. Journal of Northwest University (Natural Science Edition), 2020, 50(5): 793-801.

GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[EB/OL]. (2014-06-10) [2024-04-10]. http://arxiv.org/abs/1406.2661http://arxiv.org/abs/1406.2661.

ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2242-2251.

ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5967-5976.

ZHENG Z T, LIU J Y, ZHENG N N. P2-GAN: Efficient stroke style transfer using single style image[J]. IEEE Transactions on Multimedia, 2023, 25: 6000-6012.

WANG Z Z, ZHANG Z J, ZHAO L, et al. AesUST: Towards aesthetic-enhanced universal style transfer[C]//Proceedings of the 30th ACM International Conference on Multimedia. Lisboa: ACM, 2022: 1095-1106.

WANG X T, YU K, WU S X, et al. ESRGAN: Enhanced super-resolution generative adversarial networks[C]//Computer Vision-ECCV 2018 Workshops. Cham: Springer International Publishing, 2019: 63-79.

WANG X T, XIE L B, DONG C, et al. Real-ESRGAN: Training real-world blind super-resolution with pure synthetic data[C]//2021 IEEE/CVF International Conference on Computer Vision Workshops. Montreal: IEEE, 2021: 1905-1914.

RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation[M]//Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. Cham: Springer International Publishing, 2015: 234-241.

MIYATO T, KATAOKA T, KOYAMA M, et al. Spectral normalization for generative adversarial networks[EB/OL]. (2018-02-16) [2024-04-10]. https://arxiv.org/abs/1802.05957https://arxiv.org/abs/1802.05957.

ZHAO H, GALLO O, FROSIO I, et al. Loss functions for image restoration with neural networks[J]. IEEE Transactions on Computational Imaging, 2017, 3(1): 47-57.

孟宪佳, 傅利平, 刘栋, 等. 高性能计算发展现状及其在文化遗产保护中的应用展望[J]. 西北大学学报(自然科学版), 2021, 51(5): 807-815.

MENG X J, FU L P, LIU D, et al. Development status of high performance computing and its application prospect in cultural heritage protection[J]. Journal of Northwest University (Natural Science Edition), 2021, 51(5): 807-815.

GATYS L A, ECKER A S, BETHGE M, et al. Texture synthesis using convolutional neural networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Montreal: ACM, 2015: 262-270.

CHEN R F, HUANG W B, HUANG B H, et al. Reusing discriminators for encoding: Towards unsupervised image-to-image translation[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 8165-8174.

YI Z L, ZHANG H, TAN P, et al. DualGAN: Unsupervised dual learning for image-to-image translation[C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2868-2876.

KIM J, KIM M, KANG H, et al. U-GAT-IT: Unsupervised generative attentional networks with adaptive layer-instance normalization for image-to-image translation[EB/OL]. (2020-04-08) [2024-04-10]. https://arxiv.org/abs/1907.10830v4https://arxiv.org/abs/1907.10830v4.

张美玉, 刘跃辉, 秦绪佳, 等. 基于拉普拉斯算子抑制伪影的神经风格迁移方法[J]. 计算机科学, 2020, 47(S2): 209-214.

ZHANG M Y, LIU Y H, QIN X J, et al. Neural style transfer method based on Laplacian operator to suppress artifacts[J]. Computer Science, 2020, 47(S2): 209-214.

茹超, 周延, 陈晓璇, 等. 一种面向文本图像的颜色迁移算法[J]. 西北大学学报(自然科学版), 2017, 47(6): 815-821.

RU C, ZHOU Y, CHEN X X, et al. A color transfer algorithm for text image[J]. Journal of Northwest University (Natural Science Edition), 2017, 47(6): 815-821.

PENG X L, PENG S L, HU Q Y, et al. Contour-enhanced CycleGAN framework for style transfer from scenery photos to Chinese landscape paintings[J]. Neural Computing and Applications, 2022, 34(20): 18075-18096.

ZHANG F Q, GAO H M, LAI Y P. Detail-preserving CycleGAN-AdaIN framework for image-to-ink painting translation[J]. IEEE Access, 2020, 8: 132002-132011.

IOFFE S, SZEGEDY C, PARANHOS L, et al. Batch normalization: Accelerating deep network training by reducing internal covariate shift[EB/OL]. (2015-03-02) [2024-04-10]. https://arxiv.org/abs/1502.03167v3https://arxiv.org/abs/1502.03167v3.

LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu: IEEE, 2017: 1132-1140.

NAH S, KIM T H, LEE K M. Deep multi-scale convolutional neural network for dynamic scene deblurring[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 257-265.

WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.

KINGMA D P, BA J, HAMMAD M M. Adam: A method for stochastic optimization[EB/OL]. (2017-01-30) [2024-04-10]. https://arxiv.org/abs/1412.6980v9https://arxiv.org/abs/1412.6980v9.

CHONG M J, FORSYTH D. Effectively unbiased FID and inception score and where to find them[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6069-6078.

TALEBI H, MILANFAR P. NIMA: Neural image assessment[J]. IEEE Transactions on Image Processing, 2018, 27(8): 3998-4011.

孙满利, 张景科. 文物保护学的理论探讨[J]. 西北大学学报(自然科学版), 2022, 52(2): 192-198.

SUN M L, ZHANG J K. Theoretical discussion on conservation of cultural heritages[J]. Journal of Northwest University (Natural Science Edition), 2022, 52(2): 192-198.

HO J, JAIN A, ABBEEL P, et al. Denoising diffusion probabilistic models[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Vancouver: ACM, 2020: 6840-6851.

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于多层次瓶颈注意力模块的颅骨到面皮的生成方法

基于生成对抗网络的乳腺MRI图像生成

MeGAN:基于多任务增强生成对抗网络的图像合成