Calligraphy image harmonization algorithm based on local light and shadow perception

DONG Zhiqiang; XIAO Yun; DUAN Jiashun

doi:10.16152/j.cnki.xdxbzr.2025-01-016

您当前的位置：

首页 >

文章列表页 >

Calligraphy image harmonization algorithm based on local light and shadow perception

The digital protection of cultural heritage | 更新时间：2025-02-08

- Calligraphy image harmonization algorithm based on local light and shadow perception
- Journal of Northwest University (Natural Science Edition) Vol. 55, Issue 1, Pages: 193-200(2025)
- 作者机构：
  
  西北大学　信息科学与技术学院，陕西　西安　 710127
- 作者简介：
- 基金信息：
- DOI：10.16152/j.cnki.xdxbzr.2025-01-016
  CLC： TP391
- Published：25 February 2025，
  
  Received：20 April 2024，
- 稿件说明：
移动端阅览
DONG ZHIQIANG, XIAO YUN, DUAN JIASHUN. Calligraphy image harmonization algorithm based on local light and shadow perception. [J]. Journal of northwest university (natural science edition), 2025, 55(1): 193-200.
DOI：

DONG ZHIQIANG, XIAO YUN, DUAN JIASHUN. Calligraphy image harmonization algorithm based on local light and shadow perception. [J]. Journal of northwest university (natural science edition), 2025, 55(1): 193-200. DOI： 10.16152/j.cnki.xdxbzr.2025-01-016.

摘要

利用图像和谐化算法虚拟修复破损的书法图像，对于文物保护具有重要意义。现有的图像和谐化方法大多集中于解决前景和背景之间的不和谐问题，针对书法图像虚拟修复过程中出现的前景和背景图像视觉特征差异大、不和谐的问题，提出了一种基于局部光影感知的书法图像和谐化算法LSPNet。LSPNet通过引入参考掩膜和内容掩膜，对需要修复的区域进行精确的定位，从而确保合成的前景与背景在风格上保持一致。为了验证该算法的有效性，在破损的书法图像上进行了实验，经过实验和对比，LSPNet相比于其他算法能够明显降低前景与背景之间的亮度、对比度以及图像结构等方面的差异，使得修复后的书法作品在视觉上更加自然和统一。

Abstract

The virtual restoration of damaged calligraphy images using image harmonizati on algorithms is of great significance for cultural relic protection. Most existing image harmonization methods focus on addressing the disharmony between foreground and background. In response to the significant visual feature differences and disharmony issues between foreground and background during the virtual restoration of calligraphy images

this paper proposes a calligraphy image harmonization algorithm based on local light and shadow perception on LSPNet. LSPNet precisely locates the area to be restored by introducing reference masks and content masks

thereby ensuring that the style of the synthesized foreground and background remains consistent. To verify the effectiveness of this algorithm

experiments were conducted on damaged calligraphy images. Through numerous experiments and comparisons

LSPNet significantly outperforms other algorithms in reducing the differences in brightness

contrast

and image structure between the foreground and background

making the restored calligraphy works appear more visually natural and unified.

关键词

书法虚拟修复图像修复深度学习

Keywords

virtual restoration of calligraphy worksimage restorationdeep learning

references

耿国华, 冯龙, 李康, 等. 秦陵文物数字化及虚拟复原研究综述[J]. 西北大学学报(自然科学版), 2021, 51(5): 710-721.

GENG G H, FENG L, LI K, et al. A literature review on the digitization and virtual restoration of cultural relics in the Mausoleum of Emperor Qinshihuang[J]. Journal of Northwest University (Natural Science Edition), 2021, 51(5): 710-721.

XIAO Y, LEI W L, LU L, et al. CS-GAN: Cross-structure generative adversarial networksfor Chinese calligraphy translation[J]. Knowledge-Based Systems, 2021, 229: 107334.

张琦, 邢冠宇, 董哲镐, 等. 基于深度鉴伪的图像和谐化方法[J/OL]. 计算机辅助设计与图形学学报, 1-10[2024-04-15]. http://kns.cnki.net/kcms/detail/11.2925.TP.20240204.1345.031.htmlhttp://kns.cnki.net/kcms/detail/11.2925.TP.20240204.1345.031.html.

ZHANG Q, XING G Y, DONG Z H, et al. Image harmonization method based on deep forgery detection[J/OL]. Journal of Computer-Aided Design & Computer Graphics, 1-10. [2024-04-15]. https://kns.cnki.net/kcms/detail/11.2925.TP.20240204.1345.031.htmlhttps://kns.cnki.net/kcms/detail/11.2925.TP.20240204.1345.031.html.

GUERREIRO J J A, NAKAZAWA M, STENGER B. PCT-net: Full resolution image harmonization using pixel-wise color transformations[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver, Canada: IEEE, 2023: 5917-5926.

TAO X H, CAO J Y, HONG Y, et al. Shadow generation with decomposed mask prediction and attentive shadow filling[J]. Proceedings of the AAAI Conference on Artificial Intelligence. 2024, 38(6): 5198-5206.

HONG Y, NIU L, ZHANG J F. Shadow generation for composite image in real-world scenes[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(1): 914-922.

LING J, XUE H, SONG L, et al. Regionvaware adaptive instance normalization for image harmonization[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE, 2021: 9357-9366.

LIU D Q, LONG C J, ZHANG H P, et al. ARShadowGAN: Shadow generative adversarial network for augmented reality in single light scenes[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 8136-8145.

CONG W Y, TAO X H, NIU L, et al. High-resolution image harmonization via collaborative dual transformations[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans, USA: IEEE, 2022: 18449-18458.

TSAI Y H, SHEN X H, LIN Z, et al. Deep image harmonization[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, USA: IEEE, 2017: 2799-2807.

SOFIIUK K, POPENOVA P, KONUSHIN A. Foreground-aware semantic representations for image harmonization[C]//2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa, USA: IEEE, 2021: 1620-1629.

CONG W Y, NIU L, ZHANG J F, et al. Bargainnet: Background-guided domain translation for image harmonization[C]//2021 IEEE International Conference on Multimedia and Expo. Shenzhen, China: IEEE, 2021: 1-6.

CONG W Y, ZHANG J F, NIU L, et al. DoveNet: Deep image harmonization via domain verification[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 8394-8403.

CUN X D, PUN C M. Improving the harmony of the composite image by spatialseparated attention module[J]. IEEE Transactions on Image Processing, 2020, 29: 4759-4771.

HAO G, IIZUKA S, FUKUI K. Image harmonization with attention-based deep feature modulation[C]//The 31st British Machine Vision Virtual Conference. Manchester, UK: BMVA Press. 2020: 2.

CHEN B C, KAE A. Toward realistic image compositing with adversarial learning[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 8415-8424.

GUO Z, ZHENG H, JIANG Y, et al. Intrinsic image harmonization[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: 2021: IEEE, 16367-16376.

JIANG Y F, ZHANG H, ZHANG J M, et al. SSH: A self-supervised framework for image harmonization[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal, Canada: IEEE, 2021: 4812-4821.

HU Y M, HE H, XU C X, et al. Exposure: A white-box photo post-processing framework[J]. ACM Transactions on Graphics, 2018, 37(2): 1-17.

SHI J, XU N, XU Y H, et al. Learning by planning: Language-guided global image editing[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, USA: IEEE, 2021: 13585-13594.

GUO C L, LI C Y, GUO J C, et al. Zero-reference deep curve estimation for low-light image enhancement[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, USA: IEEE, 2020: 1777-1786.

CUN X D, PUN C M. Improving the harmony of the composite image by spatial-separated attention module[J]. IEEE Transactions on Image Processing, 2020: 4759-4771.

SARKAR S S, SHEIKH K H, MAHANTY A, et al. A harmony search-based wrapper-filter feature selection approach for microstructural image classification[J]. Integrating Materials and Manufacturing Innovation, 2021, 10: 1-19.

SRIKANTH R, BIKSHALU K. Multilevel thresholding image segmentation based on energy curve with harmony search algorithm[J]. Ain Shams Engineering Journal, 2021, 12(1): 1-20.

GUO Z H, GUO D S, ZHENG H Y, et al. Image harmonization with transformer[C]//2021 IEEE/CVF International Conference on Computer Vision. Montreal, Canada: IEEE, 2021: 14850-14859.

WANG L, HAMIDON N A. Aesthetic and value study of inscriptions from the perspective of calligraphy[J]. Art and Performance Letters, 2023, 4(11): 44-49.

BARNI M, BARTOLINI F, CAPPELLINI V. Image processing for virtual restoration of artworks[J]. IEEE MultiMedia, 2000, 7(2): 34-37.

LIU Z H. Literature review on image restoration[J]. Journal of Physics: Conference Series, 2022, 2386(1): 012041.

QIAO C Q, ZHANG W W, GONG D C, et al. In situ virtual restoration of artifacts by imaging technology[J]. Heritage Science, 2020, 8(1): 110.

PAN Z Q, YUAN F, LEI J J, et al. VCRNet: Visual compensation restoration network for No-reference image quality assessment[J]. IEEE Transactions on Image Processing, 2022, 31: 1613-1627.

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Restoration of traditional Chinese painting based on twin cascade spatial filtering

Large-scale dataset generation technology for oracle object detection

A survey of image inpainting methods based on deep learning

Prediction framework for drug-target interactions with multi-source descriptors fusion

AI-based assisted diagnostic methods for depression

Related Author

PENG Xianlin

CAO Rui

HU Qiyao

DONG Xingyu

XUE Wenzhe

MENG Ruiwei

LI Huabiao

YANG Fuyong

Related Institution

School of Art, Northwest University

Key Laboratory of Collection Resources Revitalising Technology, Ministry of Culture and Tourism

National Museum of China

Shaanxi Province Silk Road Digital Protection and Inheritance of Cultural Heritage Collaborative Innovation Center

School of Science, Jiangnan University

⁰