3DTGC: Text generation of Chinese painting elements based on 3D reconstruction

HU Qiyao; WANG Jingyu; LI Tengfei; PENG Xianlin; CAO Rui; ZHU Xuan

doi:10.16152/j.cnki.xdxbzr.2026-02-014

您当前的位置：

首页 >

文章列表页 >

3DTGC: Text generation of Chinese painting elements based on 3D reconstruction

Cultural heritage conservation | 更新时间：2026-05-11

- 3DTGC: Text generation of Chinese painting elements based on 3D reconstruction
- Journal of Northwest University (Natural Science Edition) Vol. 56, Issue 2, Pages: 370-384(2026)
- 作者机构：
  
  1.西北大学　电子信息学院，陕西　西安　710127
  2.新型网络智能信息服务国家地方联合工程研究中心，陕西　西安　710127
  3.陕西省高校生成式人工智能与混合现实重点实验室，陕西　西安　710127
  4.西北大学　艺术学院，陕西　西安　710127
  5.陕西省丝绸之路文化遗产数字化保护与传承协同创新中心，陕西　西安　710127
  6.西北大学　计算机学院（软件学院），陕西　西安　710127
- 作者简介：
- 基金信息：
- DOI：10.16152/j.cnki.xdxbzr.2026-02-014
  CLC： TP391.41
- Received：20 November 2025，
  
  Revised：2026-01-14，
  
  Published：25 April 2026
- 稿件说明：
移动端阅览
HU Qiyao, WANG Jingyu, LI Tengfei, et al. 3DTGC: Text generation of Chinese painting elements based on 3D reconstruction[J]. Journal of Northwest University (Natural Science Edition), 2026, 56(2): 370-384.
DOI：

HU Qiyao, WANG Jingyu, LI Tengfei, et al. 3DTGC: Text generation of Chinese painting elements based on 3D reconstruction[J]. Journal of Northwest University (Natural Science Edition), 2026, 56(2): 370-384. DOI： 10.16152/j.cnki.xdxbzr.2026-02-014.

摘要

在无法直接获取多视图信息的情况下，针对中国画的3D数据以及对其进行3D重建的研究较少，因此，提出了一种名为3DTGC的创新方法，用于文字生成中国画元素的3D重建。首先，文生图模块用来从文字生成中国画元素图像。然后，多视图生成模块负责多视图合成。通过一系列预处理步骤，该模块从单幅图像创建6个固定视角的图像，这些图像被用于生成局部光场融合数据集。接下来，生成的局部光场融合数据集被输入到神经辐射场合成模块中，用于进一步优化3D信息的表达。最后，在网格生成模块中，基于前两个步骤的结果创建详细的网格结构。与当前最先进的3D重建方法相比，该框架在可视化效果和技术性能上均表现出显著优势。此外，3DTGC能够有效解决其他算法在处理传统中国画数据时常见的多面问题，从而提升了重建质量和鲁棒性。

Abstract

Given the limited availability of multi-view information

research on 3D data and 3D reconstruction of Chinese paintings is scarce. Therefore

this paper proposes an innovative method called 3DTGC for the 3D reconstruction of Chinese painting elements generated from text. First

a text-to-image module generates images of Chinese painting elements from text. Then

a multi-view generation module handles multi-view synthesis. Through a series of preprocessing steps

this module creates six fixed-view images from a single image

which are used to generate a local light field fusion dataset. Next

the generated local light field fusion dataset is input into a neural radiation field synthesis module for further optimization of 3D information representation. Finally

a detailed mesh structure is created in the mesh generation module based on the results of the first two steps. Compared with several state-of-the-art 3D reconstruction methods

this framework demonstrates significant advantages in both visualization and technical performance. Furthermore

3DTGC effectively addresses the multifaceted problem commonly encountered by other algorithms when processing traditional Chinese painting data

thereby improving reconstruction quality and robustness.

关键词

Keywords

references

徐锦宁 , 温超 , 耿国华 , 等 . 面向文化遗产知识传播的百戏俑VR体验系统 [J ] . 西北大学学报(自然科学版) , 2025 , 55 ( 1 ): 85 - 97 .

XU J N , WEN C , GENG G H , et al . VR experience system for Baixi terracotta warrior oriented to cultural heritage knowledge dissemination [J ] . Journal of Northwest University (Natural Science Edition) , 2025 , 55 ( 1 ): 85 - 97 .

DEITKE M , SCHWENK D , SALVADOR J , et al . Objaverse: A universe of annotated 3D objects [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Vancouver : IEEE , 2023 : 13142 - 13153 .

DEITKE M , LIU R , WALLINGFORD M , et al . Objaverse-xl: A universe of 10m + 3D objects [C ] // Advances in Neural Information Processing Systems . New Orleans : NeurIPS , 2023 : 35799 - 35813 .

CHANG A X , FUNKHOUSER T , GUIBAS L , et al . ShapeNet: An information-rich 3D model repository [EB/OL ] . ( 2015-12-09 ) [ 2025-06-30 ] . https://arxiv.org/abs/1512.03012 https://arxiv.org/abs/1512.03012 .

POOLE B , JAIN A , BARRON J T , et al . Dreamfusion: Text-to-3Dusing2Ddiffusion [EB/OL ] . ( 2022-09-29 ) [ 2025-06-30 ] . https://arxiv.org/abs/2209.14988 https://arxiv.org/abs/2209.14988 .

LIU R , WU R , HOORICK B V , et al . Zero-1-to-3: Zero-shot one image to 3D object [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Paris : IEEE , 2023 : 9264 - 9275 .

TAN F , FENG X Y , MA Y P , et al . Two-and three-dimensional deep human detection by generating orthographic top view image from dense point cloud [J ] . Journal of Electronic Imaging , 2022 , 31 ( 3 ): 033009 .

朱耀麟 , 刘雅琪 , 杨宇峤 . 基于单幅图片的唐代服饰三维重建 [J ] . 西北大学学报(自然科学版) , 2015 , 45 ( 3 ): 379 - 383 .

ZHU Y L , LIU Y Q , YANG Y Q . 3D reconstruction of Tang Dynasty costume based on single image [J ] . Journal of Northwest University (Natural Science Edition) , 2015 , 45 ( 3 ): 379 - 383 .

王峥嵘 , 刘鑫达 , 周明全 . 融合扩散模型技术的文物面部三维模型孔洞修补 [J ] . 西北大学学报(自然科学版) , 2025 , 55 ( 1 ): 213 - 222 .

WANG Z R , LIU X D , ZHOU M Q . Hole repair of 3D cultural relic facial model integrating diffusion model technology [J ] . Journal of Northwest University (Natural Science Edition) , 2025 , 55 ( 1 ): 213 - 222 .

周宇航 , 冯宏伟 , 冯筠 , 等 . 基于多视图和显著性分割的古生物三维模型检索 [J ] . 西北大学学报(自然科学版) , 2022 , 52 ( 4 ): 590 - 601 .

ZHOU Y H , FENG H W , FENG J , et al . 3D ancient organism model retrieval based on multi-view and saliency segmentation [J ] . Journal of Northwest University (Natural Science Edition) , 2022 , 52 ( 4 ): 590 - 601 .

XU J L , CHENG W H , GAO Y M , et al . Instantmesh: Efficient 3D mesh generation from a single image with sparse-view large reconstruction models [EB/OL ] . ( 2024-04-14 ) [ 2025-06-30 ] . https://arxiv.org/html/2404.07191v2 https://arxiv.org/html/2404.07191v2 .

LIU M , SHI R , CHEN L , et al . One-2-3-45: Any single image to 3D mesh in 45 seconds without per-shape optimization [C ] // Advances in Neural Information Processing Systems . New Orleans : NeurIPS , 2023 : 22226 - 22246 .

TANG J , WANG T , ZHANG B , et al . Make-it-3D: High-fidelity 3D creation from a single image with diffusion prior [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Paris : IEEE , 2023 : 22762 - 22772 .

TOCHILKIN D , PANKRATZ D , LIU Z . et al . TripoSR: Fast 3D object reconstruction from a single image [EB/OL ] . ( 2024-03-04 ) [ 2025-06-30 ] . https://arxiv.org/abs/2403.02151 https://arxiv.org/abs/2403.02151 .

HONG Y C , ZHANG K , GU J X , et al . LRM: Large reconstruction model for single image to 3D [EB/OL ] . ( 2023-11-08 ) [ 2025-06-30 ] . https://arxiv.org/abs/2311.04400 https://arxiv.org/abs/2311.04400 .

TANG J X , REN J W , ZHOU H , et al . DreamGaussian: Generative gaussian splatting for efficient 3D content creation [EB/OL ] . ( 2023-09-28 ) [ 2025-06-30 ] . https://arxiv.org/abs/2309.16653 https://arxiv.org/abs/2309.16653 .

KEMELMACHER-SHLIZERMAN I . Internet based morphable model [C ] // 2013 IEEE International Conference on Computer Vision . Sydney : IEEE , 2013 : 3256 - 3263 .

YANG Y , FENG C , SHEN Y , et al . FoldingNet: Point cloud autoencoder via deep grid deformation [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 206 - 215 .

WANG N , ZHANG Y , LI Z , et al . Pixel2mesh: 3D mesh model generation via image guided deformation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 10 ): 3600 - 3613 .

RAKOTOSAONA M J , MANHARDT F , ARROYO D M , et al . NeRFMeshing: Distilling neural radiance fields into geometrically-accurate 3D meshes [C ] // 2024 International Conference on 3D Vision (3DV) . Davos : IEEE , 2024 : 1156 - 1165 .

SMITH E , FUJIMOTO S , ROMERO A , et al . Geometrics: Exploiting geometric structure for graph-encoded objects [C ] // Proceedings of Machine Learning Research (PMLR) . Long Beach : IEEE , 2019 : 5866 - 5876 .

WANG H , DU X , LI J , et al . Score Jacobian chaining: Lifting pretrained 2D diffusion models for 3D generation [C ] // 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Vancouver : IEEE , 2023 : 12619 - 12629 .

GUO S , WANG Q , GAO Y , et al . Depth-guided robust point cloud fusion NeRF for sparse input views [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2024 , 34 ( 9 ): 8093 - 8106 .

KIRILLOV A , MINTUN E , RAVI N , et al . Segment anything [C ] // 2023 IEEE/CVF International Conference on Computer Vision (ICCV) . Paris : IEEE , 2023 : 3992 - 4003 .

NIEMEYER M , BARRON J T , MILDENHALL B , et al . RegNeRF: Regularizing neural radiance fields for view synthesis from sparse inputs [C ] // 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . New Orleans : IEEE , 2022 : 5470 - 5480 .

PENG X , HU Q , FAN F , et al . FHS-adapter: finegrained hierarchical semantic adapter for Chinese landscape paintings generation [J ] . Heritage Science , 2024 , 12 : 265 .

MILDENHALL B , SRINIVASAN P P , CAYON R O , et al . Local light field fusion [J ] . ACM Transactions on Graphics (TOG) , 2019 , 38 ( 4 ): 1 - 14 .

SHI R , WEI X , WANG C , et al . ZeroRF: Fast sparse view 360° reconstruction with zero pretraining [C ] // 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Seattle, WA, USA : IEEE , 2024 : 21114 - 21124 .

TANG J X , ZHOU H , CHEN X K , et al . Delicate textured mesh recovery from NeRF via adaptive surface refinement [C ] // Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) . Paris : IEEE , 2023 : 17739 - 17749 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

3D model hole filling in cultural relics' facial reconstruction fusion diffusion model

Related Author

WANG Zhengrong

LIU Xinda

ZHOU Mingquan

Related Institution

National and Local Joint Engineering Research Center for Cultural Heritage Digitization, Northwest University

Institute of Visualization Technology, Northwest University

⁰