No-reference super-resolution image quality assessment based on upscaling-factor aware contrastive learning

FAN Dandan; ZHANG Kaibing; QUAN Xing; MENG Yalei

doi:10.16152/j.cnki.xdxbzr.2025-02-008

您当前的位置：

首页 >

文章列表页 >

No-reference super-resolution image quality assessment based on upscaling-factor aware contrastive learning

更新时间：2025-04-21

- No-reference super-resolution image quality assessment based on upscaling-factor aware contrastive learning
- Journal of Northwest University (Natural Science Edition) Vol. 55, Issue 2, Pages: 309-319(2025)
- 作者机构：
  
  1.西安工程大学　电子信息学院，陕西　西安　 710048
  2.西安工程大学　计算机科学学院，陕西　西安　 710048
  3.陕西省服装设计智能化重点实验室，陕西　西安　 710048
- 作者简介：
- 基金信息：
- DOI：10.16152/j.cnki.xdxbzr.2025-02-008
  CLC： TP391.41
- Received：18 October 2024，
  
  Published：25 April 2025
- 稿件说明：
移动端阅览
FAN Dandan, ZHANG Kaibing, QUAN Xing, et al. No-reference super-resolution image quality assessment based on upscaling-factor aware contrastive learning[J]. Journal of northwest university (natural science edition), 2025, 55(2): 309-319.
DOI：

FAN Dandan, ZHANG Kaibing, QUAN Xing, et al. No-reference super-resolution image quality assessment based on upscaling-factor aware contrastive learning[J]. Journal of northwest university (natural science edition), 2025, 55(2): 309-319. DOI： 10.16152/j.cnki.xdxbzr.2025-02-008.

摘要

超分辨图像的质量不仅受重建算法的影响，而且在不同的尺度因子下重建出的图像在质量退化等级方面存在一定差异。然而现有的无参考型超分辨图像质量评价方法主要关注超分辨率图像的视觉特征，忽略了可用的尺度因子信息。提出了一种尺度因子感知对比学习（upscaling-factor aware contrastive learning，UFACL）方法，该网络结构分为尺度因子识别分支和质量分数分支。其中尺度因子识别分支从数据集本身出发，将不同尺度因子的超分辨图像作为彼此的正负样本，在完成分类任务的同时引入对比学习，提高有效特征的表达能力。在质量分数分支设计了一个频域注意模块（frequency domain attention module，FDAM），考虑了全局信息和通道信息，同时，该分支使用倒残差块（inverted residuals blocks，IRB）降低模型的计算量，使得在训练过程中既保证了质量分数预测精度又提升了模型训练效率。实验结果表明，提出的UFACL能够获得与主观感知质量更好的一致性。

Abstract

The quality of super-resolution images is not only affected by the reconstruction algorithm

but also there are some differences in the quality degradation levels of the reconstructed images under different upscaling-factors. However

the existing no-reference super-resolution image quality assessment (NR-SRIQA) methods mainly focus on the visual features of super-resolution images

ignoring the available upscaling-factor information. An upscaling-factor aware contrastive learning (UFACL) method is proposed. The network structure is divided into a upscaling-factor recognition branch and a quality score branch. The upscaling-factor recognition branch starts from the dataset

and takes the super-resolution images of different upscaling-factors as positive and negative samples of each other. Contrastive learning is introduced to complete the classification task

so as to improve the expression ability of effective features. In the quality score branch

a frequency domain attention module (FDAM) is designed

which considers both global information and channel information. At the same time

this branch uses inverted residuals blocks (IRB) to reduce the calculation amount of the model

which ensures the accuracy of quality score prediction and improves the training efficiency of the model in the training process. Experimental results show that the proposed UFACL can achieve better consistency with subjective perceived quality.

关键词

Keywords

references

李云红 , 王珍 , 张凯兵 , 等 . 基于学习的图像超分辨重建方法综述 [J ] . 计算机工程与应用 , 2018 , 54 ( 15 ): 13 - 21 .

LI Y H , WANG Z , ZHANG K B , et al . Survey on example learning-based single image super-resolution technique [J ] . Computer Engineering and Applications , 2018 , 54 ( 15 ): 13 - 21 .

LI K , YANG S H , DONG R T , et al . Survey of single image super-resolution reconstruction [J ] . IET Image Processing , 2020 , 14 ( 11 ): 2273 - 2290 .

崔琛 , 张凯兵 . 基于双字典正则化的单帧图像超分辨率重建方法 [J ] . 西安工程大学学报 , 2021 , 35 ( 2 ): 66 - 72 .

CUI C , ZHANG K B . Learning dual dictionary regularization for single image super-resolution [J ] . Journal of Xi'an Polytechnic University , 2021 , 35 ( 2 ): 66 - 72 .

张凯兵 , 朱丹妮 , 王珍 , 等 . 超分辨图像质量评价综述 [J ] . 计算机工程与应用 , 2019 , 55 ( 4 ): 31 - 40 .

ZHANG K B , ZHU D N , WANG Z , et al . Survey of super-resolution images quality assessment [J ] . Computer Engineering and Applications , 2019 , 55 ( 4 ): 31 - 40 .

MANTIUK R K , TOMASZEWSKA A , MANTIUK R . Comparison of four subjective methods for image quality assessment [J ] . Computer Graphics Forum , 2012 , 31 ( 8 ): 2478 - 2491 .

HE L H , GAO F , HOU W L , et al . Objective image quality assessment: A survey [J ] . International Journal of Computer Mathematics , 2014 , 91 ( 11 ): 2374 - 2388 .

张婷悦 , 张凯兵 . 基于稀疏表示的无参考型超分辨图像质量评价方法 [J ] . 西安工程大学学报 , 2020 , 34 ( 5 ): 20 - 26 .

ZHANG T Y , ZHANG K B . No-reference super-resolution image quality assessment based on sparse representation [J ] . Journal of Xi'an Polytechnic University , 2020 , 34 ( 5 ): 20 - 26 .

ZHANG L , ZHANG L , BOVIK A C . A feature-enriched completely blind image quality evaluator [J ] . IEEE Transactions on Image Processing , 2015 , 24 ( 8 ): 2579 - 2591 .

YE P , KUMAR J , DOERMANN D . Beyond human opinion scores: Blind image quality assessment based on synthetic scores [C ] // 2014 IEEE Conference on Computer Vision and Pattern Recognition . Columbus, OH : IEEE , 2014 : 4241 - 4248 .

GU K , ZHAI G T , YANG X K , et al . Hybrid no-reference quality metric for singly and multiply distorted images [J ] . IEEE Transactions on Broadcasting , 2014 , 60 ( 3 ): 555 - 567 .

YANG J , LYU M J , QI Z Q , et al . Deep learning based image quality assessment: A survey [J ] . Procedia Computer Science , 2023 , 221 : 1000 - 1005 .

LIU X L , VAN DE WEIJER J , BAGDANOV A D . RankIQA: Learning from rankings for no-reference image quality assessment [C ] // 2017 IEEE International Conference on Computer Vision . Venice : IEEE , 2017 : 1040 - 1049 .

艾娜 , 彭进业 , 王珺 , 等 . 基于低分辨率图像自举和双重稀疏性字典训练的单幅图像超分辨率重构 [J ] . 西北大学学报(自然科学版) , 2018 , 48 ( 1 ): 57 - 65 .

AI N , PENG J Y , WANG J , et al . Single image super-resolution based on bootstrapping method combining learned double sparsity dictionaries [J ] . Journal of Northwest University (Natural Science Edition) , 2018 , 48 ( 1 ): 57 - 65 .

BOURBIA S , KARINE A , CHETOUANI A , et al . A multi-task convolutional neural network for blind stereoscopic image quality assessment using naturalness analysis [C ] // 2021 IEEE International Conference on Image Processing (ICIP) . Anchorage : IEEE , 2021 : 1434 - 1438 .

LI F , ZHANG Y F , COSMAN P C . MMMNet: An end-to-end multi-task deep convolution neural network with multi-scale and multi-hierarchy fusion for blind image quality assessment [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2021 , 31 ( 12 ): 4798 - 4811 .

李云红 , 刘杏瑞 , 谢蓉蓉 , 等 . 基于超分辨重建和公共特征子空间的低分辨率人脸识别 [J ] . 西北大学学报(自然科学版) , 2023 , 53 ( 2 ): 241 - 247 .

LI Y H , LIU X R , XIE R R , et al . Low-resolution face recognition based on super-resolution reconstruction and common feature subspace [J ] . Journal of Northwest University (Natural Science Edition) , 2023 , 53 ( 2 ): 241 - 247 .

FU J . Scale guided hypernetwork for blind super-resolution image quality assessment [EB/OL ] . ( 2023-06-04 ) [ 2024-09-20 ] . https://arxiv.org/abs/2306.02398v1 https://arxiv.org/abs/2306.02398v1 .

SANDLER M , HOWARD A , ZHU M L , et al . MobileNetV2: Inverted residuals and linear bottlenecks [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT : IEEE , 2018 : 4510 - 4520 .

ZHANG H N , MENG F , HAN Y W . No-reference image quality assessment based on a multi-feature extraction network [C ] // Proceedings of the 2020 2nd International Conference on Image, Video and Signal Processing . Singapore : ACM , 2020 : 81 - 85 .

WANG T , ZHANG T , ZHANG K B , et al . Context attention fusion network for crowd counting [J ] . Knowledge-Based Systems , 2023 , 271 : 110541 .

SAAD M A , BOVIK A C , CHARRIER C . DCT statistics model-based blind image quality assessment [C ] // 2011 18th IEEE International Conference on Image Processing . Brussels : IEEE , 2011 : 3093 - 3096 .

LIU L X , LIU B , HUANG H , et al . No-reference image quality assessment based on spatial and spectral entropies [J ] . Signal Processing: Image Communication , 2014 , 29 ( 8 ): 856 - 863 .

KANG L , YE P , LI Y , et al . Simultaneous estimation of image quality and distortion via multi-taskconvolutional neural networks [C ] // 2015 IEEE International Conference on Image Processing (ICIP) . Quebec City : IEEE , 2015 : 2791 - 2795 .

KANG L , YE P , LI Y , et al . Convolutional neural networks for no-reference image quality assessment [C ] // 2014 IEEE Conference on Computer Vision and Pattern Recognition . Columbus, OH : IEEE , 2014 : 1733 - 1740 .

YAN Q S , GONG D , ZHANG Y N . Two-stream convolutional networks for blind image quality assessment [J ] . IEEE Transactions on Image Processing , 2019 , 28 ( 5 ): 2200 - 2211 .

ZHANG W X , MA K D , YAN J , et al . Blind image quality assessment using a deep bilinear convolutionalneural network [J ] . IEEE Transactions on Circuits and Systems for Video Technology , 2020 , 30 ( 1 ): 36 - 47 .

SU S L , YAN Q S , ZHU Y , et al . Blindly assess image quality in the wild guided by a self-adaptive hyper network [C ] // 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Seattle, WA : IEEE , 2020 : 3667 - 3676 .

LI H , ZHANG K B , NIU Z X , et al . C 2 MT: A credible and class-aware multi-task transformer for SR-IQA [J ] . IEEE Signal Processing Letters , 2022 , 29 : 2662 - 2666 .

QUAN X , ZHANG K B , LI H , et al . TADSRNet: A triple-attention dual-scale residual network for super-resolution image quality assessment [J ] . Applied Intelligence , 2023 , 53 ( 22 ): 26708 - 26724 .

ZHANG X Y , ZHOU X Y , LIN M X , et al . ShuffleNet: An extremely efficient convolutional neural network for mobile devices [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City, UT : IEEE , 2018 : 6848 - 6856 .

HOWARD A G , ZHU M L , CHEN B , et al . MobileNets: Efficient convolutional neural networks for mobile vision applications [EB/OL ] . ( 2017-04-17 ) [ 2024-09-20 ] . https://arxiv.org/abs/1704.04861v1 https://arxiv.org/abs/1704.04861v1 .

SHI G M , WAN W F , WU J J , et al . SISRSet: Single image super-resolution subjective evaluation test and objective quality assessment [J ] . Neurocomputing , 2019 , 360 : 37 - 51 .

ZHANG T Y , ZHANG K B , XIAO C , et al . Joint channel-spatial attention network for super-resolution image quality assessment [J ] . Applied Intelligence , 2022 , 52 ( 15 ): 17118 - 17132 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Ceramic microscopic image classification based on the combination of contrastive learning and multi-scale methods

Related Author

Guohua GENG

Miyan XUE

Pengbo ZHOU

Dongcheng TUO

Xingrui MA

Xiaoning LIU

Related Institution

National and Local Joint Engineering Research Center for Cultural Heritage Digitization, Northwest University

College of Information Science and Technology, Northwest University

School of Arts and Communication, Beijing Normal University

⁰