联合多视角特征的口罩遮挡人脸识别算法

苏雪平; 孙丹丹; 李云红; 姚丽娜; 任颖萱; 王灿

doi:10.16152/j.cnki.xdxbzr.2025-02-006

您当前的位置：

首页 >

文章列表页 >

联合多视角特征的口罩遮挡人脸识别算法

人工智能图像处理 | 更新时间：2025-04-21

- 联合多视角特征的口罩遮挡人脸识别算法
- Mask-occluded face recognition algorithm with integrated multi-view features
- 西北大学学报（自然科学版） 2025年55卷第2期页码：286-296
- 作者机构：
  
  1.西安工程大学　电子信息学院，陕西　西安　 710048
  2.西安电子科技大学　计算机科学与技术学院，陕西　西安　 710126
- 作者简介：
  
  苏雪平，博士，副教授，从事新闻人物标志、图像标注、人脸识别研究，yifeichongtian120@163.com。
- 基金信息：
  
  国家自然科学基金(61902301);陕西省自然科学基础研究计划(2024JC-YBMS-455);陕西省哲学社会科学研究专项青年项目(2024QN107);陕西高校青年创新团队
- DOI：10.16152/j.cnki.xdxbzr.2025-02-006
  中图分类号： TP391.41
- 收稿日期：2024-10-20，
  
  纸质出版日期：2025-04-25
- 稿件说明：
移动端阅览
苏雪平, 孙丹丹, 李云红, 等. 联合多视角特征的口罩遮挡人脸识别算法[J]. 西北大学学报（自然科学版）, 2025,55(2):286-296.

SU Xueping, SUN Dandan, LI Yunhong, et al. Mask-occluded face recognition algorithm with integrated multi-view features[J]. Journal of northwest university (natural science edition), 2025, 55(2): 286-296.
苏雪平, 孙丹丹, 李云红, 等. 联合多视角特征的口罩遮挡人脸识别算法[J]. 西北大学学报（自然科学版）, 2025,55(2):286-296. DOI： 10.16152/j.cnki.xdxbzr.2025-02-006.

SU Xueping, SUN Dandan, LI Yunhong, et al. Mask-occluded face recognition algorithm with integrated multi-view features[J]. Journal of northwest university (natural science edition), 2025, 55(2): 286-296. DOI： 10.16152/j.cnki.xdxbzr.2025-02-006.

摘要

佩戴口罩是预防呼吸道传染病最经济、最有效、最实用的防护措施，但也因此会降低人脸识别准确率。提出联合多视角特征的口罩遮挡人脸识别算法，首先，利用BoTNet作为骨干特征提取网络，提高识别准确率。其次，引入人脸注意力增强网络（face attention augmentation model，FAAM），先生成未被口罩遮挡人脸区域（眼睛、眉毛、额头）的蒙版图，再精准提取蒙版图区域的特征，提高人脸识别的性能。此外，设计联合损失函数

face

提高模型的收敛速度和性能。在包括大约10 000人的50×10

张人脸图片的公共遮挡人脸数据集上进行实验，相比其他算法，提出的算法识别准确率有显著提升，与经典算法FaceNet相比提升了13.9%。

Abstract

Wearing a mask is the most economical

effective and practical protective measure to prevent respiratory infectious diseases

but it also reduces the accuracy of face recognition. Therefore

a face recognition algorithm with mask occlusion combined with multi-view features is proposed. Firstly

BoTNet is used as the backbone feature extraction network to improve the recognition accuracy. Secondly

face attention augmentation model (FAAM) is introduced to generate a non-masked map of face areas (eyes

eyebrows and forehead) that are not covered by masks

and then the features of the masked map areas are accurately extracted to improve the performance of f

ace recognition. In addition

the joint loss function

face

is designed to improve the convergence speed and performance of the model. Experiments are carried out on a public occlusion face data set with 500 000 face images of about 10 000 people. Compared with other algorithms

the recognition accuracy of this method is significantly improved

and it is 13.9% higher than that of the classic algorithm FaceNet.

关键词

Keywords

references

ABBOAH-OFFEI M , SALIFU Y , ADEWALE B , et al . A rapid review of the use of face mask in preventing the spread of COVID-19 [J ] . International Journal of Nursing Studies Advances , 2021 , 3 : 100013 .

HOWARD J , HUANG A , LI Z Y , et al . An evidence review of face masks against COVID-19 [J ] . Proceedings of the National Academy of Sciences of the United States of America , 2021 , 118 ( 4 ): e2014564118 .

WALVEKAR S , SHINDE S . Efficient medical image segmentation of COVID-19 chest CT images based on deep learning techniques [C ] // 2021 International Conference on Emerging Smart Computing and Informatics (ESCI) . Pune : IEEE , 2021 : 203 - 206 .

刘瑞明 , 徐春融 , 周韬 . 局部遮挡的人脸识别方法研究综述 [J ] . 江苏海洋大学学报(自然科学版) , 2022 , 31 ( 3 ): 63 - 71 .

LIU R M , XU C R , ZHOU T . Overview of face recognition methods with partial occlusion [J ] . Journal of Jiangsu Ocean University (Natural Science Edition) , 2022 , 31 ( 3 ): 63 - 71 .

刘显梅 , 赵洪坡 . 基于Retinex算法的光照变化条件下人脸识别研究 [J ] . 激光杂志 , 2023 , 44 ( 7 ): 218 - 222 .

LIU X M , ZHAO H P . Research on face recognition under illumination change based on Retinex algorithm [J ] . Laser Journal , 2023 , 44 ( 7 ): 218 - 222 .

胡思佳 , 赵志诚 . 基于改进SRC的局部遮挡人脸识别方法 [J ] . 太原科技大学学报 , 2024 , 45 ( 1 ): 7 - 12 .

HU S J , ZHAO Z C . Local occlusion face recognition method based on improved SRC [J ] . Journal of Taiyuan University of Science and Technology , 2024 , 45 ( 1 ): 7 - 12 .

ORTIZ E G , BECKER B C . Face recognition for web-scale datasets [J ] . Computer Vision and Image Understanding , 2014 , 118 : 153 - 170 .

LEI Z , MENG Y , FENG X C . Sparse representation or collaborative representation: Which helps face recognition? [C ] // 2011 International Conference on Computer Vision . Barcelona : IEEE , 2011 : 471 - 478 .

李小薪 , 梁荣华 . 有遮挡人脸识别综述：从子空间回归到深度学习 [J ] . 计算机学报 , 2018 , 41 ( 1 ): 177 - 207 .

LI X X , LIANG R H . Overview of occluded face recognition: From subspace regression to deep learning [J ] . Chinese Journal of Computers , 2018 , 41 ( 1 ): 177 - 207 .

WAGNER A , WRIGHT J , GANESH A , et al . Toward a practical face recognition system: Robust alignment and illumination by sparse representation [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2012 , 34 ( 2 ): 372 - 386 .

雷燕 , 李杰 , 董博 , 等 . 基于深度学习特征的多模态人脸快速识别研究 [J ] . 电子设计工程 , 2024 , 32 ( 3 ): 181 - 184 .

LEI Y , LI J , DONG B , et al . Research on fast multi-modal face recognition based on deep learning features [J ] . Electronic Design Engineering , 2024 , 32 ( 3 ): 181 - 184 .

PENG Y L , LI L J , LIU S G , et al . A new virtual samples-based CRC method for face recognition [J ] . Neural Processing Letters , 2018 , 48 ( 1 ): 313 - 327 .

CAI S J , ZHANG L , ZUO W M , et al . A probabilistic collaborative representation based approach for pattern classification [C ] // 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) . Las Vegas, NV : IEEE , 2016 : 2950 - 2959 .

LAN R S , ZHOU Y C , LIU Z B , et al . Prior knowledge-based probabilistic collaborative representation for visual recognition [J ] . IEEE Transactions on Cybernetics , 2020 , 50 ( 4 ): 1498 - 1508 .

YUAN H L , LI X C , XU F Y , et al . A collaborative-competitive representation based classifier model [J ] . Neurocomputing , 2018 , 275 : 627 - 635 .

刘加聪 , 钟桂凤 , 盛儒妤 . 基于OpenCV和MTCNN算法的遮蔽物影响下的人脸识别研究 [J ] . 电脑编程技巧与维护 , 2023 ( 11 ): 120 - 124 .

LIU J C , ZHONG G F , SHENG R Y . Research on face recognition under the influence of masks based on OpenCV and MTCNN algorithms [J ] . Computer Programming Skills and Maintenance , 2023 ( 11 ): 120 - 124 .

张宣妮 , 马秀霞 , 鲁方莹 , 等 . 智慧小区的人脸识别门禁系统 [J ] . 信息系统工程 , 2021 ( 4 ): 95 - 96 .

ZHANG X N , MA X X , LU F Y , et al . Face recognition access control system in smart community [J ] . Information System Engineering , 2021 ( 4 ): 95 - 96 .

靳若华 , 白凡 , 张洪豪 . 基于多任务学习的特征融合跨年龄人脸识别研究 [J ] . 天津理工大学学报 , 2024 , 40 ( 1 ): 84 - 91 .

JIN R H , BAI F , ZHANG H H . Cross-age face recognition based on multi-task learning [J ] . Journal of Tianjin University of Technology , 2024 , 40 ( 1 ): 84 - 91 .

雷耀花 , 刘东宏 . 基于PCA+ICA人脸识别技术的考勤系统设计 [J ] . 电脑与电信 , 2017 ( 11 ): 55 - 57 .

LEI Y H , LIU D H . Design of attendance system based on PCA+ICA face recognition technology [J ] . Computer and Telecommunications , 2017 ( 11 ): 55 - 57 .

JIA H J , MARTINEZ A M . Support vector machines in face recognition with occlusions [C ] // 2009 IEEE Conference on Computer Vision and Pattern Recognition . Miami, FL : IEEE , 2009 : 136 - 141 .

李云红 , 刘杏瑞 , 谢蓉蓉 , 等 . 基于超分辨重建和公共特征子空间的低分辨率人脸识别 [J ] . 西北大学学报(自然科学版) , 2023 , 53 ( 2 ): 241 - 247 .

LI Y H , LIU X R , XIE R R , et al . Low-resolution face recognition based on super-resolution reconstruction and common feature subspace [J ] . Journal of Northwest University (Natural Science Edition) , 2023 , 53 ( 2 ): 241 - 247 .

罗亿 . 浅谈部分遮挡下的人脸识别技术研究 [J ] . 中国新通信 , 2023 , 25 ( 2 ): 67 - 68 .

LUO Y . On the research of face recognition technology under partial occlusion [J ] . China New Communication , 2023 , 25 ( 2 ): 67 - 68 .

SIRADJUDDIN I A , REYNALDI , MUNTASA A . Faster region-based convolutional neural network for mask face detection [C ] // 2021 5th International Conference on Informatics and Computational Sciences (ICICoS) . Semarang : IEEE , 2021 : 282 - 286 .

CHENG X Y , ZHANG L . Research on occluded face recognition method based on deep learning [J ] . Electronic Technology , 2022 , 35 ( 1 ): 35 - 39 .

VU H N , NGUYEN M H , PHAM C . Masked face recognition with convolutional neural networks and local binary patterns [J ] . Applied Intelligence , 2022 , 52 ( 5 ): 5497 - 5512 .

ZENG D , VELDHUIS R , SPREEUWERS L , et al . Occlusion-invariant face recognition using simultaneous segmentation [J ] . IET Biometrics , 2021 , 10 ( 6 ): 679 - 691 .

蒋文豪 . 基于生成对抗网络的多姿态人脸识别算法 [J ] . 信息技术与信息化 , 2024 ( 2 ): 188 - 191 .

JIANG W H . Multi-pose face recognition algorithm based on generated confrontation network [J ] . Information Technology and Informatization , 2024 ( 2 ): 188 - 191 .

王跃进 , 耿国华 , 郭沛瑶 , 等 . 基于对抗生成网络的复原面貌真实感处理 [J ] . 西北大学学报(自然科学版) , 2021 , 51 ( 5 ): 742 - 749 .

WANG Y J , GENG G H , GUO P Y , et al . Realistic processing of restored faces based on generative adversarial networks [J ] . Journal of Northwest University (Natural Science Edition) , 2021 , 51 ( 5 ): 742 - 749 .

DONG X B , MIAO Z H , MA L , et al . Reconstruct face from features based on genetic algorithm using GAN generator as a distribution constraint [J ] . Computers & Security , 2023 , 125 : 103026 .

ADLER J , LUNZ S . Banach Wasserstein Gan [EB/OL ] . ( 2018-06-18 )[ 2024-08-20 ] . https://arxiv.org/abs/1806.06621 https://arxiv.org/abs/1806.06621 .

XU J B , XU D G , WAN K , et al . Alleviating sample imbalance in water quality assessment using the VAE-WGAN-GP model [J ] . Water Science & Technology , 2023 , 88 ( 11 ): 2762 - 2778 .

WANG C G , CAO Y , ZHANG S , et al . A reconstruction method for missing data in power system measurement based on LSGAN [J ] . Frontiers in Energy Research , 2021 , 9 : 651807 .

WU Q F , CHEN Y P , MENG J . DCGAN-based data augmentation for tomato leaf disease identification [J ] . IEEE Access , 2020 , 8 : 98716 - 98728 .

QIAN H J , ZHANG P P , JI S J , et al . Improving representation consistency with pairwise loss for masked face recognition [C ] // 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) . Montreal, BC : IEEE , 2021 : 1462 - 1467 .

GUO B W , LI X Y , YANG M M , et al . A robust and lightweight deep attention multiple instance learning algorithm for predicting genetic alterations [J ] . Computerized Medical Imaging and Graphics , 2023 , 105 : 102189 .

JIANG N , GAO L , DUAN F X , et al . SAN: Attention-based social aggregation neural networks for recommendation system [J ] . International Journal of Intelligent Systems , 2022 , 37 ( 6 ): 3373 - 3393 .

XIAO C , IMEL E A , PHAM N , et al . Patient-GAT: Sarcopenia prediction using multi-modal data fusion and weighted graph attention networks [J ] . Proceedings of the Symposium on Applied Computing Symposium on Applied Computing , 2023 , 2023 : 614 - 617 .

WOO S , PARK J , LEE J Y , et al . CBAM: Convolutional block attention module [C ] // Computer Vision ECCV 2018 . Cham : Springer International Publishing , 2018 : 3 - 19 .

WANG W X , HU H F . Image captioning using region-based attention joint with time-varying attention [J ] . Neural Processing Letters , 2019 , 50 ( 1 ): 1005 - 1017 .

SRINIVAS A , LIN T Y , PARMAR N , et al . Bottleneck transformers for visual recognition [C ] // 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Nashville, TN : IEEE , 2021 : 16514 - 16524

JACOB M P . Comparison of popular face detection and recognition techniques [J ] . International Research Journal of Modernization in Engineering Technology and Science , 2021 : 2582 - 5208 .

SU X P , GAO M , REN J , et al . Face mask detection and classification via deep transfer learning [J ] . Multimedia Tools and Applications , 2022 , 81 ( 3 ): 4475 - 4494 .

HUANG Q Y , TANG C Y , ZHANG T S . Periocular biometric recognition for masked faces [J ] . Wuhan University Journal of Natural Sciences , 2023 , 28 ( 2 ): 141 - 149 .

王传传 , 高婕 . 基于改进FaceNet算法的人脸智能识别方法 [J ] . 工业控制计算机 , 2024 , 37 ( 2 ): 126 - 128 .

WANG C C , GAO J . Intelligent face recognition method based on improved FaceNet algorithm [J ] . Industrial Control Computer , 2024 , 37 ( 2 ): 126 - 128 .

DING F F , PENG P X , HUANG Y R , et al . Masked face recognition with latent part detection [C ] // Proceedings of the 28th ACM International Conference on Multimedia . Seattle WA : ACM , 2020 : 2281 - 2289 .

CAO Q , SHEN L , XIE W D , et al . VGGFace2: A dataset for recognising faces across pose and age [C ] // 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) . Xi'an : IEEE , 2018 : 67 - 74 .

WANG Z Y , WANG G C , HUANG B J , et al . Masked face recognition dataset and application [EB/OL ] . ( 2020-03-20 )[ 2024-09-10 ] . https://arxiv.org/abs/2003.09093 https://arxiv.org/abs/2003.09093 .

WANG H , WANG Y T , ZHOU Z , et al . CosFace: Large margin cosine loss for deep face recognition [C ] // 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Salt Lake City : IEEE , 2018 : 5265 - 5274 .

DENG J K , GUO J , XUE N N , et al . ArcFace: Additive angular margin loss for deep face recognition [C ] // 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Long Beach, CA : IEEE , 2019 : 4690 - 4699 .

SONG L X , GONG D H , LI Z F , et al . Occlusion robust face recognition based on mask learning with pairwise differential Siamese network [C ] // 2019 IEEE/CVF International Conference on Computer Vision (ICCV) . Seoul : IEEE , 2019 : 773 - 782 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据