融合一致性正则与流形正则的半监督深度学习算法

王杰; 张松岩; 梁吉业

doi:10.11959/j.issn.2096-0271.2022027

您当前的位置：

首页 >

文章列表页 >

融合一致性正则与流形正则的半监督深度学习算法

研究 | 更新时间：2024-06-03

- 融合一致性正则与流形正则的半监督深度学习算法
- A semi-supervised deep learning algorithm combining consistency regularization and manifold regularization
- 大数据 2022年8卷第3期页码：103-114
- 作者机构：
  
  1. 山西大学计算机与信息技术学院，山西太原 030006
  2. 计算智能与中文信息处理教育部重点实验室，山西太原 030006
- 作者简介：
  
  [ "王杰（1990- ），男，山西大学计算机与信息技术学院博士生，主要研究方向为数据挖掘和机器学习。" ]
  [ "张松岩（1996- ），男，山西大学计算机与信息技术学院硕士生，主要研究方向为数据挖掘和机器学习。" ]
  [ "梁吉业（1962- ），男，博士，山西大学计算机与信息技术学院教授，中国计算机学会会士，主要研究方向为数掘挖掘与机器学习，在国内外重要期刊和会议上发表学术论文200余篇。" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(61976184);山西省重点研发计划项目;The Key Project of Research and Development Plan of Shanxi Province(201903D121162);山西省1331工程项目;The 1331 Engineering Project of Shanxi Province
- DOI：10.11959/j.issn.2096-0271.2022027
  中图分类号： TP183
- 网络首发：2022-05，
  
  纸质出版：2022-05-15
- 稿件说明：
移动端阅览
王杰, 张松岩, 梁吉业. 融合一致性正则与流形正则的半监督深度学习算法[J]. 大数据, 2022,8(3):103-114.

Jie WANG, Songyan ZHANG, Jiye LIANG. A semi-supervised deep learning algorithm combining consistency regularization and manifold regularization[J]. Big data research, 2022, 8(3): 103-114.
王杰, 张松岩, 梁吉业. 融合一致性正则与流形正则的半监督深度学习算法[J]. 大数据, 2022,8(3):103-114. DOI： 10.11959/j.issn.2096-0271.2022027.

Jie WANG, Songyan ZHANG, Jiye LIANG. A semi-supervised deep learning algorithm combining consistency regularization and manifold regularization[J]. Big data research, 2022, 8(3): 103-114. DOI： 10.11959/j.issn.2096-0271.2022027.

摘要

半监督学习已被广泛应用于大数据分析。目前，基于一致性正则的方法是半监督深度学习的研究热点之一。然而这类方法没有考虑数据的流形结构，可能会导致部分相近的样本得到差异很大的输出，进而导致分类器性能下降。针对这个问题，提出了一种融合一致性正则与流形正则的半监督深度学习算法。该算法在对模型施加一致性约束的同时，对样本构图并加入平滑性损失，实现了每个样本点局部邻域的平滑以及邻近（相连）样本点之间的平滑，从而提高半监督深度学习算法的泛化性能。在多个图像和文本数据集上的实验结果表明，与其他的半监督深度学习算法相比，所提算法更有效。

Abstract

Semi-supervised learning has been widely used in big data analysis.Currently

one of the hot research topics in semisupervised deep learning is consistency-based methods.However

such methods do not take into account the manifold structure of the data

which may cause a portion of similar samples to get very different outputs

resulting in degraded classifier performance.To address this problem

a semi-supervised deep learning algorithm that combines consistency regularization with manifold regularization was proposed.The algorithm imposed a consistency constraint on the model while constructing a graph and adding a smoothing loss to achieve smoothing within the local neighborhood of each sample point and between adjacent (connected) sample points

thus improving the generalization performance of the semisupervised learning algorithm.The results on several image and text datasets show that the proposed algorithm is more effective compared with other semi-supervised deep learning algorithms.

关键词

Keywords

references

李国杰 . 对大数据的再认识 [J ] . 大数据 , 2015 , 1 ( 1 ): 1 - 9 .

LI G J . Further understanding of big data [J ] . Big Data Research , 2015 , 1 ( 1 ): 1 - 9 .

梁吉业 , 冯晨娇 , 宋鹏 . 大数据相关分析综述 [J ] . 计算机学报 , 2016 , 39 ( 1 ): 1 - 18 .

LIANG J Y , FENG C J , SONG P . A survey on correlation analysis of big data [J ] . Chinese Journal of Computers , 2016 , 39 ( 1 ): 1 - 18 .

胡湘红 , 彭衡 , 杨灿 , 等 . 基因大数据的集成分析 [J ] . 大数据 , 2019 , 5 ( 4 ): 67 - 88 .

HU X H , PENG H , YANG C , et al . Integrative analysis for big data in genomics [J ] . Big Data Research , 2019 , 5 ( 4 ): 67 - 88 .

杨孟辉 , 杜小勇 . 政府大数据治理:政府管理的新形态 [J ] . 大数据 , 2020 , 6 ( 2 ): 3 - 18 .

YANG M H , DU X Y . Big data governance in governments:a new form of the government administration [J ] . Big Data Research , 2020 , 6 ( 2 ): 3 - 18 .

刘晓波 , 蒋阳升 , 唐优华 . 综合交通大数据应用技术创新平台 [J ] . 大数据 , 2018 , 4 ( 6 ): 78 - 84 .

LIU X B , JIANG Y S , TANG Y H . Innovation platform of integrated transportation big data application technology [J ] . Big Data Research , 2018 , 4 ( 6 ): 78 - 84 .

CAO F L , YAO K X , LIANG J Y . Deconvolutional neural network for image super-resolution [J ] . Neural Networks , 2020 , 132 : 394 - 404 .

YAO K X , CAO F L , LEUNG Y , et al . Deep neural network compression through interpretability-based filter pruning [J ] . Pattern Recognition , 2021 ,119:108056.

刘建伟 , 刘媛 , 罗雄麟 . 半监督学习方法 [J ] . 计算机学报 , 2015 , 38 ( 8 ): 1592 - 1617 .

LIU J W , LIU Y , LUO X L . Semi-supervised learning methods [J ] . Chinese Journal of Computers , 2015 , 38 ( 8 ): 1592 - 1617 .

BERTHELOT D , CARLINI N , GOODFELLOW I , et al . MixMatch:a holistic approach to semi-supervised learning [C ] // Proceedings of the 33rd International Conference on Neural Information Processing Systems . Cambridge:The MIT Press , 2019 : 5049 - 5059 .

LAINE S , AILA T M . Temporal ensembling for semi-supervised learning [C ] // Proceedings of the 5th International Conference on Learning Representations .[S.l.:s.n. ] , 2017 .

TARVAINEN A , VALPOLA H . Mean teachers are better role models:weight-averaged consistency targets improve semi-supervised deep learning results [C ] // Proceedings of the 31st Conference on Neural Information Processing Systems . Cambridge:The MIT Press , 2017 : 1195 - 1204 .

MIYATO T , MAEDA S I , KOYAMA M , et al . Virtual adversarial training:a regularization method for supervised and semi-supervised learning [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2019 , 41 ( 8 ): 1979 - 1993 .

VERMA V , KANNALA J , LAMB A , et al . Interpolation consistency training for semi-supervised learning [J ] . Neural Networks , 2022 , 145 : 90 - 106 .

XIE Q Z , DAI Z H , HOVY E , et al . Unsupervised data augmentation for consistency training [C ] // Proceedings of the 30th Advances in Neural Information Processing Systems . Cambridge:The MIT Press , 2020 : 6256 - 6268 .

VAN ENGELEN J E , HOOS H H . A survey on semi-supervised learning [J ] . Machine Learning , 2020 , 109 ( 2 ): 373 - 440 .

ZHU X J , GHAHRAMANI Z , LAFFERTY J D . Semi-supervised learning using gaussian fields and harmonic functions [C ] // Proceedings of the 20th International Conference on Machine Learning . Palo Alto:AAAI Press , 2003 : 912 - 919 .

BELKIN M , NIYOGI P , SINDHWANI V . Manifold regularization:a geometric framework for learning from labeled and unlabeled examples [J ] . Journal of Machine Learning Research , 2006 , 7 : 2399 - 2434 .

BAI L , WANG J B , LIANG J Y , et al . New label propagation algorithm with pairwise constraints [J ] . Pattern Recognition , 2020 ,106:107411.

WANG J , LIANG J Q , CUI J B , et al . Semi-supervised learning with mixedorder graph convolutional networks [J ] . Information Sciences , 2021 , 573 : 171 - 181 .

LIANG J Y , CUI J B , WANG J , et al . Graph-based semi-supervised learning via improving the quality of the graph dynamically [J ] . Machine Learning , 2021 , 110 ( 6 ): 1345 - 1388 .

JOACHIMS T , . Transductive inference for text classification using support vector machines [C ] // Proceedings of the 16th International Conference on Machine Learning . Bled:Morgan Kaufmann , 1999 : 200 - 209 .

BLUM A , MITCHELL T . Combining labeled and unlabeled data with cotraining [C ] // Proceedings of the 11th Annual Conference on Computational Learning Theory . New York:ACM Press , 1998 : 92 - 100 .

WANG W , ZHOU Z H . A new analysis of co-training [C ] // Proceedings of the 27th International Conference on Machine Learning . Haifa:Omnipress , 2010 : 1135 - 1142 .

LOOG M . Contrastive pessimistic likelihood estimation for semi-supervised classification [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2016 , 38 ( 3 ): 462 - 475 .

SHAHSHAHANI B M , LANDGREBE D A . The effect of unlabeled samples in reducing the small sample size problem and mitigating the Hughes phenomenon [J ] . IEEE Transactions on Geoscience and Remote Sensing , 1994 , 32 ( 5 ): 1087 - 1095 .

BAI L , LIANG J Y , CAO F Y . Semisupervised clustering with constraints of different types from multiple information sources [J ] . IEEE Transactions on Pattern Analysis and Machine Intelligence , 2021 , 43 ( 9 ): 3247 - 3258 .

LI Y F , ZHA H W , ZHOU Z H . Learning safe prediction for semi-supervised regression [C ] // Proceedings of the 31st AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2017 : 2217 - 2223 .

OLIVER A , ODENA A , RAFFEL C , et al . Realistic evaluation of deep semisupervised learning algorithms [C ] // Proceedings of the 32nd International Conference on Neural Information Processing Systems . Cambridge:The MIT Press , 2018 : 3239 - 3250 .

DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [C ] // Proceedings of the 13th Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies . Minneapolis:Association for Computational Linguistics , 2019 : 4171 - 4186 .

浏览量

377

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据