Research on the application of government big data platform based on federated learning

Jianping WU; Chaochao CHEN; Jiahe JIN; Chunming WU

doi:10.11959/j.issn.2096-0271.2024032

您当前的位置：

首页 >

文章列表页 >

Research on the application of government big data platform based on federated learning

TOPIC: GOVERNMENT DATA PROCESSING | 更新时间：2024-06-03

- Research on the application of government big data platform based on federated learning
- Big data research Vol. 10, Issue 3, Pages: 40-54(2024)
- 作者机构：
  
  1. 浙江大学计算机科学与技术学院，浙江杭州 310027
  2. 浙江省大数据发展中心，浙江杭州 310007
  3. 浙江省数据开放融合关键技术研究重点实验室，浙江杭州 310007
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2024032
  CLC： TP181
- Online First：2024-05，
  
  Published：15 May 2024
- 稿件说明：
移动端阅览
Jianping WU, Chaochao CHEN, Jiahe JIN, et al. Research on the application of government big data platform based on federated learning[J]. Big data research, 2024, 10(3): 40-54.
DOI：

Jianping WU, Chaochao CHEN, Jiahe JIN, et al. Research on the application of government big data platform based on federated learning[J]. Big data research, 2024, 10(3): 40-54. DOI： 10.11959/j.issn.2096-0271.2024032.

摘要

当前数字政府建设已进入深水区，政务大数据平台作为数据底座支撑各类政务信息化应用，其隐私数据的安全性和合规性一直被业界广泛关注。联邦学习是一类解决数据孤岛的重要方法，基于联邦学习的政务一体化大数据平台应用具有较高的研究价值。首先，介绍政务大数据平台及联邦学习应用现状；然后，分析政务大数据平台面临的隐私数据的采集、分类分级、共享三大管理挑战；接着，阐述基于联邦学习的推荐算法和隐私集合求交技术的解决方法；最后，对政务大数据平台隐私数据的未来应用进行了总结和展望。

Abstract

At present

the construction of digital government has entered a deepwater area.The government big data platform

as a data base

supports various government information applications.The security and compliance of its private data has been widely concerned by the industry.Federated learning is an important method to effectively solve data silos

and the application of government big data platforms based on federated learning has high research value.Firstly

the current status of government big data platforms and its federated learning application were introduced.Then this paper analyzed three major management challenges involved in the collection

classification and grading and sharing of privacy data on government big data platforms.Further

the problem-solving methods of federated learning based recommendation algorithms and privacy intersection techniques were explored.Finally

summaries and prospects were made for the future application of privacy data on government big data platforms.

关键词

Keywords

references

国务院办公厅 . 关于印发全国一体化政务大数据体系建设指南的通知 [Z ] . 2022 .

General Office of the State Council . Notice on printing and issuing the guidelines for the construction of the national integrated government big data system [Z ] . 2022 .

易成岐 , 窦悦 , 陈东 , 等 . 全国一体化大数据中心协同创新体系：总体框架与战略价值 [J ] . 电子政务 , 2021 ( 6 ): 2 - 10 .

YI C Q , DOU Y , CHEN D , et al . Collaborative innovation system of national integrated big data center:overall framework and strategic value [J ] . E-Government , 2021 ( 6 ): 2 - 10 .

国务院 . 关于印发促进大数据发展行动纲要的通知 [Z ] . 2015 .

The State Council . Notice on issuing the platform for action to promote big data development [Z ] . 2015 .

郑谐维 . 隐私计算在政务数据共享中的应用 [J ] . 上海信息化 , 2022 ( 4 ): 17 - 21 .

ZHENG X W . Application of privacy computing in government data sharing [J ] . Shanghai Informatization , 2022 ( 4 ): 17 - 21 .

张聪丛 , 郜颍颍 , 赵畅 , 等 . 开放政府数据共享与使用中的隐私保护问题研究:基于开放政府数据生命周期理论 [J ] . 电子政务 , 2018 ( 9 ): 24 - 36 .

ZHANG C C , GAO Y Y , ZHAO C , et al . Research on privacy protection in open government data sharing and use—based on open government data life cycle theory [J ] . E-Government , 2018 ( 9 ): 24 - 36 .

李瀛 , 杨芮 . 我国政府数据开放的隐私保护困境及管理框架研究 [J ] . 情报杂志 , 2023 , 42 ( 1 ): 152 - 157 .

LI Y , YANG R . Research on privacy protection dilemma and management framework of open government data in China [J ] . Journal of Intelligence , 2023 , 42 ( 1 ): 152 - 157 .

KONEČNÝ J , MCMAHAN H B , YU F X , et al . Federated learning:strategies for improving communication efficiency [EB ] . arXiv preprint,2016,arXiv:1610.05492 .

杨强 . AI与数据隐私保护：联邦学习的破解之道 [J ] . 信息安全研究 , 2019 , 5 ( 11 ): 961 - 965 .

YANG Q . AI and data privacy protection:the way to federated learning [J ] . Journal of Information Security Research , 2019 , 5 ( 11 ): 961 - 965 .

YANG Q , LIU Y , CHEN T J , et al . Federated machine learning:concept and applications [J ] . ACM Transactions on Intelligent Systems and Technology , 2019 , 10 ( 2 ): 1 - 19 .

LIU B Y , WANG L J , LIU M , et al . Federated imitation learning:a privacy considered imitation learning framework for cloud robotic systems with heterogeneous sensor data [EB ] . arXiv preprint,2019,arXiv:1909.00895 .

GONG Y , JIANG Z W , FENG Y F , et al . EdgeRec:recommender system on edge in mobile Taobao [C ] // Proceedings of the 29th ACM International Conference on Information ＆ Knowledge Management . New York:ACM , 2020 : 2477 - 2484 .

王海燕 . 抖音的算法推荐特点分析 [J ] . 新媒体研究 , 2018 , 4 ( 20 ): 21 - 22 .

WANG H Y . Analysis of algorithm recommendation characteristics of Tiktok [J ] . NewMedia Research , 2018 , 4 ( 20 ): 21 - 22 .

李宗辰 , 张颐 , 叶东 , 等 . 运用联邦学习技术推动公安大数据的融合运用 [J ] . 警察技术 , 2021 ( 6 ): 34 - 37 .

LI Z C , ZHANG Y , YE D , et al . Using federated learning technology to promote the integration and application of public security big data [J ] . Police Technology , 2021 ( 6 ): 34 - 37 .

钱学胜 , 吴寰宇 , 陈诚 , 等 . 基于联邦学习的新冠肺炎疫情跨区域智慧防控技术:以上海市实践为例 [J ] . 科技导报 , 2021 , 39 ( 24 ): 96 - 107 .

QIAN X S , WU H Y , CHEN C , et al . Cross-region smart COVID-19 pandemic management and control based on federated learning:with Shanghai as an example [J ] . Science ＆ Technology Review , 2021 , 39 ( 24 ): 96 - 107 .

SHI D Y , TONG Y X , ZHOU Z M , et al . Learning to assign:towards fair task assignment in large-scale ride hailing [C ] // Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery ＆Data Mining . New York:ACM , 2021 : 3549 - 3557 .

戴理朋 , 杨鑫 , 徐茹枝 . 联邦学习在电力数据分析中的应用及隐私保护研究 [J ] . 电力信息与通信技术 , 2022 , 20 ( 11 ): 47 - 56 .

DAI L P , YANG X , XU R Z . Research on the application of federated learning in power data analysis and privacy protection [J ] . Electric Power Information and Communication Technology , 2022 , 20 ( 11 ): 47 - 56 .

刘隽良 , 王月兵 , 覃锦端 , 等 . 数据安全实践指南 [M ] . 北京 : 机械工业出版社 , 2022 : 42 - 82 .

LIU J L , WANG Y B , QIN J D , et al . Data security practice guide [M ] . Beijing : China Machine Press , 2022 : 42 - 82 .

高磊 , 赵章界 , 林野丽 , 等 . 基于《数据安全法》的数据分类分级方法研究 [J ] . 信息安全研究 , 2021 , 7 ( 10 ): 933 - 940 .

GAO L , ZHAO Z J , LIN Y L , et al . Research on data classification and grading method based on data security law [J ] . Journal of Information Security Research , 2021 , 7 ( 10 ): 933 - 940 .

DE CRISTOFARO E , TSUDIK G . Practical private set intersection protocols with linear complexity [C ] // SION R.International Conference on Financial Cryptography and Data Security . Berlin:Springer , 2010 : 143 - 159 .

PINKAS B , SCHNEIDER T , ZOHNER M . Faster private set intersection based on OT extension [C ] // Proceedings of the 23rd USENIX conference on Security Symposium . New York:ACM , 2014 : 797 - 812 .

王力 , 张秉晟 , 陈超超 . 隐私保护机器学习 [M ] . 北京 : 电子工业出版社 , 2021 :88.

WANG L , ZHANG B S , CHEN C C . Privacy protection machine learning [M ] . Beijing : Electronic Industry Press , 2021 :88.

ADOMAVICIUS G , TUZHILIN A . Toward the next generation of recommender systems:a survey of the state-of-theart and possible extensions [J ] . IEEE Transactions on Knowledge and Data Engineering , 2005 , 17 ( 6 ): 734 - 749 .

朱智韬 , 司世景 , 王健宗 , 等 . 联邦推荐系统综述 [J ] . 大数据 , 2022 , 8 ( 4 ): 105 - 132 .

ZHU Z T , SI S J , WANG J Z , et al . Survey on federated recommendation systems [J ] . Big Data Research , 2022 , 8 ( 4 ): 105 - 132 .

梁锋 , 羊恩跃 , 潘微科 , 等 . 基于联邦学习的推荐系统综述 [J ] . 中国科学:信息科学 , 2022 , 52 ( 5 ): 713 - 741 .

LIANG F , YANG E Y , PAN W K , et al . Survey of recommender systems based on federated learning [J ] . Scientia Sinica (Informationis) , 2022 , 52 ( 5 ): 713 - 741 .

ZHANG Z Q , CHEN C C , ZHOU J , et al . An industrial-scale system for heterogeneous information card ranking in alipay [C ] // Proceedings of the International Conference on Database Systems for Advanced Applications . Cham:Springer , 2018 : 713 - 724 .

CHEN C C , ZHENG X L , WANG Y , et al . Context-aware collaborative topic regression with social matrix factorization for recommender systems [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2014 , 28 ( 1 ): 9 - 15 .

仇阿根 , 张用川 , 罗宁 , 等 . 结合用户特征的政务服务协同过滤推荐方法 [J ] . 集成技术 , 2023 , 12 ( 1 ): 42 - 55 .

QIU A G , ZHANG Y C , LUO N , et al . Government service collaborative filtering recommendation method based on user characteristics [J ] . Journal of Integration Technology , 2023 , 12 ( 1 ): 42 - 55 .

王健宗 , 孔令炜 , 黄章成 , 等 . 联邦学习算法综述 [J ] . 大数据 , 2020 , 6 ( 6 ): 64 - 82 .

WANG J Z , KONG L W , HUANG Z C , et al . Research review of federated learning algorithms [J ] . Big Data Research , 2020 , 6 ( 6 ): 64 - 82 .

刘俊旭 , 孟小峰 . 机器学习的隐私保护研究综述 [J ] . 计算机研究与发展 , 2020 , 57 ( 2 ): 346 - 362 .

LIU J X , MENG X F . Survey on privacypreserving machine learning [J ] . Journal of Computer Research and Development , 2020 , 57 ( 2 ): 346 - 362 .

陈凯 , 杨强 . 隐私计算 [M ] . 北京 : 电子工业出版社 , 2022 :130.

CHEN K , YANG Q . Privacy computing [M ] . Beijing : Electronic Industry Press , 2022 :130.

王腾 , 霍峥 , 黄亚鑫 , 等 . 联邦学习中的隐私保护技术研究综述 [J ] . 计算机应用 , 2023 , 43 ( 2 ): 437 - 449 .

WANG T , HUO Z , HUANG Y X , et al . Review on privacy-preserving technologies in federated learning [J ] . Journal of Computer Applications , 2023 , 43 ( 2 ): 437 - 449 .

杨强 , 童咏昕 , 王晏晟 , 等 . 群体智能中的联邦学习算法综述 [J ] . 智能科学与技术学报 , 2022 , 4 ( 1 ): 29 - 44 .

YANG Q , TONG Y X , WANG Y S , et al . A survey on federated learning in crowd intelligence [J ] . Chinese Journal of Intelligent Science and Technology , 2022 , 4 ( 1 ): 29 - 44 .

邱鑫源 , 叶泽聪 , 崔翛龙 , 等 . 联邦学习通信开销研究综述 [J ] . 计算机应用 , 2022 , 42 ( 2 ): 333 - 342 .

QIU X Y , YE Z C , CUI X L , et al . Survey of communication overhead of federated learning [J ] . Journal of Computer Applications , 2022 , 42 ( 2 ): 333 - 342 .

VARGHESE B , WANG N , BARBHUIYA S , et al . Challenges and opportunities in edge computing [C ] // Proceedings of the 2016 IEEE International Conference on Smart Cloud (SmartCloud) . Piscataway:IEEE Press , 2016 : 20 - 26 .

HUANG W K , YE M , DU B . Learn from others and be yourself in heterogeneous federated learning [C ] // Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . Piscataway:IEEE Press , 2022 : 10133 - 10143 .

VOGELS T , HE L , KOLOSKOVA A , et al . RelaySum for decentralized deep learning on heterogeneous data [EB ] . arXiv preprint,2021,arXiv:2110.04175 .

黄聿辰 , 赵彦超 , 郝江山 , 等 . 面向数据异构的联邦学习性能优化研究 [J ] . 小型微型计算机系统 , 2024 , 45 ( 4 ): 777 - 783 .

HUANG Y C , ZHAO Y C , HAO J S , et al . Research on performance optimization of Federated learning for data heterogeneity [J ] . Small Micro Computer Systems , 2024 , 45 ( 4 ): 777 - 783 .

Views

322

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on data asset circulation and trading scheme integrating blockchain and federated learning technologies

Technical architecture and implementation of decision-driven provincial government big data governance

A survey on the fairness of federated learning

Design and implementation of trusted gateway for privacy-preserving interconnection

Federated meta learning: a review

Related Author

MO Zijia

ZHAO Chen

Yang GAO

Qunli YANG

Fan MENG

Jianzong WANG

Shijing SI

Zhitao ZHU

Related Institution

Postdoctoral Research Center, Industrial and Commercial Bank of China

School of Cyber Science and Engineering, Ningbo University of Technology

Jiangsu Strategy and Development Research Center

State Key Laboratory for Novel Software Technology at Nanjing University

Ping An Technology (Shenzhen) Co., Ltd.

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273110 Visits Today：16

⁰