Federated meta learning: a review

Chuanyao ZHANG; Shijing SI; Jianzong WANG; Jing XIAO

doi:10.11959/j.issn.2096-0271.2022051

您当前的位置：

首页 >

文章列表页 >

Federated meta learning: a review

STUDY | 更新时间：2024-06-03

- Federated meta learning: a review
- Big data research Vol. 9, Issue 2, Pages: 122-146(2023)
- 作者机构：
  
  1. 平安科技（深圳）有限公司，广东深圳 518063
  2. 中国科学技术大学，安徽合肥 230026
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2022051
  CLC： TP181
- Online First：2023-03，
  
  Published：15 March 2023
- 稿件说明：
移动端阅览
Chuanyao ZHANG, Shijing SI, Jianzong WANG, et al. Federated meta learning: a review[J]. Big data research, 2023, 9(2): 122-146.
DOI：

Chuanyao ZHANG, Shijing SI, Jianzong WANG, et al. Federated meta learning: a review[J]. Big data research, 2023, 9(2): 122-146. DOI： 10.11959/j.issn.2096-0271.2022051.

摘要

随着移动设备的普及，海量的数据在不断产生。数据隐私政策不断细化，数据的流动和使用受到严格监管。联邦学习可以打破数据壁垒，联合利用不同客户端数据进行建模。由于用户使用习惯不同，不同客户端数据之间存在很大差异。如何解决数据不平衡带来的统计挑战，是联邦学习研究的一个重要课题。利用元学习的快速学习能力，为不同数据节点训练不同的个性化模型来解决联邦学习中的数据不平衡问题成为一种重要方式。从联邦学习背景出发，系统介绍了联邦学习的问题定义、分类方式及联邦学习面临的主要问题。主要问题包括：隐私保护、数据异构、通信受限。从联邦元学习的背景出发，系统介绍了联邦元学习在解决联邦学习数据异构、通信受限问题及提高恶意攻击下鲁棒性方面的研究工作，对联邦元学习的工作进行了总结展望。

Abstract

With the popularity of mobile devices

massive amounts of data are constantly produced.The data privacy policies are becoming more and more specified

the flow and use of data are strictly regulated.Federated learning can break data barriers and use client data for modeling.Because users have different habits

there are significant differences between different client data.How to solve the statistical challenge caused by the data imbalance becomes an important topic in federated learning research.Using the fast learning ability of meta learning

it becomes an important way to train different personalized models for different clients to solve the problem of data imbalance in federated learning.The definition and classification of federated learning

as well as the main problems of federated learning were introduced systematically based on the background of federated learning.The main problems included privacy protection

data heterogeneity and limited communication.The research work of federated metalearning in solving the heterogeneous data

the limited communication environment

and improving the robustness against malicious attacks were introduced systematically starting from the background of federated meta learning.Finally

the summary and prospect of federated meta learning were proposed.

关键词

Keywords

references

王健宗 , 孔令炜 , 黄章成 , 等 . 联邦学习算法综述 [J ] . 大数据 , 2020 , 6 ( 6 ): 64 - 82 .

WANG J Z , KONG L W , HUANG Z C , et al . Research review of federated learning algorithms [J ] . Big Data Research , 2020 , 6 ( 6 ): 64 - 82 .

BERTINETTO L , HENRIQUES J F , TORR P H S , et al . Meta-learning with differentiable closed-form solvers [J ] . arXiv preprint , 2018 ,arXiv:1805.08136.

VILALTA R , DRISSI Y . A perspective view and survey of meta-learning [J ] . Artificial Intelligence Review , 2002 , 18 ( 2 ): 77 - 95 .

MITCHELL R , MICHALSKI J , CARBONELL T . An artificial intelligence approach [M ] . Machine Learning . Heidelberg : Springer , 2013 .

FINN C , ABBEEL P , LEVINE S . Modelagnostic meta-learning for fast adaptation of deep networks [C ] // Proceedings of the 34th International Conference on Machine Learning . New York:ACM Press , 2017 : 1126 - 1135 .

YANG Q , LIU Y , CHENG Y , et al . Federated learning [J ] . Synthesis Lectures on Artificial Intelligence and Machine Learning , 2019 , 13 ( 3 ): 1 - 207 .

YANG Q , LIU Y , CHEN T J , et al . Federated machine learning:concept and applications [J ] . ACM Transactions on Intelligent Systems and Technology , 2019 , 10 ( 2 ): 1 - 19 .

ALEDHARI M , RAZZAK R , PARIZI R M , et al . Federated learning:a survey on enabling technologies,protocols,and applications [J ] . IEEE Access:Practical Innovations,Open Solutions , 2020 , 8 : 140699 - 140725 .

王健宗 , 孔令炜 , 黄章成 , 等 . 联邦学习隐私保护研究进展 [J ] . 大数据 , 2021 , 7 ( 3 ): 130 - 149 .

WANG J Z , KONG L W , HUANG Z C , et al . Research advances on privacy protection of federated learning [J ] . Big Data Research , 2021 , 7 ( 3 ): 130 - 149 .

WEI K , LI J , DING M , et al . Federated learning with differential privacy:algorithms and performance analysis [J ] . IEEE Transactions on Information Forensics and Security , 2020 , 15 : 3454 - 3469 .

DWORK C , . Differential Privacy:a survey of results [C ] // Proceedings of International Conference on Theory and Applications of Models of Computation . Heidelberg:Springer , 2008 : 1 - 19 .

DWORK C , ROTH A . The algorithmic foundations of differential privacy [J ] . Foundations and Trends® in Theoretical Computer Science , 2013 , 9 ( 3/4 ): 211 - 407 .

SATTLER F , WIEDEMANN S , MULLER K R , et al . [J ] . Robust and communication-efficient federated learning from non-i.i.d.data , 2020 , 31 ( 9 ): 3400 - 3413 .

ZHAO Y , LI M , LAI L Z , et al . Federated learning with non-IID data [J ] . arXiv preprint , 2018 ,arXiv:1806.00582.

BRIGGS C , FAN Z , ANDRAS P . Federated learning with hierarchical clustering of local updates to improve training on non-IID data [C ] // Proceedings of 2020 International Joint Conference on Neural Networks . Piscataway:IEEE Press , 2020 : 1 - 9 .

MCMAHAN H B , MOORE E , RAMAGE D , et al . Communication-efficient learning of deep networks from decentralized data [J ] . arXiv preprint , 2016 ,arXiv:1602.05629.

ARIVAZHAGAN M G , AGGARWAL V , SINGH A K , et al . Federated learning with personalization layers [J ] . arXiv preprint , 2019 ,arXiv:1912.00818.

KAIROUZ P , MCMAHAN H B , AVENT B , et al . Advances and open problems in federated learning [J ] . Foundations and Trends® in Machine Learning , 2021 , 14 ( 1–2 ): 1 - 210 .

KONEČNÝ J , MCMAHAN H B , YU F X , et al . Federated learning:strategies for improving communication efficiency [J ] . arXiv preprint , 2016 ,arXiv:1610.05492.

HSU T M H , QI H , BROWN M . Measuring the effects of non-identical data distribution for federated visual classification [J ] . arXiv preprint , 2019 ,arXiv:1909.06335.

REDDI S , CHARLES Z , ZAHEER M , et al . Adaptive federated optimization [J ] . arXiv preprint , 2020 ,arXiv:2003.00295.

DUCHI J C , HAZAN E , SINGER Y . Adaptive subgradient methods for online learning and stochastic optimization [J ] . Journal of Machine Learning Research , 2011 , 12 : 2121 - 2159 .

KINGMA D P , BA J . Adam:a method for stochastic optimization [J ] . arXiv preprint , 2014 ,arXiv:1412.6980.

ZAHEER M , REDDI S J , SACHAN D , et al . Adaptive methods for nonconvex optimization [C ] // Proceedings of the 32nd International Conference on Neural Information Processing Systems . New York:ACM Press , 2018 : 9815 - 9825 .

WANG H Y , YUROCHKIN M , SUN Y K , et al . Federated learning with matched averaging [J ] . arXiv preprint , 2020 ,arXiv:2002.06440.

ARIVAZHAGAN M G , AGGARWAL V , SINGH A K , et al . Federated learning with personalization layers [J ] . arXiv preprint , 2019 ,arXiv:1912.00818.

LI T , SAHU A K , ZAHEER M , et al . Federated optimization in heterogeneous networks [J ] . Proceedings of Machine Learning and Systems , 2020 , 2 : 429 - 450 .

KONEČNÝ J , MCMAHAN B , RAMAGE D . Federated optimization:distributed optimization beyond the datacenter [J ] . arXiv preprint , 2015 ,arXiv:1511.03575.

DENG Y Y , KAMANI M M , MAHDAVI M . Adaptive personalized federated learning [J ] . arXiv preprint , 2020 ,arXiv:2003.13461.

KARIMIREDDY S P , KALE S , MOHRI M , et al . SCAFFOLD:stochastic controlled averaging for federated learning [J ] . arXiv preprint , 2019 ,arXiv:1910.06378.

LI T , HU S Y , BEIRAMI A , et al . Ditto:fair and robust federated learning through personalization [J ] . arXiv preprint , 2020 ,arXiv:2012.04221.

LI Q B , HE B S , SONG D . Modelcontrastive federated learning [C ] // Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition . Piscataway:IEEE Press , 2021 : 10708 - 10717 .

KHODAK M , BALCAN M F , TALWALKAR A . Adaptive gradientbased meta-learning methods [J ] . arXiv preprint , 2019 ,arXiv:1906.02717.

VANSCHOREN J . Meta-learning:a survey [J ] . arXiv preprint , 2018 ,arXiv:1810.03548.

VILALTA R , DRISSI Y . A perspective view and survey of meta-learning [J ] . Artificial Intelligence Review , 2002 , 18 ( 2 ): 77 - 95 .

李凡长 , 刘洋 , 吴鹏翔 , 等 . 元学习研究综述 [J ] . 计算机学报 , 2021 , 44 ( 2 ): 422 - 446 .

LI F Z , LIU Y , WU P X , et al . A survey on recent advances in meta-learning [J ] . Chinese Journal of Computers , 2021 , 44 ( 2 ): 422 - 446 .

HSU K , LEVINE S , FINN C . Unsupervised learning via metalearning [J ] . arXiv preprint , 2018 ,arXiv:1810.02334.

KHODADADEH S , BOLONI L , SHAH M . Unsupervised meta-learning for fewshot image classification [J ] . Advances in Neural Information Processing Systems , 2019 ,32.

SNELL J , SWERSKY K , ZEMEL R . Prototypical networks for few-shot learning [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York:ACM Press , 2017 : 4080 - 4090 .

RAVI S , LAROCHELLE H . Optimization as a model for few-shot learning [J ] . International Conference on Learning Representations , 2016 .

DUAN Y , SCHULMAN J , CHEN X , et al . RL2:fast reinforcement learning via slow reinforcement learning [J ] . arXiv preprint , 2016 ,arXiv:1611.02779.

ANDRYCHOWICZ M , DENIL M , COLMENAREJO S G , et al . Learning to learn by gradient descent by gradient descent [C ] // Proceedings of the 30th International Conference on Neural Information Processing Systems . New York:ACM Press , 2016 : 3988 - 3996 .

SANTORO A , BARTUNOV S , BOTVINICK M , et al . Meta-learning with memory-augmented neural networks [C ] // Proceedings of the 33rd International Conference on Machine Learning . New York:ACM Press , 2016 : 1842 - 1850 .

MISHRA N , ROHANINEJAD M , CHEN X , et al . A simple neural attentive meta-learner [J ] . arXiv preprint , 2017 ,arXiv:1707.03141.

RAJESWARAN A , FINN C , KAKADE S , et al . Meta-learning with implicit gradients [J ] . arXiv preprint , 2019 arXiv:1909.04630.

SONG X Y , GAO W B , YANG Y X , et al . ESMAML:simple hessian-free meta learning [J ] . arXiv preprint 2019 ,arXiv:1910.01215.

SALIMANS T , HO J , CHEN X , et al . Evolution strategies as a scalable alternative to reinforcement learning [J ] . arXiv preprint , 2017 ,arXiv:1703.03864.

NICHOL A , ACHIAM J , SCHULMAN J . On first-order meta-learning algorithms [J ] . arXiv preprint , 2018 ,arXiv:1803.02999.

RUSU A A , RAO D , SYGNOWSKI J , et al . Meta-learning with latent embedding optimization [J ] . arXiv preprint , 2018 ,arXiv:1807.05960.

SUNG F , ZHANG L , XIANG T , et al . Learning to learn:meta-critic networks for sample efficient learning [J ] . arXiv preprint , 2017 ,arXiv:1706.09529.

VINYALS O , BLUNDELL C , LILLICRAP T , et al . Matching networks for one shot learning [C ] // Proceedings of the 30th International Conference on Neural Information Processing Systems . New York:ACM Press , 2016 : 3637 - 3645 .

KOCH G , ZEMEL R , SALAKHUTDINOV R . Siamese neural networks for one-shot image recognition [C ] // Proceedings of ICML Deep Learning Workshop .[S.l.:s.n. ] , 2015 .

SANTORO A , RAPOSO D , BARRETT D G T , et al . A simple neural network module for relational reasoning [C ] // Proceedings of the 31st International Conference on Neural Information Processing Systems . New York:ACM Press , 2017 : 4974 - 4983 .

CHAWLA N V , BOWYER K W , HALL L O , et al . SMOTE:synthetic minority oversampling technique [J ] . Journal of Artificial Intelligence Research , 2002 , 16 : 321 - 357 .

INOUE H . Data augmentation by pairing samples for images classification [J ] . arXiv preprint , 2018 ,arXiv:1801.02929.

GOODFELLOW I , POUGET-ABADIE J , MIRZA M , et al . Generative adversarial networks [J ] . Communications of the ACM , 2020 , 63 ( 11 ): 139 - 144 .

CUBUK E D , ZOPH B , MANE D , et al . AutoAugment:learning augmentation policies from data [J ] . arXiv preprint , 2018 ,arXiv:1805.09501.

ZHANG H P , CAO Z , YAN Z A , et al . Sill-net:feature augmentation with separated illumination representation [J ] . arXiv preprint , 2021 ,arXiv:2102.03539.

RAJENDRAN J , IRPAN A , JANG E . Metalearning requires meta-augmentation [C ] // Proceedings of the 34th International Conference on Neural Information Processing Systems . New York:ACM Press , 2020 : 5705 - 5715 .

ZHOU F W , LI J W , XIE C L , et al . MetaAugment:sample-aware data augmentation policy learning [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2021 , 35 ( 12 ): 11097 - 11105 .

ZHANG R X , CHE T , GHAHRAMANI Z , et al . MetaGAN:an adversarial approach to few-shot learning [C ] // Proceedings of the 32nd International Conference on Neural Information Processing Systems . New York:ACM Press , 2018 : 2371 - 2380 .

SUN P , OUYANG Y , ZHANG W , et al . MEDA:meta-learning with data augmentation for few-shot text classification [C ] // Proceedings of 2021 International Joint Conference on Artificial Intelligence .[S.l.:s.n. ] , 2021 : 3929 - 3935 .

CHEN F , LUO M , DONG Z H , et al . Federated meta-learning with fast convergence and efficient communication [J ] . arXiv preprint , 2018 ,arXiv:1802.07876.

JIANG Y H , KONEČNÝ J , RUSH K , et al . Improving federated learning personalization via model agnostic meta learning [J ] . arXiv preprint , 2019 ,arXiv:1909.12488.

FALLAH A , MOKHTARI A , OZDAGLAR A . Personalized federated learning with theoretical guarantees:a model-agnostic meta-learning approach [J ] . Advances in Neural Information Processing Systems , 2020 , 33 : 3557 - 3568 .

ZAFAR M B , VALERA I , GOMEZ RODRIGUEZ M , et al . Fairness beyond disparate treatment ＆ disparate impact:learning classification without disparate mistreatment [C ] // Proceedings of the 26th International Conference on World Wide Web . Switzerland:International World Wide Web Conferences Steering Committee , 2017 : 1171 - 1180 .

LI T , SANJABI M , BEIRAMI A , et al . Fair resource allocation in federated learning [J ] . arXiv preprint , 2019 ,arXiv:1905.10497.

KHODAK M , BALCAN M F , TALWALKAR A . Adaptive gradientbased meta-learning methods [J ] . arXiv preprint , 2019 ,arXiv:1906.02717.

ZHANG M , SAPRA K , FIDLER S , et al . Personalized Federated Learning with first order model optimization [C ] // Proceedings of International Conference on Learning Representations .[S.l.:s.n. ] , 2020 .

ACAR D A E , ZHAO Y , ZHU R , et al . Debiasing model updates for improving personalized federated training [C ] // Proceedings of International Conference on Machine Learning .[S.l.:s.n. ] , 2021 : 21 - 31 .

SINGHAL K , SIDAHMED H , GARRETT Z , et al . Federated reconstruction:partially local federated learning [J ] . arXiv preprint , 2021 ,arXiv:2102.03448.

YUE S , REN J , XIN J , et al . Efficient federated meta-learning over multiaccess wireless networks [J ] . IEEE Journal on Selected Areas in Communications , 2022 , 40 ( 5 ): 1556 - 1570 .

YUE S , REN J , XIN J , et al . InexactADMM based federated meta-learning for fast and continual edge learning [C ] // Proceedings of the 22nd International Symposium on Theory,Algorithmic Foundations,and Protocol Design for Mobile Networks and Mobile Computing . New York:ACM Press , 2021 : 91 - 100 .

ELGABLI A , ISSAID C B , BEDI A S , et al . Energy-efficient and federated meta-learning via projected stochastic gradient ascent [C ] // Proceedings of 2021 IEEE Global Communications Conference . Piscataway:IEEE Press , 2022 : 1 - 6 .

YAO X , HUANG T C , ZHANG R X , et al . Federated learning with unbiased gradient aggregation and controllable meta updating [J ] . arXiv preprint , 2019 ,arXiv:1910.08234.

EDMUNDS R , GOLMANT N , RAMASESH V , et al . Transferability of adversarial attacks in model-agnostic meta-learning [C ] // Proceedings of 2017 Deep Learning and Security Workshop .[S.l.:s.n. ] , 2017 .

YIN C X , TANG J , XU Z Y , et al . Adversarial meta-learning [J ] . arXiv preprint , 2018 ,arXiv:1806.03316.

LIN S , YANG G , ZHANG J S . A collaborative learning framework via federated meta-learning [C ] // Proceedings of 2020 IEEE 40th International Conference on Distributed Computing Systems . Piscataway:IEEE Press , 2021 : 289 - 299 .

CHEN C L , GOLUBCHIK L , PAOLIERI M . Backdoor attacks on federated metalearning [J ] . arXiv preprint , 2020 ,arXiv:2006.07026.

YIN D , CHEN Y D , RAMCHANDRAN K , et al . Byzantine-robust distributed learning:towards optimal statistical rates [J ] . arXiv preprint , 2018 ,arXiv:1803.01498.

ARAMOON O , CHEN P Y , QU G , et al . Meta federated learning [J ] . arXiv preprint , 2021 ,arXiv:2102.05561.

BONAWITZ K , IVANOV V , KREUTER B , et al . Practical secure aggregation for privacy-preserving machine learning [C ] // Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security . New York:ACM Press , 2017 : 1175 - 1191 .

ZHENG W B , YAN L , GOU C , et al . Federated meta-learning for fraudulent credit card detection [C ] // Proceedings of the 29th International Joint Conference on Artificial Intelligence . New York:ACM Press , 2021 : 4654 - 4660 .

LI X F , LIU S H , LI Z F , et al . FlowScope:spotting money laundering based on graphs [J ] . Proceedings of the AAAI Conference on Artificial Intelligence , 2020 , 34 ( 4 ): 4731 - 4738 .

DAL POZZOLO A , BORACCHI G , CAELEN O , et al . Credit card fraud detection:a realistic modeling and a novel learning strategy [J ] . IEEE Transactions on Neural Networks and Learning Systems , 2018 , 29 ( 8 ): 3784 - 3797 .

ZHAO H , JI F , LI Q , et al . Federated meta-learning enhanced acoustic radio cooperative framework for ocean of things [J ] . IEEE Journal of Selected Topics in Signal Processing , 2022 , 16 ( 3 ): 474 - 486 .

LIN Y J , REN P J , CHEN Z M , et al . Meta matrix factorization for federated rating predictions [C ] // Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval . New York:ACM Press , 2020 : 981 - 990 .

JALALIRAD A , SCAVUZZO M , CAPOTA C , et al . A simple and efficient federated recommender system [C ] // Proceedings of the 6th IEEE/ACM International Conference on Big Data Computing,Applications and Technologies . New York:ACM Press , 2019 : 53 - 58 .

BEEL J . Federated meta-learning:democratizing algorithm selection across disciplines and software libraries [J ] . Science (AICS) , 2018 ,210:219.

吴建汉 , 司世景 , 王健宗 , 等 . 联邦学习攻击与防御综述 [J ] . 大数据 , 2022 , 8 ( 5 ): 12 - 32 .

WU J H , SI S J , WANG J Z , et al . Threats and defenses of federated learning:a survey [J ] . Big Data Research , 2022 , 8 ( 5 ): 12 - 32 .

Views

420

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Threats and defenses of federated learning: a survey

Research advances on privacy protection of federated learning

Research on data asset circulation and trading scheme integrating blockchain and federated learning technologies

Research on the application of government big data platform based on federated learning

A survey on the fairness of federated learning

Related Author

Jianhan WU

Jing XIAO

Chunxi LU

Yi LIU

Linjie CHEN

Zhangcheng HUANG

Lingwei KONG

Jianzong WANG

Related Institution

School of Cyber Science and Engineering, Ningbo University of Technology

Postdoctoral Research Center, Industrial and Commercial Bank of China

Big Data Development Center of Zhejiang Province

College of Computer Science and Technology, Zhejiang University

Key Laboratory of Key Technologies for Open Data Fusion in Zhejiang Province

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273114 Visits Today：20

⁰