A dynamic bidirectional matching method of tasks and resources oriented to wide-area distributed computing

Jing SHANG; Limin XIAO; Zhiwen XIAO; Jinquan WANG; Zhihui WU; Huiyang LI; Yifei ZHANG; Yao SONG; Jibin WANG

doi:10.11959/j.issn.2096-0271.2024050

您当前的位置：

首页 >

文章列表页 >

A dynamic bidirectional matching method of tasks and resources oriented to wide-area distributed computing

TOPIC: BIG DATA AND CLOUD STORAGE | 更新时间：2024-08-07

- A dynamic bidirectional matching method of tasks and resources oriented to wide-area distributed computing
- Big data research Vol. 10, Issue 4, Pages: 51-65(2024)
- 作者机构：
  
  1. 中移动信息技术有限公司，北京 100033
  2. 复杂关键软件环境全国重点实验室，北京 100191
  3. 北京航空航天大学计算机学院，北京 100191
  4. 中国信息通信研究院，北京 100191
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2024050
  CLC： TP316
- Online First：2024-07，
  
  Published：15 July 2024
- 稿件说明：
移动端阅览
Jing SHANG, Limin XIAO, Zhiwen XIAO, et al. A dynamic bidirectional matching method of tasks and resources oriented to wide-area distributed computing[J]. Big data research, 2024, 10(4): 51-65.
DOI：

Jing SHANG, Limin XIAO, Zhiwen XIAO, et al. A dynamic bidirectional matching method of tasks and resources oriented to wide-area distributed computing[J]. Big data research, 2024, 10(4): 51-65. DOI： 10.11959/j.issn.2096-0271.2024050.

摘要

广域分布式计算环境可提供大规模的计算和存储资源，是支持算力互联和数据流转的重要基础设施。在广域分布式计算环境中，任务与资源的匹配对于提高系统性能具有重要意义。然而，任务与资源的多样性、地理位置分散的资源会增加二者匹配的复杂性。针对响应延迟高、匹配效率低等问题，提出了面向广域分布式计算环境的任务与资源动态匹配方法，通过建立统一的任务需求模型和资源能力模型来简化匹配过程，降低响应延迟。此外，定义了任务向匹配度和资源向匹配度以刻画任务视角和资源视角的偏好，并权衡二者；定义了任务和资源的双向综合匹配度以量化任务需求和资源能力的适配程度。最后通过动态计算每一组任务与资源间的双向综合匹配度以优化匹配效果。实验结果表明，与现有的方法相比，该方法可提升匹配效果，并大幅降低平均响应延迟。

Abstract

Due to the huge capacities of computing and storage resources

wide-area distributed computing environment has become important infrastructures supporting computing power and data interconnection.In wide-area distributed computing environment

matching of tasks and resources is important to improve system performance.However

the diversity of tasks and resources and the geographical dispersion of resources increase the complexity of matching problems.To solve the problems of high response delay and low matching efficiency

a dynamic bidirectional matching method of tasks and resources oriented to wide-area distributed computing environments is proposed.The matching process is simplified and the response delay is mitigated by building a unified task requirement model and resource capability model.Moreover

the task-oriented and resource-oriented matching degrees are defined to express the preference of task-perspective and resource-perspective; the two-side comprehensive matching degree of tasks and resources is defined by the trade-off of the task-oriented and resource-oriented matching degree.The two-side comprehensive matching degrees are dynamically calculated for each task group and the resources to improve the matching quality.The experimental results show that the proposed method can effectively promoting the matching quality and significantly reduce the response delay compared with the existing methods.

关键词

Keywords

references

CHEN Q , ZHENG Z M , HU C , et al . On-edge multi-task transfer learning:model and practice with data-driven task allocation [J ] . IEEE Transactions on Parallel and Distributed Systems , 2020 , 31 ( 6 ): 1357 - 1371 .

TOWNS J , COCKERILL T , DAHAN M , et al . XSEDE:accelerating scientific discovery [J ] . Computing in Science ＆Engineering , 2014 , 16 ( 5 ): 62 - 74 .

GAGLIARDI F . The EGEE European grid infrastructure project [M ] // High Performance Computing for Computational Science - VECPAR 2004 . Heidelberg : Springer , 2005 : 194 - 203 .

DEPEI Q . CNGrid:a test-bed for Grid technologies in China [C ] // Proceedings of 10th IEEE International Workshop on Future Trends of Distributed Computing Systems . Piscataway:IEEE Press , 2004 : 135 - 139 .

KANG S , VEERAVALLI B , AUNG K M M . Dynamic scheduling strategy with efficient node availability prediction for handling divisible loads in multicloud systems [J ] . Journal of Parallel and Distributed Computing , 2018 , 113 : 1 - 16 .

CHOUDHARY A . A walkthrough of Amazon elastic compute cloud (amazon EC2):a review [J ] . International Journal for Research in Applied Science and Engineering Technology , 2021 , 9 ( 11 ): 93 - 97 .

王建冬 , 于施洋 , 窦悦 . 东数西算：我国数据跨域流通的总体框架和实施路径研究 [J ] . 电子政务 , 2020 ( 3 ): 13 - 21 .

WANG J D , YU S Y , DOU Y . East digital computing and west computing:research on the overall framework and implementation path of cross-domain data circulation in China [J ] . E-Government , 2020 ( 3 ): 13 - 21 .

高文 . 中国算力网的机遇与挑战 [J ] . 中国计算机学会通讯 , 2023 , 1 : 1 - 6 .

GAO W . The opportunities and challenges of China’s computing power network [J ] . Communictions of the CCF , 2023 ( 2 ): 1 - 6 .

钱德沛 , 栾钟治 , 刘轶 . 从网格到“东数西算”:构建国家算力基础设施 [J ] . 北京航空航天大学学报 , 2022 , 48 ( 9 ): 1561 - 1574 .

QIAN D P , LUAN Z Z , LIU Y . From grid to “East-west Computing Transfer”:constructing national computing infrastructure [J ] . Journal of Beijing University of Aeronautics and Astronautics , 2022 , 48 ( 9 ): 1561 - 1574 .

XU K , LYU L , LI T , et al . Minimizing tardiness for data-intensive applications in heterogeneous systems:a matching theory perspective [J ] . IEEE Transactions on Parallel and Distributed Systems , 2020 , 31 ( 1 ): 144 - 158 .

毕娅 , 原惠群 , 初叶萍 , 等 . 大数据环境下基于公共服务平台的资源多级智能寻租与匹配策略和价值创造 [J ] . 计算机科学 , 2019 , 46 ( 2 ): 42 - 49 .

BI Y , YUAN H Q , CHU Y P , et al . Multilevel and intelligent rent-seeking and matching resource strategy and value creation of public service platform in big data environment [J ] . Computer Science , 2019 , 46 ( 2 ): 42 - 49 .

ZHAO L P , YANG Y N , MUNIR A , et al . Optimizing geo-distributed data analytics with coordinated task scheduling and routing [J ] . IEEE Transactions on Parallel and Distributed Systems , 2020 , 31 ( 2 ): 279 - 293 .

HE L , QIAN Z C . Intent-based resource matching strategy in cloud [J ] . Information Sciences , 2020 , 538 : 1 - 18 .

RAVEENDRAN N , ZHANG H Q , SONG L Y , et al . Pricing and resource allocation optimization for IoT fog computing and NFV:an EPEC and matching based perspective [J ] . IEEE Transactions on Mobile Computing , 2022 , 21 ( 4 ): 1349 - 1361 .

FENG W J , ZHENG J L , JIANG W H . Joint pilot and data transmission power control and computing resource allocation algorithm for massive MIMO-MEC networks [J ] . IEEE Access , 2020 , 8 : 80801 - 80811 .

CHEN Y F , LI Z Y , YANG B , et al . A Stackelberg game approach to multiple resources allocation and pricing in mobile edge computing [J ] . Future Generation Computer Systems , 2020 , 108 : 273 - 287 .

LI B D , YANG Y , SU J F , et al . Two-sided matching decision-making model with hesitant fuzzy preference information for configuring cloud manufacturing tasks and resources [J ] . Journal of Intelligent Manufacturing , 2020 , 31 ( 8 ): 2033 - 2047 .

LI C L , BAI J P , TANG J H . Joint optimization of data placement and scheduling for improving user experience in edge computing [J ] . Journal of Parallel and Distributed Computing , 2019 , 125 : 93 - 105 .

CHEN L T , CHEN S Q . Volunteer multi-person multi-task optimization dispatch method considering two-sided matching [J ] . Soft Computing , 2022 , 26 ( 8 ): 3837 - 3861 .

WANG W , LI B C , LIANG B , et al . Multiresource fair sharing for datacenter jobs with placement constraints [C ] // Proceedings of the International Conference for High Performance Computing,Networking,Storage and Analysis . Piscataway:IEEE Press , 2016 : 1003 - 1014 .

LEONG S H , PARODI A , KRANZLMÜLLER D . A robust reliable energy-aware urgent computing resource allocation for flash-flood ensemble forecasting on HPC infrastructures for decision support [J ] . Future Generation Computer Systems , 2017 , 68 : 136 - 149 .

KAMECKE U , ROTH A E , SOTOMAYOR M A O . Two sided matching:a study in game-theoretic modeling and analysis [J ] . Economica , 1992 , 59 ( 236 ): 487 .

MA Y T , WANG H J , XIONG J , et al . Joint allocation on communication and computing resources for fog radio access networks [J ] . IEEE Access , 2020 , 8 : 108310 - 108323 .

YUAN Y L , YANG T , HU Y L , et al . Two-timescale resource allocation for cooperative D2D communication:a matching game approach [J ] . IEEE Transactions on Vehicular Technology , 2021 , 70 ( 1 ): 543 - 557 .

DING D , FAN X C , LUO S W . Useroriented cloud resource scheduling with feedback integration [J ] . The Journal of Supercomputing , 2016 , 72 ( 8 ): 3114 - 3135 .

WANG Y F , LIU J , TONG Y , et al . Resource scheduling in mobile edge computing using improved ant colony algorithm for space information network [J ] . International Journal of Satellite Communications and Networking , 2023 , 41 ( 4 ): 331 - 356 .

SHEN H X , LI S G , LIANG Y Y . Faster algorithms for bicriteria scheduling of identical jobs on uniform machines [J ] . Journal of Industrial and Management Optimization , 2023 , 19 ( 7 ): 5398 - 5406 .

SONG Y , WANG L , XIAO L M , et al . Dynamic two-side matching of tasks and resources in wide-area distributed computing environments [J ] . The Journal of Supercomputing , 2023 , 79 ( 9 ): 10208 - 10231 .

LI C Y , TANG X Y . On fault-tolerant Bin packing for online resource allocation [J ] . IEEE Transactions on Parallel and Distributed Systems , 2020 , 31 ( 4 ): 817 - 829 .

KHIYAITA A , EL BAKKALI H , ZBAKH M , et al . Load balancing cloud computing:state of art [C ] // Proceedings of the 2012 National Days of Network Security and Systems . Piscataway:IEEE Press , 2012 : 106 - 109 .

Views

153

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273110 Visits Today：16

⁰