Optimizing distributed systems with remote direct memory access

Xingda WEI; Rong CHEN; Haibo CHEN

doi:10.11959/j.issn.2096-0271.2018036

您当前的位置：

首页 >

文章列表页 >

Optimizing distributed systems with remote direct memory access

TOPIC：SYSTEM ARCHITECTURE FOR BIG DATA-DRIVEN INTELLIGENT | 更新时间：2024-06-03

- Optimizing distributed systems with remote direct memory access
- Big Data Research Vol. 4, Issue 4, Pages: 2018036(2018)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2018036
  CLC： TP316
- Online First：2018-07，
  
  Published：15 July 2018
- 稿件说明：
移动端阅览
Xingda WEI, Rong CHEN, Haibo CHEN. Optimizing distributed systems with remote direct memory access[J]. Big Data Research, 2018, 4(4): 2018036.
DOI：

Xingda WEI, Rong CHEN, Haibo CHEN. Optimizing distributed systems with remote direct memory access[J]. Big Data Research, 2018, 4(4): 2018036. DOI： 10.11959/j.issn.2096-0271.2018036.

摘要

高速的RDMA网络设备已经被广泛部署在现代数据中心。RDMA可以从两方面加速分布式系统：首先可以提供一种快速的消息处理机制，其次RDMA提供了新的硬件原语。这极大地提升了处理器的利用率以及对RDMA的使用率，但是需要重新设计系统。介绍了RDMA的研究进展，概述了近年来利用RDMA加速分布式系统的工作，包括基于RDMA重新设计的系统以及如何更好地利用RDMA的设计，并给出了未来的研究方向。

Abstract

Fast network devices with RDMA support have been price-compatible with traditional network primitives such as Ethernet

and it’s now widely deployed in modern data centers.RDMA can be used in two ways.Firstly

it can optimize the messaging primitive in distributed applications.The second way is to redesign the applications with RDMA’s onesided features.One-sided features provide high CPU utilizations and high network performance

but the system should be redesigned.The research progress of RDMA was introduced.An overview on the research efforts on using RDMA for distributed systems was presented.The works on how to use RDMA to redesign systems and the works on how to better leverage RDMA were included.The future research directions were also put forward.

关键词

Keywords

references

FAN B , ANDERSEN D G , KAMINSKY M . MemC3:compact and concurrent MemCache with dumber caching and smarter hashing [C ] // The 10th USENIX Conference on Networked Systems Design and Implementation,April2-5,2013,San Francisco,USA . Berkeley:USENIX Association , 2013 : 371 - 384 .

GUO C , WU H , DENG Z , et al . RDMA over commodity ethernet at scale [C ] // The 2016 ACM SIG-COMM Conference,August 22-26,2016,Florianopolis,Brazil . New York:ACM Press , 2016 : 202 - 215 .

TSAI S Y , ZHANG Y . Lite kernel rdma support for datacenter applications [C ] // The 26th ACM Symposium on Operating Systems Principles,October 28-31,2017,Shanghai,China.[S.l.:s.n] . 2017 : 306 - 324 .

NARAYANAN D , HODSON O , CASTRO M , et al . FaRM:fast remote memory [C ] // The 11th USENIX Conference on Networked Systems Design and Implementation,April 2-4,2014,Seattle,USA . Berkeley:USENIX Association , 2014 : 401 - 414 .

DRAGOJEVIĆ A , NARAYANAN D , NIGHTINGALE E B , et al . No compromises:distributed transactions with consistency,availability,and performance [C ] // The 25th Symposium on Operating Systems Principles,October 4-7,2015,Monterey,USA . New York:ACM Press , 2015 : 54 - 70 .

NEWMARCH J . Remote procedure call [D ] . Pittsburgh:Carnegie Mellon University , 1981 .

KALIA A , KAMINSKY M , ANDERSEN D G . Design guidelines for high performance RDMA systems [C ] // 2016 USENIX Annual Technical Conference,June 22-24,Denver,USA . Berkeley:USENIX Association , 2016 : 437 - 450 .

KALIA A , KAMINSKY M , ANDERSEN D G . FaSST:fast,scalable and simple distributed transactions with two-sided (RDMA) datagram RPCs [C ] // The 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16),November 2-4,2016,Savannah,USA . Berkeley:USENIX Association , 2016 : 185 - 201 .

LU Y , SHU J , CHEN Y , et al . Octopus:an RDMA-enabled distributed persistent memory file system [C ] // 2017 USENIX Annual Technical Conference (USENIX ATC 17),July 12-14,2017,Santa Clara,USA . Berkeley:USENIX Association , 2017 : 773 - 785 .

SU M , ZHANG M , CHEN K , et al . RFP:when RPC is faster than server-bypass with RDMA [C ] // The 12th European Conference on Computer Systems,April 23-26,2017,Belgrade,Serbia . New York:ACM Press , 2017 : 1 - 15 .

KALIA A , KAMINSKY M , ANDERSEN D G . Using RDMA efficiently for key-value services [C ] // The 2014 ACM Conference on SIGCOMM,August 17-22,2014,Chicago,USA . New York:ACM Press , 2014 , 44 ( 4 ): 295 - 306 .

TSAI S Y , ZHANG Y . Lite kernel rdma support for datacenter applications [C ] // The 26th ACM Symposium on Operating Systems Principles,October 28-31,2017,Shanghai,China . New York:ACM Press , 2017 : 306 - 324 .

BRONSON N , AMSDEN Z , CABRERA G , et al . TAO:Facebook's distributed data store for the social graph [C ] // The 2013 USENIX Conference on Annual Technical Conference,June 26-28,2013,San Jose,USA . Berkeley:USENIX Association , 2013 : 49 - 60 .

DECANDIA G , HASTORUN D , JAMPANI M , et al . Dynamo:amazon’s highly available key-value store [J ] . ACM SIGOPS Operating Systems Review , 2007 , 41 ( 6 ): 205 - 220 .

ATIKOGLU B , XU Y H , FRACHTENBERG E , et al . Workload analysis of a large-scale key-value store [J ] . ACM SIGMETRICS Performance Evaluation Review , 2012 , 40 ( 1 ): 53 - 64 .

NISHTALA R , FUGAL H , GRIMM S , et al . Scaling memcache at Facebook [C ] // The 10th USENIX Conference on Networked Systems Design and Implementation (NSDI’13),April 2-5,2013,Lombard,USA . Berkeley:USENIX Association , 2013 : 385 - 398 .

MITCHELL C , GENG Y , LI J . Using onesided RDMA reads to build a fast,CPUefficient key-value store [C ] // The 2013 USENIX Annual Technical Conference,June 26-28,2013,San Jose,USA . Berkeley:USENIX Association , 2013 : 103 - 114 .

PAGH R , RODLER F F . Cuckoo hashing [J ] . Journal of Algorithms , 2004 , 51 ( 2 ): 122 - 144 .

HERLIHY M , SHAVIT N , TZAFRIR M . Hopscotch Hashing [C ] // The 22nd International Symposium on Distributed Computing,September 22-24,2008,Arcachon,France . Berkeley:USENIX Association , 2008 : 350 - 364 .

MONTGOMERY C M K , NELSON L , SEN S , et al . Balancing CPU and network in the cell distributed B-Tree store [C ] // 2016 USENIX Annual Technical Conference,June 22-24,2016,Denver,USA . Berkeley:USENIX Association , 2016 .

ZAMANIAN E , BINNIG C , HARRIS T , et al . The end of a myth:distributed transactions can scale [J ] . Proceedings of the VLDB Endowment , 2017 , 10 ( 6 ): 685 - 696 .

BERENSON H , BERNSTEIN P , GRAYJ , et al . A critique of ANSI SQL isolation levels [C ] // The 1995 ACM SIGMOD International Conference on Management of Data,May 22-25,1995,San Jose,USA . New York:ACM Press , 1995 : 1 - 10 .

WU M , YANG F , XUE J , et al . GraM:Scaling graph computation to the trillions [C ] // The 6th ACM Symposium on Cloud Computing (SoCC’15),August 27-29,2015,Kohala Coast,USA . New York:ACM Press , 2015 : 408 - 421 .

WEI X , SHEN S , CHEN R , et al . Replication-driven live reconfiguration for fast distributed transaction processing [C ] // 2017 USENIX Annual Technical Conference,July 12-14,2017,Santa Clara,USA . Berkeley:USENIX Association , 2017 : 335 - 347 .

POKE M , HOEFLER T . Dare:highperformance state machine replication on RDMA networks [C ] // The 24th International Symposium on High-Performance Parallel and Distributed Computing,June 15-19,2015,Portland,USA . New York:ACM Press , 2015 : 107 - 118 .

WANG C , JIANG J , CHEN X , et al . APUS:fast and scalable paxos on RDMA [C ] // The 2017 Symposium on Cloud Computing,September 25-27,2017,Santa Clara,USA . New York:ACM Press , 2017 : 94 - 107 .

WEI X , SHI J , CHEN Y , et al . Fast inmemory transaction processing using RDMA and HTM [C ] // The 25th Symposium on Operating Systems Principles,October 4-7,2015,Monterey,USA . New York:ACM Press , 2015 : 87 - 104 .

DEAN J , BARROSO L A . The tail at scale [J ] . Communications of the ACM , 2013 , 56 ( 2 ): 74 - 80 .

SHI J , YAO Y , CHEN R , et al . Fast and concurrent RDF queries with RDMAbased distributed graph exploration [C ] // The 12th USENIX Conference on Operating Systems Design and Implementation,November 2-4,2016,Savannah,USA . Berkeley:USENIX Association , 2016 : 317 - 332 .

AJOUX P , BRONSON N , KUMAR S , et al . Challenges to adopting stronger consistency at scale [C ] // The 15th USENIX Conference on Hot Topics in Operating Systems,May 18-20,2015,Kartause Ittingen,Switzerland . Berkeley:USENIX Association , 2015 .

DAGLIS A , USTIUGOV D , NOVAKOVIĆS , et al . Sabres:atomic object reads for in-memory rack-scale computing [C ] // The 49th Annual IEEE/ACM International Symposium on Microarchitecture,October 15-19,2016,Taipei,China . Piscataway:IEEE Press , 2016 : 1 - 13 .

RAIKIN S , LISS L , SHACHAR A , et al . Remote transactional memory:20150269116 [P ] .2015-09-24.

LI B , RUAN Z , XIAO W , et al . KV-direct:high-performance in-memory keyvalue store with programmable NIC [C ] // The 26th Symposium on Operating Systems Principles,October 28-31,2017,Shanghai,China . New York:ACM Press , 2017 : 137 - 152 .

Views

2311

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Survey on data caching technology of distributed dataflow system

Research on the consensus of big data systems based on RDMA and NVM

Graph processing engine based on graph query system

Related Author

Xuchu YUAN

Guo FU

Jize BI

Yanfeng ZHANG

Tiezheng NIE

Yu GU

Yubin BAO

Ge YU

Related Institution

Department of Computer Science and Technology, Tsinghua University

Beijing National Research Center for Information Science and Technology

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273380 Visits Today：286

⁰