Applications and challenges of language virtual machines in big data

Mingyu WU; Haibo CHEN; Binyu ZANG

doi:10.11959/j.issn.2096-0271.2020035

您当前的位置：

首页 >

文章列表页 >

Applications and challenges of language virtual machines in big data

TOPIC：HETEROGENEOUS PARALLEL SYSTEMS FOR BIG DATA | 更新时间：2024-06-03

- Applications and challenges of language virtual machines in big data
- Big Data Research Vol. 6, Issue 4, Pages: 2020035-1(2020)
- 作者机构：
  
  1. 领域操作系统教育部工程研究中心，上海 200240
  2. 上海交通大学软件学院并行与分布式系统研究所，上海 200240
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2020035
  CLC： TP315
- Online First：2020-07，
  
  Published：15 July 2020
- 稿件说明：
移动端阅览
Mingyu WU, Haibo CHEN, Binyu ZANG. Applications and challenges of language virtual machines in big data[J]. Big Data Research, 2020, 6(4): 2020035-1.
DOI：

Mingyu WU, Haibo CHEN, Binyu ZANG. Applications and challenges of language virtual machines in big data[J]. Big Data Research, 2020, 6(4): 2020035-1. DOI： 10.11959/j.issn.2096-0271.2020035.

摘要

语言虚拟机为大数据应用提供了与平台无关的执行环境，简化了应用的开发和部署，因此在大数据场景中得到了较广泛的应用。主要分析了两种主流语言虚拟机——JVM和CLR在大数据场景中的应用，并阐述了使用语言虚拟机面临的4个挑战：初始化及“热身”开销、垃圾回收暂停、异构内存支持、数据格式转换。之后，分别针对4个挑战讨论了现有的解决方案，并分析了这些方案的不足之处及未来可能的优化方向。

Abstract

Language virtual machines provide a platform-independent execution environment for big-data applications and simplify their development and deployment phases

so they are widely used in the big-data scenario.The applications of two different kinds of mainstream language virtual machines:JVM and CLR

were analyzed

and four challenges when adopting language virtual machines:initialization and warm-up overhead

garbage collection pauses

heterogeneous memory support

and data layout transformation

were summarized.Afterward

existing approaches to the challenges were discussed and their shortcomings and possible optimizations in the future were analyzed.

关键词

Keywords

references

WHITE T . Hadoop:the definitive guide [M ] . Sebastopol : O’Reilly Media,Inc.Press , 2012 :647.

ZAHARIA M , XIN R , WENDELL P , et al . Apache Spark:a unified engine for big data processing [J ] . Communications of the ACM , 2016 , 59 ( 11 ): 56 - 65 .

CARBONE P , KATSIFODIMOS A , EWEN S , et al . Apache Flink:stream and batch processing in a single engine [J ] . Bulletin of the IEEE Computer Society Technical Committee on Data Engineering , 2015 , 36 ( 4 ): 28 - 38 .

YU Y , ISARD M , FETTERLY D , et al . DryadLINQ:a system for general-purpose distributed data-parallel computing using a high-level language [C ] // The 8th USENIX Symposium on Operating Systems Designand Implementation . Berkeley:USENIX Association , 2008 : 383 - 400 .

LINDHOLM T , YELLIN F , BRACHA G , et al . The Java virtual machine specification [M ] . London : Pearson EducationPress , 2014 :584.

KENNEDY A , SYME D.Design and implementation of generics for the . NET common language runtime [C ] // The ACM SIGPLAN 2001 Conference on Programming Language Design and Implementation . New York:ACM Press , 2001 : 1 - 12 .

DU D Y . Apache Hive essentials [M ] . Birmingham : Packt Publishing LtdPress , 2015 :191.

BORKAR V , CAREY M , GROVER R , et al . Hyracks:a flexible and extensible foundation for data-intensive computing [C ] // The IEEE 27th International Conference on Data Engineering . Piscataway:IEEE Press , 2011 : 1151 - 1162 .

IQBAL M H , SOOMRO T R . Big data analysis:Apache Storm perspective [J ] . International Journal of Computer Trends and Technology , 2015 , 19 ( 1 ): 9 - 14 .

ISARD M , BUDIU M , YU Y , et al . Dryad:distributed data-parallel programs from sequential building blocks [C ] // The 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems . New York:ACM Press , 2014 : 59 - 72 .

CHAIKEN R , JENKINS B , LARSON P , et al . SCOPE:easy and efficient parallel processing of massive data sets [J ] . Proceedings of the VLDB Endowment , 2008 , 1 ( 2 ): 1265 - 1276 .

PIALORSI P , RUSSO M . Introducing microsoft® LINQ [M ] . Redmond : Microsoft PressPress , 2007 :282.

LION D , CHIU A , SUN H , et al . Don’t get caught in the cold,warm-up your JVM:understand and eliminate JVM warm-up overhead in data-parallel systems [C ] // The 12th USENIX Symposium on Operating Systems Design and Implementation . Berkeley:USENIX Association , 2016 : 383 - 400 .

UNGAR D , . Generation scavenging:a nondisruptive high performance storage reclamation algorithm [C ] // The 1st ACM SIGSOFT/SIGPLAN Software Engineering Symposium on Practical Software Development Environments . New York:ACM Press , 1984 : 157 - 167 .

NGUYEN K , FANG L , XU G Q , et al . Yak:a high-performance big-data-friendly garbage collector [C ] // The 12th USENIX Symposium on Operating Systems Design and Implementation . Berkeley:USENIX Association , 2016 : 349 - 365 .

OpenJDK . JEP 316:heap allocation on alternative memory devices [Z ] . 2016 .

DRABAS T , LEE D . Learning PySpark [M ] . Birmingham : Packt Publishing LtdPress , 2017 :250.

VENKATARAMAN S , YANG Z H , LIU D , et al . SparkR:scaling R programs with Spark [C ] // The 2016 International Conference on Management of Data . New York:ACM Press , 2016 : 1099 - 1104 .

NGUYEN K , FANG L , NAVASCA C , et al . Skyway:connecting managed heaps in distributed big data systems [C ] // The 23rd International Conference on Architectural Support for Programming Languages and Operating Systems . New York:ACM Press , 2018 : 56 - 69 .

OpenJDK . JEP 310:application class-data sharing [Z ] . 2017 .

PIRVU M . Optimize JVM start-up with Eclipse OpenJ9 [R ] . 2018 .

WANG K A , HO R , WU P . Replayable execution optimized for page sharing for a managed runtime environment [C ] // The 14th EuroSys Conference 2019 . New York:ACM Press , 2019 : 1 - 16 .

BRUNO R , OLIVEIRA L P , FERREIRA P . NG2C:pretenuring garbage collection with dynamic generations for HotSpot big data applications [C ] // The 2017 ACM SIGPLAN International Symposium on Memory Management . New York:ACM Press , 2017 : 2 - 13 .

NGUYEN K , WANG K , BU Y Y , et al . FAÇADE:a compiler and runtime for (almost) object-bounded big data applications [C ] // The 20th International Conference on Architectural Support for Programming Languages and Operating Systems . New York:ACM Press , 2015 : 675 - 690 .

XIN R , ROSEN J . Project Tungsten:bringing Apache Spark closer to BareMetal [R ] . 2015 .

SHI X H , KE Z X , ZHOU Y L , et al . Deca:a garbage collection optimizer for in-memory data processing [J ] . ACM Transactions on Computer Systems , 2019 , 36 ( 1 ): 1 - 47 .

FANG L , NGUYEN K , XU G Q , et al . Interruptible tasks:treating memory pressure as interrupts for highly scalable data-parallel programs [C ] // The 25th Symposium on Operating Systems Principles . New York:ACM Press , 2015 : 394 - 409 .

TAN J , CHIN A , HU Z Z , et al . DynMR:dynamic MapReduce with reducetask interleaving and maptask backfilling [C ] // The 9th European Conference on Computer Systems . New York:ACM Press , 2014 : 1 - 14 .

YANG J , KIM J , HOSEINZADEH M , et al . An empirical guide to the behavior and use of scalable persistent memory [C ] // The 18th USENIX Conference on File and Storage Technologies . Berkeley:USENIX Association , 2020 : 169 - 182 .

WANG C X , CUI H M , CAO T , et al . Panthera:holistic memory management for big data processing over hybrid memories [C ] // The 40th ACM SIGPLAN Conference on Programming Language Design and Implementation . New York:ACM Press , 2019 : 347 - 362 .

WU M Y , CHEN H B , ZHU H , et al . GCPersist:an efficient GC-assisted lazy persistency framework for resilient Java applications on NVM [C ] // The 16th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments . New York:ACM Press , 2020 : 1 - 14 .

XU G Q , VEANES M , BARNETT M , et al . Niijima:sound and automated computation consolidation for efficient multilingual data-parallel pipelines [C ] // The 27th ACM Symposium on Operating Systems Principles . New York:ACM Press , 2019 : 306 - 321 .

NAVASCA C , CAI C , NGUYEN K , et al . Gerenuk:thin computation over big native data using speculative program transformation [C ] // The 27th ACM Symposium on Operating Systems Principles . New York:ACM Press , 2019 : 538 - 553 .

KRYO . Java binary serialization and cloning:fast,efficient,automatic [R ] . 2020 .

COLFER . The Colfer serializer [R ] . 2017 .

OpenJDK.Valhalla [R ] . 2020 .

GraalVM.Run programs faster anywhere [R ] . 2020 .

Views

710

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

No data

Related Author

No data

Related Institution

No data

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273228 Visits Today：134

⁰