Research Progress on Big Data Machine Learning System

Yihua Huang

doi:10.11959/j.issn.2096-0271.2015.01.004

您当前的位置：

首页 >

文章列表页 >

Research Progress on Big Data Machine Learning System

Topic:Key technologies of big data | 更新时间：2024-06-03

- Research Progress on Big Data Machine Learning System
- BIG DATA RESEARCH Vol. 1, Issue 1, Pages: 26-45(2015)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2015.01.004
  CLC：
- Online First：2015-05，
  
  Published：03 May 2015
- 稿件说明：
移动端阅览
Yihua Huang. Research Progress on Big Data Machine Learning System[J]. BIG DATA RESEARCH, 2015, 1(1): 26-45.
DOI：

Yihua Huang. Research Progress on Big Data Machine Learning System[J]. BIG DATA RESEARCH, 2015, 1(1): 26-45. DOI： 10.11959/j.issn.2096-0271.2015.01.004.

摘要

要实现高效的大数据机器学习，需要构建一个能同时支持机器学习算法设计和大规模数据处理的一体化大数据机器学习系统。研究设计高效、可扩展且易于使用的大数据机器学习系统面临诸多技术挑战。近年来，大数据浪潮的兴起，推动了大数据机器学习的迅猛发展，使大数据机器学习系统成为大数据领域的一个热点研究问题。介绍了国内外大数据机器学习系统的基本概念、基本研究问题、技术特征、系统分类以及典型系统；在此基础上，进一步介绍了本实验室研究设计的一个跨平台统一大数据机器学习系统——Octopus（大章鱼）。

Abstract

To achieve efficient big data machine learning

we need to construct a unified big data machine learning system to support both machine learning algorithm design and big data processing. Designing an efficient

scalable and easy-to-use big data machine learning system still faces a number of challenges. Recently

the upsurge of big data technology has promoted rapid development of big data machine learning

making big data machine learning system to become a research hotspot. The basic concepts

research issues

technical characteristics

关键词

Keywords

references

Banko M , Brill E , KIEKINTVELD C , . Scaling to very large corpora for natural language disambiguation . Proceedings of the 39th Annual Meeting on Association for Computational Linguistics (ACL) ， Toulouse, France , 2001 , 26 ～ 33 .

Brants T , Popat C A , Xu P , et al . Large language models in machine translation . Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Language Learning ， Prague, Czech Republic , 2007 , 858 - 867 .

Wang Y , Zhao X M , Sun Z L , et al . Peacock: learning long-tail topic features for industrial applications . ACM Transactions on Intelligent Systems and Technology , 2014 , 9 ( 4 )

中国计算机学会大数据专家委员会 . 2015年中国大数据发展趋势预测 . 中国计算机学会通讯 , 2014 , 11 ( 1 ): 48 - 52 .

CCF Task Force on Big Data . Forecast for the development trend of big data in 2015 Communications of the China Computer Federation (CCCF) , 2014 , 11 ( 1 ): 48 - 52 .

Gonzalez J E Emerging systems for large-scale machine learning . Proceedings of Tutorial on International Conference for Machine Learning(ICML) 2014 ， Beijing, China , 2014

中国计算机学会大数据专家委员会 . 2014年中国大数据技术与产业发展白皮书 . 2014中国大数据技术大会 , 北京, 中国 , 2014

CCF Task Force on Big Data . White paper of China’s big data technology and industrial development in 2014 Proceedings of Big Data Conference China , Beijing, China , 2014

Boehm M , Tatikonda S , Reinwald B . et al . Hybrid parallelization strategies for large-scale machine learning in systemML . Proceedings of the VLDB Endowment ， Hangzhou, China 2014

Markl V , YIN Z . Breaking the chains: on declarative data analysis and data independence in the big data era . Proceedings of the VLDB Endowment ， Hangzhou, China 2014

Kraska T T . MLbase: a distributed machine-learning system . Proceedings of the 6th Conference on Innovative Data Systems Research(CIDR) , Asilomar, CA, USA , 2013

Fan W F , Geerts F , Neven F , . Making queries tractable on big data with preprocessing: through the eyes of complexity theory . Proceedings of the VLDB Endowment , Trento, Italy , 2011 : 685 ～ 696 .

Dean J , Ghemawat S . MapReduce:simplified data processing on large clusters . Communications of the ACM , 2004 , 51 ( 1 ): 107 ～ 113 .

Zaharia M , Chowdhury M , Das T , et al . Resilient distributed datasets:a fault-tolerant abstraction for in-memory cluster computing . Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation(NSDI) , San Jose, CA, USA , 2012 141 - 146 .

Venkataraman S , Bodzsar E , Roy I , et al . Presto: distributed machine learning and graph processing with sparse matrices . Proceedings of the 8th ACM European Conference on Computer Systems(EuroSys) , Prague, Czech Republic , 2013 197 - 210 .

Ghoting A , Krishnamurthy S , Pednault E , et al . SystemML: declarative machine learning on MapReduce . Proceedings of International Conference on Data Engineering (ICDE) , Hannover, Germany , 2011 231 ～ 242 .

Boehm M , Tatikonda S , Reinwald B , et al . Hybrid parallelization strategies for large-scale machine learning in SystemML . Proceedings of the VLDB Endowment , Hangzhou, China , 2011 231 - 242 .

Low Y , Bickson D , Gonzalez J , et al . Distributed graphLab: a framework for machine learning and data mining in the cloud . Proceedings of the VLDB Endowment , Istanbul, Turkey , 2012 716 ～ 727 .

Li M , Andersen G D , Park W J , et al . Scaling distributed machine learning with the parameter server . Proceedings of Operating Systems Design and Implementation (OSDI) , Broomfield, CD, USA , 2014 ； 583 ～ 598 .

Ho Q , Cipar J , Cui H W J , et al . More effective distributed ml via a stale synchronous parallel parameter server . Proceedings of Advances in Neural Information Processing Systems (NIPS) , Nevada, USA , 2013 ： 1223 ～ 1231 .

Alexandrov A , Bergmann R , Ewen S , et al . The stratosphere platform for big data analytics . Vldb Journal , 2014 , 23 ( 6 ): 939 - 964 .

Battré D , Ewen S , Hueske F , et al . Nephele/PACTs: a programming model and execution framework for web-scale analytical processing . Proceedings of ACM Symposium on Cloud Computing(SoCC) , Indianapolis, Indiana, USA , 2010 ： 119 ～ 130 .

Dai W , Wei J , Zheng X , et al . Petuum:a framework for iterative-convergent distributed ML . Proceedings of Advances in Neural Information Processing Systems 26, Big Learning Workshop , California, USA , 2013

邹永强 . Mariana—腾讯深度学习平台的进展与应用 . 2014年中国大数据技术大会，北京，中国， 2014

Zou Y Q . Marina-the progress and application of deep learning platform of Tencent . Proceedings of Database Technology Conference China 2015 ， Beijing, China , 2014

刘伟 . 百度机器学习云平台 . 2014年中国大数据技术大会，北京，中国， 2015

Liu W . Machine learning cloud platform of Baidu Proceedings of Database Technology Conference China 2015 ， Beijing, China , 2015

Views

1661

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Programming frameworks for deep learning algorithms

A Model of Pre-Warning Based on the Big Data Technology for P2P Lending Platform

A scalable parallel sorting algorithm by regular sampling for big data

Data expansion method for genetic engineering of special materials with small sample data

Gansu smart tourism system based on big data

Related Author

Bingrui WANG

Huiying LAN

Yunji CHEN

Chunyu Lin

Chonggang Li

Fangyuan Xu

Huiquan Xu

Lei Shi

Related Institution

University of Science and Technology of China

Institute of Computing Technology Chinese Academy of Sciences

University of Chinese Academy of Sciences

Beijing JinXinWangYin Financial Information Service Co., Ltd.

State Grid Energy Research Institute

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273651 Visits Today：557

⁰