Towards unified big data platform

Ling QIAN; Shaoling SUN; Zaihui SHI

doi:10.11959/j.issn.2096-0271.2017032

您当前的位置：

首页 >

文章列表页 >

Towards unified big data platform

APPLICATION | 更新时间：2024-06-03

- Towards unified big data platform
- Big data research Vol. 3, Issue 3, Pages: 2017032(2017)
- 作者机构：
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2017032
  CLC： TP311.1
- Online First：2017-05，
  
  Published：20 May 2017
- 稿件说明：
移动端阅览
Ling QIAN, Shaoling SUN, Zaihui SHI. Towards unified big data platform[J]. Big data research, 2017, 3(3): 2017032.
DOI：

Ling QIAN, Shaoling SUN, Zaihui SHI. Towards unified big data platform[J]. Big data research, 2017, 3(3): 2017032. DOI： 10.11959/j.issn.2096-0271.2017032.

摘要

企业在大数据引入过程中，由于平台技术不成熟、缺乏统一技术架构、组织架构不适应等问题，形成了应用竖井和复杂的系统混搭结构，造成了资源利用率不均、数据重复存储、运维成本高、难以形成企业级数据共享等问题。目前大数据平台技术已经逐步成熟，大数据统一平台逐步成为企业分析系统的核心。深入分析了企业级统一大数据平台的整合思路，从管理、业务和技术3个层面入手，分别从组织架构体系建设、大数据平台运营流程重构与实施、大数据平台技术规划与实现等角度提出对策建议。最后，给出中国移动在统一大数据平台整合方面的实践经验。

Abstract

Due to problems like immaturity of platform technology

lack of unified technical framework and inadaptation of organization structure

silo applications and complicated mashup structures were formed when enterprises were introducing big data technology.With maturity of big data platform technology

unified big data platform was gradually becoming the core of enterprise analysis system.The thoughts of enterprise-level big data platform integration from three aspects

management

operation and technology were analyzed.Suggestions about organization structure development

reconstruction and implementation of big data platform operation process

technology planning and realization of platform were provided.At last

practical experiences of unified big data platform integration of China Mobile were presented.

关键词

Keywords

references

李国杰 . 对大数据的再认识 [J ] . 大数据 , 2015 , 1 ( 1 ):2015001.

LI G J . Further understanding of big data [J ] . Big Data Research , 2015 , 1 ( 1 ):2015001.

程学旗 , 靳小龙 , 王元卓 , 等 . 大数据系统和分析技术综述 [J ] . 软件学报 , 2014 , 25 ( 9 ): 1889 - 1908 .

CHENG X Q , JIN X L , WANG Y Z , et al . Survey on big data system and analytic technology [J ] . Journal of Software , 2014 , 25 ( 9 ): 1889 - 1908 .

GHEMAWAT S , GOBIOFF H , LEUNG S T . The Google file system [C ] // The 19th ACM Symposium on Operating Systems Principles,October 19-22,2003,New York,USA . New York:ACM Press , 2003 : 1 - 15 .

DEAN J , GHEMAWAT S . MapReduce:simplifed data processing on large clusters [C ] // The 6th Conference on Symposium on Operating Systems Design＆ Implementation,December 6-8,2004,San Francisco,CA,USA . New York:ACM Press , 2004 :10.

CHANG F , DEAN J , GHEMAWAT S , et al . Bigtable:a distributed storage system for structured data [J ] . ACM Transactions on Computer Systems , 2008 , 26 ( 2 ): 15 .

孟祥飞 , 冯景华 , 赵洋 , 等 . 应用驱动的大数据融合平台建设 [J ] . 大数据 , 2017 , 3 ( 2 ): 67 - 77 .

MENG X F , FENG J H , ZHAO Y , et al . Application-oriented integration platform construction on big data [J ] . Big Data Research , 2017 , 3 ( 2 ): 67 - 77 .

马朝辉 , 聂瑞华 , 谭昊翔 , 等 . 大数据治理的数据模式与安全 [J ] . 大数据 , 2016 , 2 ( 3 ): 83 - 95 .

MA C H , NIE R H , TAN H X , et al . Research on data schema and security in data governance [J ] . Big Data Research , 2016 , 2 ( 3 ): 83 - 95 .

孙少陵 , 周大 , 钱岭 . 云数据仓库高性能查询技术研究 [J ] . 邮电设计技术 , 2011 ( 10 ): 23 - 26 .

SUN S L , ZHOU D , QIAN L . High performance query technique of cloud data warehouse [J ] . Designing Techniques of Posts and Telecommunications , 2011 ( 10 ): 23 - 26 .

周大 , 钱岭 , 郭磊涛 , 等 . HugeTable:一种面向电信行业的云数据仓库 [J ] . 2011中国计算机学会服务计算学术会议,济南,中国,2011-08-18 . 济南:[出版者不详] , 2011 : 1 - 4 .

ZHOU D , QIAN L , GUO L T , et al . HugeTable:telecom oriented data warehouse [C ] // 2011CCF National Conference on Service Computing,August 18,2011,Jinan,China . Jinan:[s.n.] : 2011 : 1 - 4 .

BAO Y , WANG Z , BAI Q , et al . BC-BSP:a BSP-based system with disk cache for large-scale graph processing [J ] . Open Cirrus Summit , 2012 , 7215 ( 1 ): 35 - 39 .

GUO L T , SUN H W , LUO Z G . A data distribution aware task scheduling strategy for mapreduce system [C ] // The 1st International Conference on Cloud Computing,December 1-4,2009,Beijing,China . Berlin:Springer Press , 2009 : 694 - 699 .

YU L , DUAN S Q , SHEN C W , et al . BC-PDM:data mining,social network analysis and text mining system based on cloud computing [C ] // The 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining,August 12-16,2012,Beijing,China . New York:ACM Press , 2012 : 1496 - 1499 .

Views

941

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

System performance optimization practice for big data scenarios

Design and application of unified big data platform of information system for the 14th National Games of the People’s Republic of China

Research on the integration of water environment model and big data technology

Cluster computing mode for water environment simulation based on Hadoop

Related Author

Jiaxi WU

Yifei ZHANG

Zhiwen XIAO

Kelun LEI

Minda ZHANG

Xin SUN

Kai FENG

Hailong YANG

Related Institution

China Mobile Information Technology Center

Beihang University

China Mobile Information Technology Co., Ltd.

State Key Laboratory of Urban and Regional Ecology, Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences

niversity of Chinese Academy of Sciences

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273386 Visits Today：292

⁰