流式大数据实时处理技术、平台及应用

陈纯

doi:10.11959/j.issn.2096-0271.2017036

您当前的位置：

首页 >

文章列表页 >

流式大数据实时处理技术、平台及应用

聚焦 | 更新时间：2024-06-03

- 流式大数据实时处理技术、平台及应用
- Real-time processing technology,platform and application of streaming big data
- 大数据 2017年3卷第4期页码：1-8
- 作者机构：
- 作者简介：
  
  [ "陈纯（1955-），男，博士，浙江大学计算机科学与技术学院教授，中国工程院院士，计算机应用专家，主要研究方向为大数据智能计算、计算机图形图像处理等。" ]
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2017036
  中图分类号： TP319
- 网络首发：2017-07，
  
  纸质出版：2017-07-20
- 稿件说明：
移动端阅览
陈纯. 流式大数据实时处理技术、平台及应用[J]. 大数据, 2017,3(4):1-8.

Chun CHEN. Real-time processing technology,platform and application of streaming big data[J]. Big data research, 2017, 3(4): 1-8.
陈纯. 流式大数据实时处理技术、平台及应用[J]. 大数据, 2017,3(4):1-8. DOI： 10.11959/j.issn.2096-0271.2017036.

Chun CHEN. Real-time processing technology,platform and application of streaming big data[J]. Big data research, 2017, 3(4): 1-8. DOI： 10.11959/j.issn.2096-0271.2017036.

摘要

大数据处理系统根据其时效性可分为批式大数据和流式大数据两类。上述两类系统均无法满足“事中”感知查询分析处理模式的需求。为此，从分析大数据应用场景入手，提出了“流立方”流式大数据实时处理技术和平台，在完整大数据集上实现了低迟滞、高实时的即席查询分析。目前基于“流立方”平台开发的业务系统已应用到金融风控反欺诈、机器防御等领域，具有广阔的应用前景。

Abstract

According to its timeliness

big data processing systems can be categorized into two groups

namely batching big data processing and streaming big data processing.Both systems mentioned above are unable to meet the real-time requirement for censoring and query analysis tasks.To this end

the “stream cube” real-time data analysis technology and platform were presented

which can perform timely query with low lag.Currently

this technology has been applied to many fields

including financial risk management

anti-fraud as well as web bots defense

and offers promising prospects for further applications.

关键词

Keywords

references

SHVACHKO K , KUANG H , RADIA S , et al . The Hadoop distributed file system [C ] // 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST),May 3-7,2010,Incline Village,NV,USA,USA . New Jersey:IEEE Press , 2010 : 1 - 10 .

DEAN J , GHEMAWAT S . MapReduce:simplified data processing on large clusters [J ] . Communications of the ACM , 2008 , 51 ( 1 ): 107 - 113 .

CHANG F , DEAN J , GHEMAWAT S , et al . Bigtable:a distributed storage system for structured data [J ] . ACM Transactions on Computer Systems (TOCS) , 2008 , 26 ( 2 ): 4 .

ZAHARIA M , DAS T , LI H , et al . Discretized streams:an efficient and faulttolerant model for stream processing on large clusters [J ] . HotCloud , 2012 ( 12 ): 10 .

TOSHNIWAL A , TANEJA S , SHUKLA A , et al . Storm@ twitter [C ] // The 2014 ACM SIGMOD International Conference on Management of Data,June 22-27,2014,Snowbird,Utah,USA . New York:ACM Press , 2014 : 147 - 156 .

CARBONE P , KATSIFODIMOS A , EWEN S , et al . Apache flink:stream and batch processing in a single engine [J ] . Bulletin of the IEEE Computer Society Technical Committee on Data Engineering , 2015 , 36 ( 4 ): 28 - 38 .

浏览量

2233

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

暂无数据