Distributed processing practice of the massive GIS data based on HBase

Xuemei LI; Junfeng XING; IUDawei L; Haiyang WANG; Wei LIU

doi:10.11959/j.issn.2096-0271.2016032

您当前的位置：

首页 >

文章列表页 >

Distributed processing practice of the massive GIS data based on HBase

Topic:big data and smart city | 更新时间：2024-06-03

- Distributed processing practice of the massive GIS data based on HBase
- BIG DATA RESEARCH Vol. 2, Issue 3, Pages: 2016032(2016)
- 作者机构：
  
  1. 烟台中科网络技术研究所，山东烟台 264003
  2. 中国科学院计算技术研究所，北京 100080
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2016032
  CLC： TP301
- Online First：2016-05，
  
  Published：20 May 2016
- 稿件说明：
移动端阅览
Xuemei LI, Junfeng XING, IUDawei L, et al. Distributed processing practice of the massive GIS data based on HBase[J]. BIG DATA RESEARCH, 2016, 2(3): 2016032.
DOI：

Xuemei LI, Junfeng XING, IUDawei L, et al. Distributed processing practice of the massive GIS data based on HBase[J]. BIG DATA RESEARCH, 2016, 2(3): 2016032. DOI： 10.11959/j.issn.2096-0271.2016032.

摘要

设计了一种基于分布式数据库HBase的GIS数据管理系统。系统优化了栅格数据的生成和存储过程，将海量栅格数据直接写入HBase存储、索引。同时，针对矢量空间数据的存储、索引与检索，提出了一种新的rowkey设计，既考虑经纬度，又考虑空间数据类型和属性，使得在按空间位置检索矢量地理信息时，能通过HBase的rowkey迅速定位需要返回的数据。在HBase的集群环境上用真实GIS数据对上述方法进行了验证，结果表明，提出的系统具有较高的海量数据存储和检索性能，实现了海量地理信息数据的高效存储和实时高速检索。

Abstract

Based on the distributed database HBase

a kind of GIS data management system was designed.The system optimized the generated and stored procedures of raster data

which could be directly written into the storage and indexing of the HBase.At the same time

in view of the storing

indexing and retrieval of the vector spatial data

a new design for rowkey was proposed that considering both the latitude and longitude

and the spatial data types and attributes.So that the data needed to be returned could be quickly located by rowkey of the HBase

when retrieving vector geographic information according to the spatial location.The above methods had been verified on the HBase cluster environment with real GIS data.The results show that the proposed system has high performance for storage and retrieval of mass data

and realizes the efficient storage and real-time high-speed retrieval of the vast geographic information data.

关键词

Keywords

references

潘柱延 , 程学旗 , 袁晓如 , 等 . CCF大专委2016年大数据发展趋势预测——解读和行动建议 [J ] . 大数据 , 2016 , 2 ( 1 ): 105 - 113 .

PAN Z Y , CHENG X Q , YUAN X R , et al . Developing trend forecasting of big data in 2016 from CCF TFBD:interpretation and proposals [J ] . Big Data Research , 2016 , 2 ( 1 ): 105 - 113 .

CHANG F , DEAN J , GHEMAWA S , et al . Bigyable:a distributed storage system for structured data [C ] // The 7th USENIX Symposium on Operating Systems Design and Implementation,November 6-8,2006,Seattle,WA,USA . New York : ACM Press , 2006 : 205 - 218 .

陈嘉恒 . Hadoop实战 [M ] . 北京 : 机械工业出版社 , 2011 : 1 - 2,260-261 .

CHEN J H . Hadoop in action [M ] . Beijing : China Machine Press , 2011 : 1 - 2,260-261 .

陈崇成 , 林剑峰 , 吴小竹 , 等 . 基于NoSQL的海量空间数据云存储与服务方法 [J ] . 地球信息科学学报 , 2013 , 15 ( 2 ): 166 - 174 .

CHEN C C , LIN J F , WU X Z , et al . Massive geo-spatial data cloud storage and services based on NoSQL database technique [J ] . Journal of Geo-Information Science , 2013 , 15 ( 2 ): 166 - 174 .

范建永 , 龙明 , 熊伟 . 基于HBase的矢量空间数据分布式存储研究 [J ] . 地理信息与地理信息科学 , 2012 , 28 ( 5 ): 39 - 42 .

FAN J L , LONG M , XIONG W . Research of vector spatial data distributed storage based on HBase [J ] . Geography and Geo-Information Science , 2012 , 28 ( 5 ): 39 - 42 .

张叶 , 许国艳 , 花青 . 基于HBase的矢量空间数据存储与访问优化 [J ] . 计算机应用 , 2015 , 35 ( 11 ): 3102 - 3105 .

ZHANG Y , XU G Y , HUA Q . Storage and access optimization of vector space data based on HBase [J ] . Journal of Computer Applications , 2015 , 35 ( 11 ): 3102 - 3105 .

WANG L , CHEN B , LIU Y . Distributed storage and index of vector spatial data based on HBase [C ] // The 2013 21st International Conference on Geoinformatics,June 20-22,2013,Kaifeng,China . New Jersey : IEEE Press , 2013 : 1 - 5 .

陆锋 , 周成虎 . 一种基于Hilbert排列编码的GIS空间索引方法 [J ] . 计算机辅助设计与图形学学报 , 2001 , 13 ( 5 ): 424 - 429 .

LU F , ZHOU C H . A GIS spatial indexing approach based on Hilbert ordering code [J ] . Journal of Computer-Aided Design and Computer Graphics , 2001 , 13 ( 5 ): 424 - 429 .

SHEKHAR S , CHAWLA S . 空间数据库 [M ] . 谢昆青 ,译. 北京 : 机械工业出版社 , 2004 : 108 - 113 .

SHEKHAR S , CHAWLA S . Spatial database [M ] .Translated by XIE K Q . Beijing : China Machine Press , 2004 : 108 - 113 .

滕连泽 , 刘洪斌 , 武伟 . 基于空间数据不同索引方法的比较 [J ] . 计算机与现代化 , 2006 ( 1 ): 60 - 62 .

TENG L Z , LIU H B , WU W . Comparison of indexes based on spatial data [J ] . Computer and Modernization , 2006 ( 1 ): 60 - 62 .

HSU Y T , PAN Y C , WEI L , et al . Key formulation schemes for spatial index in cloud data managements [C ] // The 2012 IEEE 13th International Conference on Mobile Data Management,July 23-26,2012,Bengaluru,India . New Jersey : IEEE Press , 2012 : 21 - 26 .

Views

2585

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

A scalable parallel sorting algorithm by regular sampling for big data

Gansu smart tourism system based on big data

Big data and computing models

Analysis and research on the strategy and problems of “Channel Computing Resources from the East to the West”

Digital transformation in higher education:a systematic review

Related Author

Ying WANG

Zhiguang CHEN

Yutong LU

Liang GUO

Yi YANG

Bingfeng QIN

Jianwen CAO

Min LI

Related Institution

School of Information Science and Engineering, Lanzhou University

Key Laboratory of Tourism Information Fusion Processing and Data Ownership Protection, Ministry of Culture and Tourism, Lanzhou University

Technology Information Department, Gansu Provincial Department of Culture and Tourism

School of Computer and Artificial Intelligence, Lanzhou Polytechnical College

Key Laboratory of Artificial Intelligence and Computing Power Technology

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273493 Visits Today：399

⁰