Cluster computing mode for water environment simulation based on Hadoop

Jinfeng MA; Li TANG; Kaifeng RAO; Gang HONG; Mei MA

doi:10.11959/j.issn.2096-0271.2019051

您当前的位置：

首页 >

文章列表页 >

Cluster computing mode for water environment simulation based on Hadoop

STUDY | 更新时间：2024-06-03

- Cluster computing mode for water environment simulation based on Hadoop
- Big Data Research Vol. 5, Issue 6, Pages: 2019051-1(2019)
- 作者机构：
  
  1. 中国科学院饮用水科学与技术重点实验室（中国科学院生态环境研究中心），北京 100085
  2. 深圳市环境监测中心站，广东深圳 518057
  3. 石家庄市环境信息中心，河北石家庄 050051
  4. 中国科学院大学资源与环境学院，北京 100190
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2019051
  CLC： TP311
- Online First：2019-11，
  
  Published：15 November 2019
- 稿件说明：
移动端阅览
Jinfeng MA, Li TANG, Kaifeng RAO, et al. Cluster computing mode for water environment simulation based on Hadoop[J]. Big Data Research, 2019, 5(6): 2019051-1.
DOI：

Jinfeng MA, Li TANG, Kaifeng RAO, et al. Cluster computing mode for water environment simulation based on Hadoop[J]. Big Data Research, 2019, 5(6): 2019051-1. DOI： 10.11959/j.issn.2096-0271.2019051.

摘要

水环境数值模型是模拟、分析及预测水体中物质迁移转化过程及其效应的有效工具。水环境模型的高性能批量计算是当前水环境模拟研究的热点。大数据技术中的分布式集群计算模式为水环境模拟批量计算提供一种可行的解决方案。探索了水环境数值模型在大数据分布式计算框架下的适应性，提出了一种适用于水环境模拟的大数据分布式集群运算模式，并通过实例验证了该运算模式的可行性。

Abstract

Water environment numerical models are effective tools for the simulation

analysis and prediction of the processes of pollutant transport and transformation in water.The development of high-performance batch computation of water environment models has long been a hot topic.The distributed cluster computing mode based on big data technology is a promising approach for massive data management and batch computation

which provides a viable solution to large-scale water environment simulations.The adaptability of water environmental models under the framework of big data technology was explored

and a distributed cluster computing mode for water environment simulations was proposed.Moreover

the feasibility of adapting Delft3D model for cluster computing under Hadoop MapReduce environment was verified with real examples.

关键词

Keywords

references

陈永灿 , 刘昭伟 , 朱德军 . 水动力及水环境模拟方法与应用 [M ] . 北京 : 科学出版社 , 2012 .

CHEN Y C , LIU Z W , ZHU D J . Hydrodynamic and water environment simulation methods and applications [M ] . Beijing : Science PressPress , 2012 .

彭泽洲 , 杨天行 , 梁秀娟 . 水环境数学模型及其应用 [M ] . 北京 : 化学工业出版社 , 2007 : 98 - 103 .

PENG Z Z , YANG T X , LIANG X J . Mathematical model of water environment and its application [M ] . Beijing : Chemical Industry PressPress , 2007 .

NGUYEN T T , KEUPERS I , WILLEMS P . Conceptual river water quality model with flexible model structure [J ] . Environmental Modelling ＆ Software , 2018 , 104 : 102 - 117 .

MELCHING C S . Application of a water quality model for determining instream aeration station location and operational rules:a case study [J ] . Water Science and Engineering , 2018 , 11 ( 1 ): 8 - 16 .

KEUPERS I , WILLEMS P . Development and testing of a fast conceptual river water quality model [J ] . Water Research , 2017 , 113 : 62 - 71 .

REDER K , ALCAMO J , FLORKE M . A sensitivity and uncertainty analysis of a continental-scale water quality model of pathogen pollution in African rivers [J ] . Ecological Modelling , 2017 , 351 : 129 - 139 .

贾仰文 , 王浩 , 严登华 . 黑河流域水循环系统的分布式模拟(Ⅰ)-模型开发与验证 [J ] . 水利学报 , 2006 ( 5 ): 534 - 542 .

JIA Y W , WANG H , YAN D H . Distributed model of hydrological cycle system in Heihe River basin Ⅰ-model development and verification [J ] . Journal of Hydraulic Engineering , 2006 ( 5 ): 534 - 542 .

王永桂 . 流域大尺度水环境模型的高效能集群计算方法研究及其在三峡库区的应用 [D ] . 武汉:武汉大学 , 2015 .

WANG Y G . Research on high productivity cluster computing technology of large scale water environment model and the application in the Three Gorges Area [D ] . Wuhan:Wuhan University , 2015 .

WANG H , FU X D , WANG G Q , et al . A common parallel computing framework for modeling hydrological processes of river basins [J ] . Parallel Computing , 2011 , 37 ( 6 ): 302 - 315 .

左一鸣 , 崔广柏 . 二维水动力模型的并行计算研究 [J ] . 水科学进展 , 2008 , 19 ( 6 ): 846 - 850 .

ZUO Y M , CUI G B . Parallel computation for 2D flow model [J ] . Advances in Water Science , 2008 , 19 ( 6 ): 846 - 850 .

许栋 PAyet D 及春宁 . 等 . 浅水方程大规模并行计算模拟城市洪水演进 [J ] . 天津大学学报(自然科学与工程技术版) , 2016 , 49 ( 4 ): 341 - 348 .

XU D , PAYET D , JI C N , et al . Simulation of flood propagation in cities using large scale parallel computation of shallow water equations [J ] . Journal of Tianjin University , 2016 , 49 ( 4 ): 341 - 348 .

靖争 . 湖泊水环境数值模拟及非静压水流并行计算方法研究 [D ] . 武汉:华中科技大学 , 2017 .

JING Z . Numerical simulations of the lake water environment and parallel computations of the non-hydrostatic flows [D ] . Wuhan:Huazhong University of Science and Technology , 2017 .

赵旭东 , 梁书秀 , 孙昭晨 , 等 . 基于GPU并行算法的水动力数学模型建立及其效率分析 [J ] . 大连理工大学学报 , 2014 , 54 ( 2 ): 204 - 209 .

ZHAO X D , LIANG S X , SUN Z C , et al . Foundation and analysis of computational efficiency for hydrodynamic model based on GPU parallel algorithm [J ] . Journal of Dalian University of Technology , 2014 , 54 ( 2 ): 204 - 209 .

宋利祥 , 范光伟 , 张文明 , 等 . 洪水演进高速模拟系统设计与实现 [J ] . 水电能源科学 , 2017 , 35 ( 12 ): 33 - 36 .

SONG L X , FAN G W , ZHANG W M , et al . Design and implementation of high speed simulation for flood routing [J ] . Water Resources and Power , 2017 , 35 ( 12 ): 33 - 36 .

ZHANG D , CHEN X , YAO H , et al . Moving SWAT model calibration and uncertainty analysis to an enterprise Hadoop-based cloud [J ] . Environmental Modelling ＆Software , 2016 , 84 ( C ): 140 - 148

WHITE T . Hadoop:the definitive guide [M ] . Sebastopol : O’Reilly Media,Inc.Press , 2010 .

MURTHY A , VAVILAPALLI V , EADLINE D , et al . Apache Hadoop YARN [M ] . Boston : Addison-WesleyPress , 2014 .

Views

175

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Research on the integration of water environment model and big data technology

A scalable parallel sorting algorithm by regular sampling for big data

System performance optimization practice for big data scenarios

Gansu smart tourism system based on big data

Big data and computing models

Related Author

Jinfeng MA

Kaifeng RAO

Ruonan LI

Jing ZHANG

Hua ZHENG

Ying WANG

Zhiguang CHEN

Yutong LU

Related Institution

State Key Laboratory of Urban and Regional Ecology, Research Center for Eco-Environmental Sciences, Chinese Academy of Sciences

niversity of Chinese Academy of Sciences

China Mobile Information Technology Center

Beihang University

China Mobile Information Technology Co., Ltd.

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273228 Visits Today：134

⁰