Data wrangling:a key technique of data governance

Xiaoyong DU; Yueguo CHEN; Ju FAN; Wei LU

doi:10.11959/j.issn.2096-0271.2019020

您当前的位置：

首页 >

文章列表页 >

Data wrangling:a key technique of data governance

TOPIC：BIG DATA GOVERNANCE | 更新时间：2024-06-03

- Data wrangling:a key technique of data governance
- Big Data Research Vol. 5, Issue 3, Pages: 2019020-1(2019)
- 作者机构：
  
  1. 中国人民大学信息学院，北京 100872
  2. 数据工程与知识工程教育部重点实验室（中国人民大学），北京 100872
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2019020
  CLC： TP315
- Online First：2019-05，
  
  Published：15 May 2019
- 稿件说明：
移动端阅览
Xiaoyong DU, Yueguo CHEN, Ju FAN, et al. Data wrangling:a key technique of data governance[J]. Big Data Research, 2019, 5(3): 2019020-1.
DOI：

Xiaoyong DU, Yueguo CHEN, Ju FAN, et al. Data wrangling:a key technique of data governance[J]. Big Data Research, 2019, 5(3): 2019020-1. DOI： 10.11959/j.issn.2096-0271.2019020.

摘要

数据是政府、企业和机构的重要资源。数据治理关注数据资源有效利用的众多方面，如数据资产确权、数据管理、数据开放共享、数据隐私保护等。从数据管理的角度，探讨了数据治理中的一项关键技术：数据整理。介绍了以数据拥有者和直接使用者（行业用户）为核心的数据整理的关键技术，包括数据结构化处理、数据质量评估及数据清洗、数据规范化、数据融合与摘取、数据整理的发布共享等。最后，针对加强数据整理方面的研究提出了一些思考。

Abstract

Data is an important resource for governments

businesses and institutions.Data governance focuses on many aspects of effective use of data resources

such as data asset

data management

data sharing

and data privacy.A key techniquedata wrangling-in data governance from the perspective of data management was explored.The key technologies of data wrangling based on data owners and direct users-industry users were emphasized

including data structure processing

data quality assessment and data cleaning

data normalization

data fusion and extraction

data publishing and sharing

etc.Finally

some thoughts on strengthening the research on data organization were put forward.

关键词

Keywords

references

HELLERSTEIN J M , HEER J , KANDEL S . Self-service data preparation:research to practice [J ] . IEEE Data Engineering Bulletin , 2018 , 41 ( 2 ): 23 - 34 .

HEER J , HELLERSTEIN J M , KANDEL S . Data wrangling [M ] // Encyclopedia of big data technologies 2019.[S.l.:s.n] . 2019 .

LI G L , ZHENG Y D , FAN J , et al . Crowdsourced data management:overview and challenges [C ] // The 2017 ACM International Conference on Management of Data,May 1419,2017,Chicago,USA . New York:ACM Press , 2017 : 1711 - 1716 .

DOAN A H , ARDALAN A , BALLARD J R , et al . Toward a system building agenda for data integration [J ] . IEEE Data Engineering Bulletin , 2018 , 41 ( 2 ): 35 - 46 .

宋晓宇 , 王永会 . 数据集成与应用集成 [M ] . 北京 : 中国水利水电出版社 , 2008 .

SONG X Y , WANG Y H . Data integration and application integration [M ] . Beijing : China Water and Power PressPress , 2008 .

ABEDJAN Z , CHU X , DENG D , et al . Detecting data errors:where are we and what needs to be done [J ] . Proceedings of the VLDB Endowment , 2016 , 9 ( 12 ): 993 - 1004 .

BOHANNON P , FAN W F , GEERTS F , et al . Conditional functional dependencies for data cleaning [C ] // 2007 IEEE 23rd International Conference on Data Engineering,April 15-20,2007,Istanbul,Turkey . Piscataway:IEEE Press , 2007 : 746 - 755 .

CHU X , ILYAS I F , PAPOTTI P . Holistic data cleaning:putting violations into context [C ] // 2013 IEEE 29th International Conference on Data Engineering (ICDE),April 8-12,2013,Brisbane,Australia . Piscataway:IEEE Press , 2013 : 458 - 469 .

CHU X , MORCOS J , ILYAS I F , et al . KATARA:a data cleaning system powered by knowledge bases and crowdsourcing [C ] // The 2015 ACM SIGMOD International Conference on Management of Data,May 31-June 4,2015,Melbourne,Australia . New York:ACM Press , 2015 : 1247 - 1261 .

YAKOUT M , BERTI-ÉQUILLE L , ELMAGARMID A K . Don’t be scared:use scalable automatic repairing with maximal likelihood and bounded changes [C ] // The 2013 ACM SIGMOD International Conference on Management of Data,June 22-27,2013,New York,USA . New York:ACM Press , 2013 : 553 - 564 .

REKATSINAS T , CHU X , ILYAS I F , et al . HoloClean:holistic data repairs with probabilistic inference [J ] . Proceedings of the VLDB Endowment , 2017 , 10 ( 11 ): 1190 - 1201 .

Views

2630

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Progress on human-in-the-loop data preparation

Study on big data governance standard system

Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology

Challenges and technical development of storage systems in the era of large language models

Research progress of government data identification technology and the next generation government data identification system

Related Author

Xiaoyong DU

Ju FAN

Yueguo CHEN

Hong DAI

Qun ZHANG

Zhuo YIN

JIANG Nan

SHU Jiwu

Related Institution

School of Information,Renmnin University of China

DEKE Lab ＆Information School,Renmnin University of China

China Petroleum & Chemical Corporation

College of Engineering, Peking University

Department of Computer Science and Technology, Tsinghua University

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273380 Visits Today：286

⁰