摘要:With the rapid development of data analytics,data preparation has become a major bottleneck.The two essential challenges for data preparation on cost and time were analyzed.To address the challenges,the research progress on human-in-theloop data preparation was reviewed.Firstly,interactive data preparation was reviewed,which aimed to reduce the time for data preparation by predictively interacting with the end users.Then,crowdsourced data preparation was introduced,which utilize human’s computational power from the crowd to support foundamental data preparation tasks,and developed algorithms for controlling result quality and reducing crowdsourcing cost.Finally,future research directions were summarized and discussed.
摘要:Industrial big data has become an important strategic resource for the transformation and upgrading of China’s manufacturing industry,and industrial big data analysis is attracting more and more attention.As an important data form of industrial big data,time series have a lot of quality problems,which is necessary to be detected and handled effectively by designing data cleaning methods.The characteristics of industrial time series big data and the difficulties of industrial data quality management were introduced.Then,the recent developments in the area of that was analyzed and summarized.At last,the quality management method of temporal big data and the improvement direction of system performance were put forward.
关键词:data quality management;time series data;industrial big data analysis;data cleaning
摘要:Data curation is emerged in order to process,store and applied efficiency.Data curation processes active and continuous management the data through the whole lifecycle of it.And utilizing data curation techniques,data could be used to the maximum extent,and the speed of its elimination can be effectively slowed down.The process and key techniques of data curation aroundits goals,solutions and applications were described.For the crucial techniques,existing solutions were analyzed and introduced.In addition,the applications of data curation in the various domains were also introduced and compared.Finally,the development prospect and future challenges were expounded.
摘要:In order to build an efficient,scalable and easy-to-manage data integration and application platform,using data space structure,electronic medical records were integrated in original data space,anonymous data space,and model data space according to data sensitivity,and anonymous data were used for data mining and secondary analysis.Different storage,security protection and data access mechanisms were designed and implemented for each data space.The platform has been applied in the analysis of national health care performance and the evaluation of medical capabilities,quality,and efficiency of affiliated hospitals of Peking University.
关键词:electronic medical records;data platform;data space;data quality;data desensitization
摘要:In order to achieve more robust and accurate outpatient volume prediction,a hybrid prediction model based on SARIMALSTM was constructed.SARIMA model was used to build a single index model of outpatient volume to extract the cycle,trend and other information contained in outpatient volume index.Then multiple related indexes,including holiday days,legal working days,average maximum temperature,were used as input of a many-to-one LSTM model,in order to further learn the residual of SARIMA model and extract the nonlinear relationship between residual and multiple variables.The empirical results show that the SARIMA-LSTM hybrid model constructed in this paper has higher prediction accuracy than the five mainstream prediction methods,so it has good practical application value.
关键词:seasonal auto-regressive integrated moving average model;long short term memory;outpatient forecast;residual
摘要:With the development and application of container cloud,the demand for high concurrency,high availability,high flexibility,and high flexibility of resources is becoming more and more intense.After investigating the current research status of container cloud resource prediction,a container cloud resource prediction model which using an adaptive probability multiselection strategy genetic algorithm (APMSSGA) to optimize the long short term memory network (LSTM) was proposed.The experimental results show that compared with the simple genetic algorithm (SGA),APMSSGA is more efficient in LSTM parameter optimal solution combination search,and the APMSSGA-LSTM model has higher prediction accuracy.
关键词:container cloud;resource prediction;long short term memory network;genetic algorithm
摘要:Water environment numerical models are effective tools for the simulation,analysis and prediction of the processes of pollutant transport and transformation in water.The development of high-performance batch computation of water environment models has long been a hot topic.The distributed cluster computing mode based on big data technology is a promising approach for massive data management and batch computation,which provides a viable solution to large-scale water environment simulations.The adaptability of water environmental models under the framework of big data technology was explored,and a distributed cluster computing mode for water environment simulations was proposed.Moreover,the feasibility of adapting Delft3D model for cluster computing under Hadoop MapReduce environment was verified with real examples.
摘要:Internet lending fraud prediction method based on association graph limits the mining efficiency and depth of features,as well as the reusability and expressibility of features.To solve this problem,the network embedding technology was introduced,and the structure and semantic information in the network by using the vector was expressed.The network update method based on periodic time window and decision batch method were proposed to improve the performance of network embedding in the two business requirements of accuracy and real-time.The experiment shows that the network embedding technology can automatically and effectively learn the implicit relationship and characteristics of the network.By combining the traditional method and the network embedding method,the fraud prediction performance has been significantly improved.