摘要:With the development of ocean observation technology and numerical simulation technology, larger scale and higher resolution ocean data can be obtained, which brings opportunities for the analysis of complex ocean environmental elements and structures, and also brings great challenges to traditional analysis methods.For this reason, the method of big data visual analysis was introduced and its application value in the analysis of multivariate ocean spatiotemporal data, the spatiotemporal characteristics and evolution analysis of important ocean structures was explored.Some visual analysis systems were developed and the basic framework of visual analysis of ocean data through case studies of data analysis of some sea areas around the world and China was summarized, showing that visual analysis is a promising technology for ocean complex data analysis in the era of big data.
关键词:visual analysis;multiple ocean temporal and spatial data;ocean structure;visual analysis application
摘要:Visual analysis for soccer match data can help soccer data analysts gain hidden insights in matches clearly and intuitively, which has attracted attention of a number of researchers.Firstly, current works on visual analysis for soccer match data were surveyed.According to data type, the works can be divided as statistic-based analysis, event-based analysis, and trajectory-based analysis.Secondly, the current facing challenges of visual analysis for soccer match data were discussed further.Finally, directions for future development were prospected.
关键词:soccer match analysis;data analysis;visual analysis
摘要:With the popularity of mobile phones, large-scale communication data has brought people unprecedented opportunities to observe the urban microstructure and dynamics.However the complexity of such large-scale high-dimensional heterogeneous spatio-temporal data has brought challenges to effective data interpretation.Accordingly, visualization, as an important means of big data analysis, is being applied to this field more and more.The recent research work on urban visual analysis based on communication data was reviewed.Firstly, the major sources, characteristics and common data processing methods of the mobile phone data were summarized.Secondly, the corresponding visualization methods from three aspects were elaborated, namely, the internal objects of mobile communication data such as human, communication equipment and urban space, and the frequent urban visual analysis task and features based on communication data were summarized.Finally, the prospect of urban visual analysis based on communication data was presented.
摘要:With the improvement of detection technology and the wide application of Internet technology, the scale and types of food safety data continue to increase, which poses great challenges to data analysis technology.Visual analysis, which has emerged in recent years, can help domain experts gain a deeper understanding of the data and insight into the hidden patterns in the data by providing a graphical interactive interface.This in turn can improve the detection, analysis, early warning and traceability of food safety risks, providing new tools for food safety monitoring and surveillance.Firstly, the main sources, characteristics and analysis tasks of food safety big data were analyzed.Then, a classification method for visual associations analysis techniques was proposed, and the visual associations analysis methods for food safety big data in the past 10 years were described from four aspects: attribute correlation, entity associations, comparative analysis and spatio-temporal analysis.Finally, the problems and challenges in this field were presented.
摘要:As a special form of culture, Chinese Baijiu culture has its unique position in traditional Chinese culture.Applying visualization technology to the Chinese Baijiu culture plays an important role in spreading and popularizing the rich connotation of Chinese Baijiu culture, enhancing national confidence and promoting international cultural exchange.The application of Chinese Baijiu culture visualization was summarized, then, the basic process of visual analysis using big data in Chinese Baijiu culture was described, after that, the research situation of Chinese Baijiu cultural visualization was analyzed from three dimensions: literature, archaeology and history.And then, application scenarios of several visualization methods in the field of cultural visualization were summarized, including text data visualization, network data visualization, space-time data visualization, multidimensional data visualization and so on.Finally, future hot research issues on Chinese Baijiu culture visualization were discussed.
摘要:High-performance computing (HPC) environment is the core information infrastructure supporting national scientific and technological innovation, economic development and national defense construction.High-performance computing powers around the world have been building wide-area HPC environments based on multi-supercomputing center resources.However, in the high-performance computing environment, there are many kinds of resources and wide geographical distribution, which cannot effectively exert the aggregation effect of resources, and it is difficult to meet the requirements of large-scale applications for unified management and efficient access to wide-area distributed data.To this end, a complete set of technologies were proposed, which could be used to build wide-area global virtual data space, including virtual data space model, cross-domain virtual data space constructing, efficiently migrating data in a wide-area environment, co-scheduling of storage resources and computing job and cross-domain high concurrency data aggregation processing, etc.Based on the above, a virtual data space system has been developed for the national high-performance computing environment (NHPCE), which can effectively support the unified and efficient access to the wide area distributed heterogeneous storage resources, and the distributed data in the wide-area environment can be shared and cooperative processed in a cross-domain manner.At present, the system was experimental deployed in NHPCE and three typical large-scale applications, such as molecular docking, genome-wide association study and weather forecasting model, have been verified.The verification results show that the developed technology and software system can effectively aggregate the wide area distributed storage resources and meet the data space requirements of large-scale applications.
关键词:high-performance computing environment;large-scale computing problem;virtual data space;wide-area distributed storage;unified namespace
摘要:The wide-area high-performance computing environment is the core information infrastructure to support technology innovation, economic development, and national defense.However, heterogeneous storage resources are geographically distributed in wide-area high-performance computing environments, resulting in the barriers between applications and data.The requirements of unified data management and efficient data access cannot be met.A method of establishing virtual data space and a data access optimization method was presented, and a global virtual data space (GVDS) for wide-area high-performance computing environments to satisfy the requirements was implemented.GVDS aggregates geographically distributed and heterogeneous storage resources, creating a unified virtual data space to provide unified and efficient data access.Sharing and collaborative processing of geographically distributed data were achieved in widearea environments.The experimental results indicate that compared with the state-of-the-art wide-area storage system in the field of high-performance computing, such as OneData and GFFS, GVDS has similar functions and improves the read bandwidth significantly.
关键词:global virtual data space;wide-area high-performance computing environment;efficient data access;heterogeneous storage resource
摘要:To address the problems of write amplification, long I/O path, and high access latency in distributed storage systems, an efficient SSD-based caching approach for distributed storage systems was proposed.This approach adopts read/write bypassing and lazy caching methods to manage the cache system, considers last access time and historical access frequency when performing cache replacement, and adjusts the flushing speed according to the foreground workload.It improves significantly the reading and writing performance of storage systems.
关键词:distributed storage;distributed data caching;read/write bypassing;lazy caching;replacement strategy
摘要:The national high performance computing environment (NHPCE) is characterized by wide area dispersion, heterogeneous system and storage isolation autonomy, which posed great challenges to storage management and data sharing.Firstly, the concept of virtual data space system was described.Secondly, how the virtual data space system effectively reduce the cost of cross domain access through viewing-access, data sharing, computing environment docking , was analyzed.Thirdly, the virtual data space system were deeply integrated with the NHPCE by means of modularization and its functions were added to the NHPCE.Finally, the unified virtual and real space user management framework was used to achieve cross domain unified, transparent and secure storage service and the support of large-scale computing application, which is of great significance to the development of NHPCE.
关键词:virtual data space system;national high performance computing environment;data distribution;user view;environment integration
摘要:In an HPC system where multiple computation nodes of an MPI application simultaneously access files in underlying storage systems, the I/O overhead is affected by the access mode and the properties of external storage devices.Based on the patterns of MPI applications to access files, an optimization for MPI-IO interface for persistent memories was introduced on high-bandwidth, low-latency, byte-addressable, data-persistent memories.By constructing distributed data cache, maintaining persistent metadata and leveraging optimizations on data movements among processes, applications were enabled to efficiently manage and utilize persistent memories with data consistency guaranteed, resulting in tens of times improvement on read/write bandwidth.Further optimizations on parallelism were set for future work.
关键词:non-volatile memory;MPI-IO;distributed data cache