摘要:Big data has become a new technology, which has drawn much attention of media and public. Widely applications of big data indicated that the information age will enter into a new stage. However, the understanding of big data is a process of deepening. The big data from the height of “new information age stage”, data culture and epistemology was expounded. Then how to correctly understand the value and benefit of big data through the explanation of driving effect and wisdom in cyberspace was discussed. The challenges for the research and application of big data technology from the angle of the complexity were analyzed. Finally, some views on avoiding the pitfalls when developing big data technologies were proposed.
关键词:big data;epistemology;wisdom in cyberspace;complexity
摘要:Big data computing is a necessary way to acquire the “great value” behind the big data, and a computing system is an effective tool for big data computing. Big data computing from a system perspective was reviewed. Based on the fact that big data has the macro characteristics of huge volume, growing fast, complex structure, and quality disparity, the typical features of big data computing by analyzing batch computing, stream computing, and graph computing respectively, were discussed. These features may bring technical challenges to the design and implementation of big data computing system. The related works for overcoming these challenges were further categoried. In the end, some prospective research directions of big data computing from the system perspective were listed.
关键词:big data computing;batch computing;stream computing;graph computing;system instance
摘要:Both big data and high performance computing (HPC) are based on the computer technologies. The main methodology of HPC is simulation, which is called the third paradigm of scientific discovery. Big data explore data for correlations even without much knowledge on the object of study, which is called the fourth paradigm of scientific discovery. Big data and HPC with several aspects were compared, such as the research paradigm, main application domain and underlying hardware/software systems.
关键词:big data;high performance computing;parallel and distributed computing;correlation
摘要:To achieve efficient big data machine learning, we need to construct a unified big data machine learning system to support both machine learning algorithm design and big data processing. Designing an efficient, scalable and easy-to-use big data machine learning system still faces a number of challenges. Recently, the upsurge of big data technology has promoted rapid development of big data machine learning, making big data machine learning system to become a research hotspot. The basic concepts, research issues, technical characteristics, categories, and typical systems for big data machine learning system, were reviewed. Then a unified and cross-platform big data machine learning system, Octopus, was presented.
关键词:big data;machine learning;distributed and parallel computing;big data processing platform
摘要:OLAP (online analytical processing) is a key technology of business intelligence based on relational data. In big data era, people want to achieve high performance OLAP using a large cluster of ordinary nodes. However, the performance of such systems is a big challenge. Recently, many SQL on Hadoop systems have been proposed to address this challenge. We have seen a significant performance improvement of such systems. A survey of technology development of OLAP technologies was first provided. Then, a study of the performance of three representatives SQL on Hadoop systems was focused on. Based on the results, it is expected that such systems will play an very important role in the market of low cost OLAP analysis.
关键词:big data;online analytical processing;SQL analysis;SQL on Hadoop
摘要:Fusion and analysis of urban big data is the key to achieving city intelligence. Specifically, to solve for the complex urban problems, it is imperative to build an immersive and interactive visual analytics environment where humans can make sense of out of the otherwise disconnected and abstract data. The state of the art and challenges in urban big data were summarized, and then a work flow for urban data analysis and the trend in the future intelligent city were presented.
摘要:Generally, big data is regarded as a term about data sets so large or complex that conventional data technologies cannot handle. This statement of big data leads to confusion: none of big data has been handled by existing data technologies;or none of current successful data applications can be called as big data. Therefore, what is the best way to define big data becomes a problem. Data, technology, and application were regarded as three associated key factors of big data by analyzing the state-of-the-art of big data. A comprehensive definition on big data was defined as the umbrella of big data set, big data technology, and big data application. Here, big data set means all data that can be acquired and were related to one decision-making application instead of all data in an area or an enterprise. In addition, the issues in big data applications and the main challenges in big data technologies were discussed. Finally, the future directions of big data research were presented including data science and the technologies of big data reservation and development.
摘要:Database benchmarking has stimulated the development of data management systems and technologies. In big data environments, benchmarking should be revisited. Therefore, research on benchmarks for big data management systems is a key problem for big data research and applications. Benchmark design can be achieved from three different perspectives, i.e. data, workload, and performance measurements. After the brief introduction to these three elements and the progress of big data management system research, the requirements and challenges to benchmarking big data management systems were analyzed. Through the introduction to a benchmark for analytical queries over social media data, named as BSMA, the issues of design and implementation of a benchmark for big data management systems were discussed.
关键词:big data management system;benchmark;data generation;workload generation;performance measurement
摘要:Big data and the related applications which derived from the internet originally, are now expanding to other industries, and becoming the key driving force of their innovation and transition. The evolvement of the search engine driven by big data technologies was described, based on Baidu’s innovations and practices in the big data area over the years. Baidu big data engine and its explorations in other industries were introduced. Finally, a vision was discussed that big data and artificial intelligence will be prospected in the future information communication technology.
摘要:Big data is a kind of critical development resources. The ability of control and analysis on big data become the foundation of a country’s competitiveness in future. Although China's big data development has a good start, there are a couple of challenges ahead, e.g., the open government and public data are insufficient, big data application is not widespread, core technologies R&D still need to be strengthened, privacy and information security are facing new risks. Currently, developing a national big data strategy is approaching a common agreement. While how to develop the strategy and what should to be considered are hot topics among the industry and government experts. Therefore, some considerations were proposed based on both other countries practice and domestic research.
摘要:Data application is the key element of data technology (DT) era, the difference between DT and information technology (IT) is whether data is the key product element. IT suppose to solve the “process” problem to make business more effective, but DT suppose to solve the “cognitive” and “collaborative” intelligence to make business to renewable and creative. The catalog of data application product was given. A general architecture of data application platform and organization was proposed. The key and difficult point of data sharing and exchange were analyzed.
关键词:big data application;data sharing and exchange;information technology;data technology