一种软硬件结合的大数据访存踪迹收集分析工具集

李作骏; 潘海洋; 陈明宇; 包云岗

doi:10.11959/j.issn.2096-0271.2019031

您当前的位置：

首页 >

文章列表页 >

一种软硬件结合的大数据访存踪迹收集分析工具集

专题：大数据的系统结构 | 更新时间：2024-06-03

- 一种软硬件结合的大数据访存踪迹收集分析工具集
- A hybrid memory trace collection and analysis toolkit for big data applications
- 大数据 2019年5卷第4期页码：41-49
- 作者机构：
  
  1. 中国科学院大学，北京 100049
  2. 中国科学院计算技术研究所先进计算机系统研究中心，北京 100190
- 作者简介：
  
  [ "李作骏（1990- ），男，中国科学院大学博士生，主要研究方向为计算机系统结构。" ]
  [ "潘海洋（1990- ），男，中国科学院大学博士生，主要研究方向为计算机系统结构。" ]
  [ "陈明宇（1972- ），男，博士，中国科学院计算技术研究所研究员，主要研究方向为高性能计算机体系结构、操作系统及并行算法优化。" ]
  [ "包云岗（1980- ），男，博士，中国科学院计算技术研究所研究员，计算机体系结构国家重点实验室教授，先进计算机系统研究中心主任，中国科学院大学岗位教授。担任中国计算机学会理事、普及工作委员会主任，中国科学院青年创新促进会理事。主要研究方向为计算机系统结构，主持研制多款达到国际先进水平的系统，在国际会议期刊发表了40余篇论文，相关技术在华为技术有限公司、阿里巴巴集团、英特尔公司等国内外企业应用，多次受邀担任ASPLOS、ISCA、MICRO、SC等国际会议的程序委员会委员。" ]
- 基金信息：
  
  “十三五”国家重点研发计划基金资助项目;The National Key Research and Development Program During the Thirteenth Five-Year Plan Period(2017YFB1001602)
- DOI：10.11959/j.issn.2096-0271.2019031
  中图分类号： TP391
- 网络首发：2019-07，
  
  纸质出版：2019-07-15
- 稿件说明：
移动端阅览
李作骏, 潘海洋, 陈明宇, 等. 一种软硬件结合的大数据访存踪迹收集分析工具集[J]. 大数据, 2019,5(4):41-49.

Zuojun LI, Haiyang PAN, Mingyu CHEN, et al. A hybrid memory trace collection and analysis toolkit for big data applications[J]. Big Data Research, 2019, 5(4): 41-49.
李作骏, 潘海洋, 陈明宇, 等. 一种软硬件结合的大数据访存踪迹收集分析工具集[J]. 大数据, 2019,5(4):41-49. DOI： 10.11959/j.issn.2096-0271.2019031.

Zuojun LI, Haiyang PAN, Mingyu CHEN, et al. A hybrid memory trace collection and analysis toolkit for big data applications[J]. Big Data Research, 2019, 5(4): 41-49. DOI： 10.11959/j.issn.2096-0271.2019031.

摘要

以Spark为代表的内存计算框架的兴起、新型非易失性内存研究的逐步深入以及数据安全形势的日益严峻，使得现有的访存行为分析工具无法满足对大数据应用访存行为进行分析的需求。提出了一种软硬件结合的大数据访存踪迹收集分析工具集，在由硬件收集的基本访存踪迹的基础上，结合软件信息同步及离线标注的方式，可以高速、准确、无失真地获取具备丰富语义信息的访存行为信息，且为大数据访存的实时安全监控提供了一种实现方式。最后，通过实验对一组真实的大数据应用进行了访存踪迹采集和分析。

Abstract

The rise of in-memory computing framework represented by Spark

the gradual deepening of new non-volatile memory research and the increasingly severe data security situation made the existing memory behavior analysis tools unable to meet the demand for big data applications.A software-hardware hybrid memory trace collection and analysis toolkit for big data applications was proposed.Based on the basic memory trace collected by hardware

the memory behavior information with rich semantic information can be obtained quickly

accurately and undistorted by combining software information synchronization and offline annotation.It also provides an implementation method for real-time security monitoring of large data access.Finally

a group of real big data applications were analyzed by this toolkit.

关键词

Keywords

references

ZAHARIA M , CHOWDHURY M , FRANKLIN M J , et al . Spark:cluster computing with working sets [C ] // The 2nd USENIX Conference on Hot Topics in Cloud Computing,June 22-25,2010,Boston,USA . Berkeley:USENIX Association , 2010 :10.

BEZ R , PIROVANO A . Non-volatile memory technologies:emerging concepts and new materials [J ] . Materials Science in Semiconductor Processing , 2004 , 7 ( 4-6 ): 349 - 355 .

ROSENFELD P , COOPERBALIS E , JACOB B . DRAMSim2:a cycle accurate memory system simulator [J ] . IEEE Computer Architecture Letters , 2011 , 10 ( 1 ): 16 - 19 .

MAGNUSSON P , CHRISTENSSON M , ESKILSON J , et al . Simics:a full system simulation platform [J ] . Computer , 2002 , 35 ( 2 ): 50 - 58 .

BINKERT N , BECKMANN B , BLACK G , et al . The Gem5 simulator [J ] . ACM Sigarch Computer Architecture News , 2011 , 39 ( 2 ): 1 - 7 .

JALEEL A , COHN R , LUK C K , et al . CMP$im:a pin-based on-the-fly multicore cache simulator [C ] // The 4th Annual Workshop on Modeling,Benchmarking and Simulation,June 22,2008,Beijing,China.[S.l.:s.n . ] , 2008 : 28 - 36 .

PATEL A , AFRAM F , CHEN S F , et al . MARSSx86:a full system simulator for multicore x86 CPUs [C ] // The 48th ACM/EDAC/IEEE Design Automation Conference,June 5-9,2011,New York,USA . Piscataway:IEEE Press , 2011 : 1050 - 1055 .

LUK C K , COHN R , MUTH R , et al . Pin:building customized program analysis tools with dynamic instrumentation [C ] // The 2005 ACM SIGPLAN Conference on Programming Language Design and Implementation,June 12-15,2005,Chicago,USA . New York:ACM Press , 2005 : 190 - 200 .

NETHERCOTE N , SEWARD J . Valgrind:a framework for heavyweight dynamic binary instrumentation [C ] // The 28th ACM SIGPLAN Conference on Programming Language Design and Implementation,June 10-13,2007,San Diego,USA . New York:ACM Press , 2007 : 89 - 100 .

SRIVASTAVA A , EUSTACE A . ATOM:a system for building customized program analysis tools [C ] // The ACM SIGPLAN 1994 Conference on Programming Language Design and Implementation,June 20-24,1994,Orlando,USA . New York:ACM Press , 1994 : 196 - 205 .

LAURENZANO M , TIKIR M , CARRINGTON L , et al . PEBIL:efficient static binary instrumentation for Linux [C ] // IEEE International Symposium on Performance Analysis of Systems ＆Software,March 28-30,2010,White Plains,USA . Piscataway:IEEE Press , 2010 : 175 - 183 .

REINDERS J . VTune performance analyzer essentials:measurement and tuning techniques for software developers [M ] . Santa Clara : Intel PressPress , 2004 : 1 - 500 .

GREGG B , MAURO J . DTrace:dynamictracing in Oracle Solaris,Mac OS X,and free BSD [M ] . Upper Saddle River : Prentice Hall ProfessionalPress , 2011 : 1 - 1152 .

TAN Z X , WATERMAN A , AVIZIENIS R , et al . RAMP gold:an FPGAbased architecture simulator for multiprocessors [C ] // The 47th Design Automation Conference,June 13-18,2010,Anaheim,USA . New York:ACM Press , 2010 : 463 - 468 .

ROTHMAN J , CHEN C . BEE technology overview [C ] // International Conference on Embedded Computer Systems,July 16-19,2012,Samos,Greece . Piscataway:IEEE Press , 2012 :277.

NANDA A , MAK K K , SUGAVANAM K , et al . MemorIES:a programmable,real-time hardware emulation tool for multiprocessor server design [J ] . ACM SIGPLAN Notices , 2000 , 35 ( 11 ): 37 - 48 .

CHALAINANONT N , NURVITADHI E , MORRISON R , et al . Real-time L3 cache simulations using the programmable hardware-assisted cache emulator (PHA$E) [C ] // IEEE International Conference on Communications,October 27,2003,Austin,USA . Piscataway:IEEE Press , 2003 : 86 - 95 .

HONG J , NURVITADHI E , LU S L . Design,implementation,and verification of active cache emulator (ACE) [C ] // The 2006 ACM/SIGDA 14th International Symposium on Field Programmable Gate Arrays,February 22-24,2006,Monterey,USA . New York:ACM Press , 2006 : 63 - 72 .

YOON H M,PARK G H , LEE K W , et a l . Reconfigurable address collector and flying cache simulator [C ] // The HighPerformance Computing on the Information Superhighway,April 28-May 2,1997,Seoul,Korea . Piscataway:IEEE Press , 1997 :552.

阮元 , 陈明宇 , 包云岗 , 等 . 基于硬件的内存trace工具——MTT的设计与实现 [J ] . 电子学报 , 2008 , 36 ( 8 ): 1519 - 1525 .

RUAN Y , CHEN M Y , BAO Y G , et al . The design and implementation of MIT a hardware-based memory trace tool [J ] . Acta Electronica Sinica , 2008 , 36 ( 8 ): 1519 - 1525 .

朱晏 . 内存行为、功耗监控平台的硬件设计 [D ] . 北京:中国科学院大学 , 2011 .

ZHU Y . Hardware design of memory behavior and power monitoring platform [D ] . Beijing:University of Chinese Academy of Sciences , 2011 .

李作骏 . 访存踪迹收集工具的逻辑设计与实现 [D ] . 北京:中国科学院大学 , 2016 .

LI Z J . Logical design and implementation of memory trace collection tool [D ] . Beijing:University of Chinese Academy of Sciences , 2016 .

李作骏 , 陈明宇 . 一种支持DDR4的软硬件结合的访存踪迹收集分析工具集 [C ] // 全国高性能计算学术年会,2018年10月18-20日,青岛,中国 . 北京 :中国计算机学会 , 2018 : 224 - 231 .

LI Z J , CHEN M Y . Hybrid memory trace collection and analysis toolkit for DDR4 [C ] // National Annual Conference on High Performance Computing,October 18-20,2018,Qingdao,China . Beijing:China Computer Federation , 2018 : 224 - 231 .

TANG D , BAO Y G , HU W W , et al . DMA cache:using on-chip storage to architecturally separate I/O data from CPU data for improving I/O p erformance [C ] // The 16th International Symposium on High-Performance Computer Architecture,January 9-14,2010,Bangalore,India . Piscataway:IEEE Press , 2010 .

CHEN L C , CUI Z H , BAO Y G , et al . A lightweight hybrid hardware/software approach for object-relative memory profiling [C ] // The 2012 IE EE International Symposium on Performance Analysis of Systems ＆ Software,April 1-3,2012,New Brunswick,USA . Piscataway:IEEE Press , 2012 : 46 - 57 .

ZHANG J T , LIU Y H , LI H F , et al . PTAT:an efficient and precise tool for tracing and profiling detailed TLB misses [C ] // ACM Transactions on Embedded Computing Systems (TECS),April 24-25,2017,Santa Rosa,USA . Piscataway:IEEE Press , 2017 .

HUANG Y B , CUI Z H , CHEN L C , et al . HaLock:hardware-assisted lock contention detection in multithreaded applications [C ] // The 21st Int ernational Conference on Parallel Architectures and Compilation Techniques,September 19-23,2012,Minneapolis,USA . New York:ACM Press , 2012 .

BAO Y G , CHEN M Y , RUAN Y , et al . HMTT:a platform independent fullsystem memory trace monitoring system [C ] // The 2008 ACM Internationa l Conference on Measurement and Modeling of Computer Systems,June 2-6,2008,Annapolis,USA . New York:ACM Press , 2008 : 229 - 240 .

CUI Z H , ZHU Y , BAO Y G , et al . A fine-grained component-level power measurement method [C ] // 2011 International Green Computing Conferen ce and Workshops,July 25-28,2011,Orlando,USA . Piscataway:IEEE Press , 2011 .

LU T Y , LIU Y H , PAN H Y , et al . TDV cache:organizing off-chip DRAM cache of NVMM from a fusion perspective [C ] // The 35th IEEE International Conference on Computer Design (ICCD),November 5-8,2017,Boston,USA . Piscataway:IEEE Press , 2017 .

HENNING J . SPEC CPU2006 benchmark descriptions [J ] . ACM SIGARCH Computer Architecture News , 2006 , 34 ( 4 ): 1 - 17 .

BIENIA C , KUMAR S , SINGH J , et al . The PARSEC benchmark suite:characterization and architectural implications [C ] // The 17th International Conference on Parallel Architectures and Compilation Techniques,October 25-29,2008,Toronto,Canada . New York:ACM Press , 2008 .

WANG L , ZHAN J F , LUO C J , et al . BigDataBench:a big data benchmark suite from Internet services [C ] // IEEE20th International Symposium on High Performance Computer Architecture,February 15-19,2014,Orlando,USA . Piscataway:IEEE Press , 2014 .

浏览量

413

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

面向大数据的可扩展正则采样并行排序算法

基于大数据技术的甘肃智慧旅游系统

大数据与计算模型

“东数西算”战略与问题的分析研究

高等教育数字化转型的现状与发展研究