实体摘要系统的解释性评测

刘庆霞; 李俊宥; 程龚

doi:10.11959/j.issn.2096-0271.2021023

您当前的位置：

首页 >

文章列表页 >

实体摘要系统的解释性评测

专题：基于大数据的知识图谱及其应用 | 更新时间：2024-06-03

- 实体摘要系统的解释性评测
- An interpretive evaluation of entity summarization system
- 大数据 2021年7卷第3期页码：2021023
- 作者机构：
- 作者简介：
  
  [ "刘庆霞（1990- ），女，南京大学计算机软件新技术国家重点实验室博士生，主要研究方向为数据摘要和智能问答。" ]
  [ "李俊宥（1996- ），男，南京大学计算机软件新技术国家重点实验室硕士生，主要研究方向为数据摘要和强化学习。" ]
  [ "程龚（1984- ），男，博士，南京大学计算机软件新技术国家重点实验室副教授，主要研究方向为语义搜索、数据摘要、智能问答。" ]
- 基金信息：
  
  国家重点研发计划资助项目;The National Key Research and Development Program of China(2018YFB1004300);国家自然科学基金资助项目;The National Natural Science Foundation of China(62072224)
- DOI：10.11959/j.issn.2096-0271.2021023
  中图分类号： TP31
- 网络首发：2021-05，
  
  纸质出版：2021-05-15
- 稿件说明：
移动端阅览
刘庆霞, 李俊宥, 程龚. 实体摘要系统的解释性评测[J]. 大数据, 2021,7(3):2021023.

Qingxia LIU, Junyou LI, Gong CHENG. An interpretive evaluation of entity summarization system[J]. Big data research, 2021, 7(3): 2021023.
刘庆霞, 李俊宥, 程龚. 实体摘要系统的解释性评测[J]. 大数据, 2021,7(3):2021023. DOI： 10.11959/j.issn.2096-0271.2021023.

Qingxia LIU, Junyou LI, Gong CHENG. An interpretive evaluation of entity summarization system[J]. Big data research, 2021, 7(3): 2021023. DOI： 10.11959/j.issn.2096-0271.2021023.

摘要

任务是从知识图谱中描述实体的大量三元组中选取最优子集作为摘要。现有实体摘要系统通常以较复杂的方式集成多种摘要技术特征。已开展的评测工作对现有系统进行了总体效果的评测和对比，但未能解释系统所用各摘要特征对最终效果的作用。为此，提出对实体摘要系统开展解释性评测。提出两种新指标：特征效用率和特征显著率，两者分别度量各摘要特征在标准摘要和系统生成摘要中的显示度，两者对比分析的结果在一定程度上可为系统取得的最终效果提供解释。基于3个评测集实现了这种评测新方法，运用6种常见的摘要特征，对9个非监督实体摘要系统和两个有监督实体摘要系统进行了解释性评测，相关代码和数据已开源。

Abstract

The task of entity summarization (ES) is to select an optimum subset from a large set of triples describing an entity in a knowledge graph.ES systems often integrate many and various ES features in a complex way.While state-of-the-art ES systems have been evaluated and compared by recent benchmarking efforts

it was unclear whether and how much each constituent ES feature had contributed to the performance of an ES system.An interpretive evaluation of ES systems was proposed.Two novel evaluation metrics were proposed

feature effectiveness ratio and feature significance ratio

to characterize how much ground-truth summaries and machine-generated summaries exhibit each ES feature.Their comparison would help to interpret the performance of an ES system.Based on three benchmarks

metrics with six popular ES features were implemented

and an interpretive evaluation of nine unsupervised ES systems and two supervised ES systems were presented.The code and data are open source.

关键词

Keywords

references

NOY N , GAO Y Q , JAIN A , et al . Industry-scale knowledge graphs:lessons and challenges [J ] . Communications of the ACM , 2019 , 62 ( 8 ): 36 - 43 .

GUNARATNA K . Semantics-based summarization of entities in knowledge graphs [D ] . Dayton:Wright State University , 2017 .

THALHAMMER A . Linked data entity summarization [D ] . Karlsruhe:Karlsruher Institut für Technologie , 2017 .

GUNARATNA K , THIRUNARAYAN K , SHETH A , et al . Gleaning types for literals in RDF triples with application to entity summarization [C ] // Proceedings of the ICWE 2016 . Cham:Springer , 2016 : 85 - 100 .

LIU Q X , CHENG G , GUNARATNA K , et al . Entity summarization:state of the art and future challenges [J ] . arXiv preprint , 2019 ,arXiv:1910.08252v1.

LIU Q X , CHENG G , GUNARATNA K , et al . ESBM:an entity summarization BenchMark [C ] // Proceedings of the ICWE 2020 . Cham:Springer , 2020 : 548 - 564 .

CHENG G , TRAN T , QU Y Z . RELIN:relatedness and informativeness-based centrality for entity summarization [C ] // Proceedings of the ICWE 2011 . Cham:Springer , 2011 : 114 - 129 .

SYDOW M , PIKUŁA M , SCHENKEL R . The notion of diversity in graphical entity summarisation on semantic knowledge graphs [J ] . Journal of Intelligent Information Systems , 2013 , 41 ( 2 ): 109 - 149 .

GUNARATNA K , THIRUNARAYAN K , SHETH A . FACES:diversity-aware entity summarization using incremental hierarchical conceptual clustering [C ] // Proceedings of the 29th AAAI Conference on Artificial Intelligence . Palo Alto:AAAI Press , 2015 : 116 - 122 .

XU D Y , ZHENG L , QU Y Z . CD at ENSEC 2016:Generating characteristic and diverse entity summaries [C ] // Proceedings of the ESWC 2016 .[S.l.:s.n. ] , 2016 .

THALHAMMER A , LASIERRA N , RETTINGER A . LinkSUM:using link analysis to summarize entity data [C ] // Proceedings of the ICWE 2016 . Cham:Springer , 2016 : 244 - 261 .

KROLL H , NAGEL D , BALKE W T . BAFREC:balancing frequency and rarity for entity characterization in linked open data [C ] // Proceedings of the 1st International Workshop on Entity Retrieval .[S.l.:s.n. ] , 2018 .

POURIYEH S , ALLAHYARI M , KOCHUT K , et al . ES-LDA:entity summarization using knowledge-based topic modeling [C ] // Proceedings of the 8th International Joint Conference on Natural Language Processing .[S.l.:s.n. ] , 2017 : 316 - 325 .

POURIYEH S , ALLAHYARI M , KOCHUT K J , et al . Combining word embedding and knowledge-based topic modeling for entity summarization [C ] // Proceedings of the 12th International Conference on Semantic Computing . Piscataway:IEEE Press , 2018 : 252 - 255 .

WEI D J , GAO S Y , LIU Y X , et al . MPSUM:entity summarization with predicate-based matching [J ] . arXiv preprint , 2020 ,arXiv:2005.11992.

KIM E , CHOI K S . Entity summarization based on formal concept analysis [C ] // Proceedings of the EYRE 2018 .[S.l.:s.n. ] , 2018 .

WEI D J , LIU Y X , ZHU F Q , et al . ESA:entity summarization with attention [J ] . arXiv preprint , 2019 ,arXiv:1905.10625.

LIU Q X , CHENG G , QU Y Z . DeepLENS:deep learning for entity summarization [J ] . arXiv preprint , 2020 arXiv:2003.03736.

THALHAMMER A , KNUTH M , SACK H . Evaluating entity summarization using a game-based ground truth [C ] // Proceedings of the ISWC 2012 . Heidelberg:Springer , 2012 : 350 - 361 .

LANGER P , SCHULZE P , GEORGE S , et al . Assigning global relevance scores to DBpedia facts [C ] // Proceedings of the 2014 IEEE 30th International Conference on Data Engineering Workshops . Piscataway:IEEE Press , 2014 : 248 - 253 .

WAITELONIS J , SACK H , BOBIC T . FRanCo-a ground truth corpus for fact ranking evaluation [C ] // Proceedings of the 1st International Workshop on Summarizing and Presenting Entities and Ontologies .[S.l.:s.n. ] , 2015 .

RDF Working Group . W3C:resource description framework (RDF) [R ] . 2014 .

STOILOS G , STAMOU G , KOLLIAS S . A string metric for ontology alignment [C ] // Proceedings of the ISWC 2005 . Heidelberg:Springer , 2005 : 624 - 637 .

浏览量

785

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于知识增强大语言模型的历史学科试题生成系统

知识增强策略引导的交互式强化推荐系统

空天大数据在新型智慧城市建设中的关键技术与应用探索

知识图谱多跳问答推理研究进展、挑战与展望

事件图谱的构建、推理与应用