基于BERT阅读理解框架的司法要素抽取方法

黄辉; 秦永彬; 陈艳平; 黄瑞章

doi:10.11959/j.issn.2096-0271.2021057

您当前的位置：

首页 >

文章列表页 >

基于BERT阅读理解框架的司法要素抽取方法

专题：大数据支撑的智能应用 | 更新时间：2024-06-03

- 基于BERT阅读理解框架的司法要素抽取方法
- Legal element extraction method based on BERT reading comprehension framework
- 大数据 2021年7卷第6期页码：19-29
- 作者机构：
  
  1. 贵州大学计算机科学与技术学院，贵州贵阳 550025
  2. 公共大数据国家重点实验室，贵州贵阳 550025
- 作者简介：
  
  [ "黄辉（1994- ），男，贵州大学计算机科学与技术学院硕士生，主要研究方向为自然语言处理、智能问答" ]
  [ "秦永彬（1980- ），男，博士，贵州大学计算机科学与技术学院教授、院长，主要研究方向为大数据处理、云计算、文本挖掘" ]
  [ "陈艳平（1980- ），男，博士，贵州大学计算机科学与技术学院副教授，主要研究方向为人工智能、自然语言处理" ]
  [ "黄瑞章（1979- ），女，博士，贵州大学计算机科学与技术学院副教授，主要研究方向为信息检索、文本挖掘" ]
- 基金信息：
  
  国家自然科学基金资助项目;The National Natural Science Foundation of China(62066008);贵州省科学技术基金重点项目;The Key Projects of Science and Technology of Guizhou Province([2020]1Z055)
- DOI：10.11959/j.issn.2096-0271.2021057
  中图分类号： TP391.1
- 网络首发：2021-11，
  
  纸质出版：2021-11-15
- 稿件说明：
移动端阅览
黄辉, 秦永彬, 陈艳平, 等. 基于BERT阅读理解框架的司法要素抽取方法[J]. 大数据, 2021,7(6):19-29.

Hui HUANG, Yongbin QIN, Yanping CHEN, et al. Legal element extraction method based on BERT reading comprehension framework[J]. Big data research, 2021, 7(6): 19-29.
黄辉, 秦永彬, 陈艳平, 等. 基于BERT阅读理解框架的司法要素抽取方法[J]. 大数据, 2021,7(6):19-29. DOI： 10.11959/j.issn.2096-0271.2021057.

Hui HUANG, Yongbin QIN, Yanping CHEN, et al. Legal element extraction method based on BERT reading comprehension framework[J]. Big data research, 2021, 7(6): 19-29. DOI： 10.11959/j.issn.2096-0271.2021057.

摘要

司法要素抽取是司法智能化辅助应用的重要基础，其目的是判别裁判文书涉及的关键案情要素。以往司法要素抽取通常采用多标签分类方法进行建模，模型主要依赖于裁判文书文本特征，忽略了要素标签的语义信息。同时，由于司法数据集存在样本分布不均衡的情况，分类方法会因负例过多而导致模型性能不佳。针对上述问题，提出基于BERT阅读理解框架的司法要素抽取方法。该方法引入标签信息和法律先验知识构造辅助问句，利用BERT机器阅读理解模型建立辅助问句和裁判文书之间的语义联系。同时，在问句中标签所在位置前后增加特殊标识符以增强模型的学习能力。实验结果表明，该方法在CAIL2019要素抽取公开数据集上性能得到显著提升，在婚姻家庭、劳动争议、借款合同3种案由上分别提升F1值2.7%、11.3%、5.6%。

Abstract

Extraction of legal elements is an important basis for judicial intelligent auxiliary applications

and its purpose is to identify the key elements involved in the judgment document.In the past

extracting legal elements usually used multi-label classification methods for modeling.These methods mainly relied on the text features of the judgment document

thereby ignoring the label features.Besides

due to the imbalanced data problem in judicial data sets

the classification method will lead to poor model performance because of too many negative examples.To solve the above problems

a legal element extraction method based on BERT reading comprehension framework was proposed.This method constructed auxiliary questions with label information and legal prior knowledge

and used the machine reading comprehension model based on BERT to establish the semantic associations between question and judgment document.And this method added special tokens before and after the label in the question to enhance the learning ability of the model.Experiments were conducted on the legal element extraction data sets of the CAIL2019.Experiment results show that the performance is improved significantly

and the F1 value has been increased by 2.7%

11.3%

and 5.6% respectively on the data sets of marriage and family case

labor dispute case

and loan contract dispute case.

关键词

Keywords

references

ELISSEEFF A , WESTON J . A kernel method for multi-labelled classification [C ] // Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic . Cambridge:MIT Press , 2001 : 681 - 687 .

GHAMRAWI N , MCCALLUM A . Collective multi-label classification [C ] // Proceedings of the 14th ACM International Conference on Information and Knowledge Management . New York:ACM Press , 2005 : 195 - 200 .

LI C , WANG B , PAVLU V , et al . Conditional bernoulli mixtures for multilabel classification [C ] // Proceedings of the 2016 International Conference on Machine Learning .[S.l.:s.n. ] , 2016 : 2482 - 2491 .

KIM Y . Convolutional neural networks for sentence classification [J ] . arXiv preprint,2014,arXiv:1408.5882 .

CONNEAU A , SCHWENK H , BARRAULT L , et al . Very deep convolutional networks for text classification [J ] . arXiv preprint,2016,arXiv:1606.01781 .

SUN X , MA X H , NI Z W , et al . A new LSTM network model combining TextCNN [C ] // Proceedings of the 2018 International Conference on Neural Information Processing . Heidelberg:Springer , 2018 : 416 - 424 .

LIN J Y , SU Q , YANG P C , et al . Semantic-unit-based dilated convolution for multi-label text classification [C ] // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2018 .

PETERS M E , NEUMANN M , IYYER M , et al . Deep contextualized word representations [J ] . arXiv preprint,2018,arXiv:1802.05365 .

RADFORD A , NARASIMHAN K , SALIMANS T , et al . Improving language understanding by generative pretraining [Z ] . 2018 .

DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [J ] . arXiv preprint,2018,arXiv:1810.04805 .

YANG Z L , DAI Z H , YANG Y M , et al . XLNet:generalized autoregressive pretraining for language understanding [J ] . arXiv preprint,2019,arXiv:1906.08237 .

LUO B F , FENG Y S , XU J B , et al . Learning to predict charges for criminal cases with legal basis [C ] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2017 : 2727 - 2736 .

ZHONG H X , GUO Z P , TU C C , et al . Legal judgment prediction via topological learning [C ] // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2018 : 3540 - 3549 .

陈文哲 , 秦永彬 , 黄瑞章 , 等 . 基于犯罪行为序列的法律条文预测方法 [J ] . 计算机工程与应用 , 2019 , 55 ( 22 ): 245 - 249 , 264 .

CHEN W Z , QIN Y B , HUANG R Z , et al . Legal text prediction method based on criminal behavior sequence [J ] . Computer Engineering and Applications , 2019 , 55 ( 22 ): 245 - 249 , 264 .

秦永彬 , 冯丽 , 陈艳平 , 等 . “智慧法院”数据融合分析与集成应用 [J ] . 大数据 , 2019 , 5 ( 3 ): 35 - 46 .

QIN Y B , FENG L , CHEN Y P , et al . “Intelligent Court” data fusion analysis and integrated application [J ] . Big Data Research , 2019 , 5 ( 3 ): 35 - 46 .

Natural Language Computing Group,Microsoft Research Asia . R-NET:machine reading comprehension with self-matching networks [R ] . 2017 .

SEO M , KEMBHAVI A , FARHADI A , et al . Bidirectional attention flow for machine comprehension [J ] . arXiv preprint,2016,arXiv:1611.01603 .

LEVY O , SEO M , CHOI E , et al . Zeroshot relation extraction via reading comprehension [J ] . arXiv preprint,2017,arXiv:1706.04115 .

MCCANN B , KESKAR N S , XIONG C M , et al . The natural language decathlon:multitask learning as question answering [J ] . arXiv preprint,2018,arXiv:1806.08730 .

LI X Y , YIN F , SUN Z J , et al . Entityrelation extraction as multi-turn question answering [J ] . arXiv preprint,2019,arXiv:1905.05529 .

刘奕洋 , 余正涛 , 高盛祥 , 等 . 基于机器阅读理解的中文命名实体识别方法 [J ] . 模式识别与人工智能 , 2020 , 33 ( 7 ): 653 - 659 .

LIU Y Y , YU Z T , GAO S X , et al . Chinese named entity recognition method based on machine reading comprehension [J ] . Pattern Recognition and Artificial Intelligence , 2020 , 33 ( 7 ): 653 - 659 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceedings of the 31st Conference on Neural Information Processing Systems .[S.l:s.n. ] , 2017 : 5998 - 6008 .

SOCHER R , PERELYGIN A , WU J , et al . Recursive deep models for semantic compositionality over a sentiment treebank [C ] // Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2013 : 1631 - 1642 .

ZHOU P , SHI W , TIAN J , et al . Attentionbased bidirectional long short-term memory networks for relation classification [C ] // Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers) . Stroudsburg:Association for Computational Linguistics , 2016 : 207 - 212 .

LAN Z Z , CHEN M D , GOODMAN S , et al . ALBERT:a lite BERT for self-supervised learning of language representations [J ] . arXiv preprint,2019,arXiv:1909.11942 .

SUN Y , WANG S H , LI Y K , et al . ERNIE:enhanced representation through knowledge integration [J ] . arXiv preprint,2019,arXiv:1904.09223 .

CUI Y M , CHE W X , LIU T , et al . Pretraining with whole word masking for Chinese BERT [J ] . arXiv preprint,2019,arXiv:1906.08101 .

浏览量

630

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于机器阅读理解的论文辅助阅读系统构建

面向自然语言理解的多教师BERT模型蒸馏研究

适用于特殊类型自然语言分类的自适应特征谱神经网络