Legal element extraction method based on BERT reading comprehension framework

Hui HUANG; Yongbin QIN; Yanping CHEN; Ruizhang HUANG

doi:10.11959/j.issn.2096-0271.2021057

您当前的位置：

首页 >

文章列表页 >

Legal element extraction method based on BERT reading comprehension framework

TOPIC: BIG DATA-ENABLED INTELLIGENT APPLICATIONS | 更新时间：2024-06-03

- Legal element extraction method based on BERT reading comprehension framework
- Big data research Vol. 7, Issue 6, Pages: 19-29(2021)
- 作者机构：
  
  1. 贵州大学计算机科学与技术学院，贵州贵阳 550025
  2. 公共大数据国家重点实验室，贵州贵阳 550025
- 作者简介：
- 基金信息：
- DOI：10.11959/j.issn.2096-0271.2021057
  CLC： TP391.1
- Online First：2021-11，
  
  Published：15 November 2021
- 稿件说明：
移动端阅览
Hui HUANG, Yongbin QIN, Yanping CHEN, et al. Legal element extraction method based on BERT reading comprehension framework[J]. Big data research, 2021, 7(6): 19-29.
DOI：

Hui HUANG, Yongbin QIN, Yanping CHEN, et al. Legal element extraction method based on BERT reading comprehension framework[J]. Big data research, 2021, 7(6): 19-29. DOI： 10.11959/j.issn.2096-0271.2021057.

摘要

司法要素抽取是司法智能化辅助应用的重要基础，其目的是判别裁判文书涉及的关键案情要素。以往司法要素抽取通常采用多标签分类方法进行建模，模型主要依赖于裁判文书文本特征，忽略了要素标签的语义信息。同时，由于司法数据集存在样本分布不均衡的情况，分类方法会因负例过多而导致模型性能不佳。针对上述问题，提出基于BERT阅读理解框架的司法要素抽取方法。该方法引入标签信息和法律先验知识构造辅助问句，利用BERT机器阅读理解模型建立辅助问句和裁判文书之间的语义联系。同时，在问句中标签所在位置前后增加特殊标识符以增强模型的学习能力。实验结果表明，该方法在CAIL2019要素抽取公开数据集上性能得到显著提升，在婚姻家庭、劳动争议、借款合同3种案由上分别提升F1值2.7%、11.3%、5.6%。

Abstract

Extraction of legal elements is an important basis for judicial intelligent auxiliary applications

and its purpose is to identify the key elements involved in the judgment document.In the past

extracting legal elements usually used multi-label classification methods for modeling.These methods mainly relied on the text features of the judgment document

thereby ignoring the label features.Besides

due to the imbalanced data problem in judicial data sets

the classification method will lead to poor model performance because of too many negative examples.To solve the above problems

a legal element extraction method based on BERT reading comprehension framework was proposed.This method constructed auxiliary questions with label information and legal prior knowledge

and used the machine reading comprehension model based on BERT to establish the semantic associations between question and judgment document.And this method added special tokens before and after the label in the question to enhance the learning ability of the model.Experiments were conducted on the legal element extraction data sets of the CAIL2019.Experiment results show that the performance is improved significantly

and the F1 value has been increased by 2.7%

11.3%

and 5.6% respectively on the data sets of marriage and family case

labor dispute case

and loan contract dispute case.

关键词

Keywords

references

ELISSEEFF A , WESTON J . A kernel method for multi-labelled classification [C ] // Proceedings of the 14th International Conference on Neural Information Processing Systems:Natural and Synthetic . Cambridge:MIT Press , 2001 : 681 - 687 .

GHAMRAWI N , MCCALLUM A . Collective multi-label classification [C ] // Proceedings of the 14th ACM International Conference on Information and Knowledge Management . New York:ACM Press , 2005 : 195 - 200 .

LI C , WANG B , PAVLU V , et al . Conditional bernoulli mixtures for multilabel classification [C ] // Proceedings of the 2016 International Conference on Machine Learning .[S.l.:s.n. ] , 2016 : 2482 - 2491 .

KIM Y . Convolutional neural networks for sentence classification [J ] . arXiv preprint,2014,arXiv:1408.5882 .

CONNEAU A , SCHWENK H , BARRAULT L , et al . Very deep convolutional networks for text classification [J ] . arXiv preprint,2016,arXiv:1606.01781 .

SUN X , MA X H , NI Z W , et al . A new LSTM network model combining TextCNN [C ] // Proceedings of the 2018 International Conference on Neural Information Processing . Heidelberg:Springer , 2018 : 416 - 424 .

LIN J Y , SU Q , YANG P C , et al . Semantic-unit-based dilated convolution for multi-label text classification [C ] // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2018 .

PETERS M E , NEUMANN M , IYYER M , et al . Deep contextualized word representations [J ] . arXiv preprint,2018,arXiv:1802.05365 .

RADFORD A , NARASIMHAN K , SALIMANS T , et al . Improving language understanding by generative pretraining [Z ] . 2018 .

DEVLIN J , CHANG M W , LEE K , et al . BERT:pre-training of deep bidirectional transformers for language understanding [J ] . arXiv preprint,2018,arXiv:1810.04805 .

YANG Z L , DAI Z H , YANG Y M , et al . XLNet:generalized autoregressive pretraining for language understanding [J ] . arXiv preprint,2019,arXiv:1906.08237 .

LUO B F , FENG Y S , XU J B , et al . Learning to predict charges for criminal cases with legal basis [C ] // Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2017 : 2727 - 2736 .

ZHONG H X , GUO Z P , TU C C , et al . Legal judgment prediction via topological learning [C ] // Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2018 : 3540 - 3549 .

陈文哲 , 秦永彬 , 黄瑞章 , 等 . 基于犯罪行为序列的法律条文预测方法 [J ] . 计算机工程与应用 , 2019 , 55 ( 22 ): 245 - 249 , 264 .

CHEN W Z , QIN Y B , HUANG R Z , et al . Legal text prediction method based on criminal behavior sequence [J ] . Computer Engineering and Applications , 2019 , 55 ( 22 ): 245 - 249 , 264 .

秦永彬 , 冯丽 , 陈艳平 , 等 . “智慧法院”数据融合分析与集成应用 [J ] . 大数据 , 2019 , 5 ( 3 ): 35 - 46 .

QIN Y B , FENG L , CHEN Y P , et al . “Intelligent Court” data fusion analysis and integrated application [J ] . Big Data Research , 2019 , 5 ( 3 ): 35 - 46 .

Natural Language Computing Group,Microsoft Research Asia . R-NET:machine reading comprehension with self-matching networks [R ] . 2017 .

SEO M , KEMBHAVI A , FARHADI A , et al . Bidirectional attention flow for machine comprehension [J ] . arXiv preprint,2016,arXiv:1611.01603 .

LEVY O , SEO M , CHOI E , et al . Zeroshot relation extraction via reading comprehension [J ] . arXiv preprint,2017,arXiv:1706.04115 .

MCCANN B , KESKAR N S , XIONG C M , et al . The natural language decathlon:multitask learning as question answering [J ] . arXiv preprint,2018,arXiv:1806.08730 .

LI X Y , YIN F , SUN Z J , et al . Entityrelation extraction as multi-turn question answering [J ] . arXiv preprint,2019,arXiv:1905.05529 .

刘奕洋 , 余正涛 , 高盛祥 , 等 . 基于机器阅读理解的中文命名实体识别方法 [J ] . 模式识别与人工智能 , 2020 , 33 ( 7 ): 653 - 659 .

LIU Y Y , YU Z T , GAO S X , et al . Chinese named entity recognition method based on machine reading comprehension [J ] . Pattern Recognition and Artificial Intelligence , 2020 , 33 ( 7 ): 653 - 659 .

VASWANI A , SHAZEER N , PARMAR N , et al . Attention is all you need [C ] // Proceedings of the 31st Conference on Neural Information Processing Systems .[S.l:s.n. ] , 2017 : 5998 - 6008 .

SOCHER R , PERELYGIN A , WU J , et al . Recursive deep models for semantic compositionality over a sentiment treebank [C ] // Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing . Stroudsburg:Association for Computational Linguistics , 2013 : 1631 - 1642 .

ZHOU P , SHI W , TIAN J , et al . Attentionbased bidirectional long short-term memory networks for relation classification [C ] // Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2:Short Papers) . Stroudsburg:Association for Computational Linguistics , 2016 : 207 - 212 .

LAN Z Z , CHEN M D , GOODMAN S , et al . ALBERT:a lite BERT for self-supervised learning of language representations [J ] . arXiv preprint,2019,arXiv:1909.11942 .

SUN Y , WANG S H , LI Y K , et al . ERNIE:enhanced representation through knowledge integration [J ] . arXiv preprint,2019,arXiv:1904.09223 .

CUI Y M , CHE W X , LIU T , et al . Pretraining with whole word masking for Chinese BERT [J ] . arXiv preprint,2019,arXiv:1906.08101 .

Views

630

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Construction of a paper-assistant reading system based on machine reading comprehension

Multi-teacher distillation BERT model in NLU tasks

Adaptive feature spectrum neural networks for special types of natural language classification

Related Author

Rongxin MI

Wenwen YAO

Hongkun RUAN

Jialai SHI

Weibin GUO

Yifeng WANG

Liru SUN

Liangle CUI

Related Institution

National Computer Network Emergency Response Technical Team/Coordination Center of China

Faculty of Computer Science, Beijing University of Posts and Telecommunications

Postal code：100079
Tel：（010）53879208、53859533 Email：bdr@bjxintong.com.cn
Technical support is provided by Beijing Founder electronics co., LTD 京ICP备09064830号-19 京公网安备11010802024621
It is recommended to read the content of this site in Chrome&IE9+. Please switch to extreme mode in browser 360.
Cookies We use cookies to help provide and enhance our service and tailor content. By continuing, you agree to the use of cookies.
Total Visits：273162 Visits Today：68

⁰