nach oben

Soft Computing

08.02.2024 | Application of soft computing

Joint entity and relation extraction model based on directed-relation GAT oriented to Chinese patent texts

verfasst von: Yushan Zhao, Kuan-Ching Li, Tengke Wang, Shunxiang Zhang

Erschienen in: Soft Computing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

The joint extraction task aims to construct an entity-relation triple comprising two entities and the relation between them. Existing joint models make it difficult to process too many overlapping relations in Chinese patent texts (CPT). This article introduces a joint entity and relation extraction model based on directed-relation graph attention network (DGAT) oriented to CPT to locate this problem. First, word-character tokens are obtained from CPT using BERT as the DGAT model input. Global tokens are expanded using the BiLSTM network to enhance contextual connection from the model input. Second, the DGAT model encodes the global tokens as a fully connected graph whose nodes represent the global tokens and edges denote the relations between global tokens. The edges with directed relation in the fully connected graph are assigned weights by the DGAT model, and other edges are pruned, resulting in a directed-relation-connected graph. Finally, the entity-relation triples are decoded using conditional random fields (CRF) from the directed relation-connected graph. Experimental results show that the proposed model was highly accurate based on the CPT dataset.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Bekoulis G, Deleu J, Demeester T, Develder C (2018) Joint entity recognition and relation extraction as a multi-head selection problem. Expert Syst Appl 114:34–45. https://doi.org/10.1016/j.eswa.2018.07.032CrossRef

Dai D, Xiao X, Lyu Y, Dou S, She Q, Wang H (2019) Joint extraction of entities and overlapping relations using position-attentive sequence labeling. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 6300–6308. https://doi.org/10.1609/aaai.v33i01.33016300

Eberts M, Ulges A (2020) Span-based joint entity and relation extraction with transformer pre-training. In: the 24th European conference on artificial intelligence 2020, pp 2006–2013. https://doi.org/10.3233/FAIA200321

Fu T-J, Li P-H, Ma W-Y (2019) Graphrel: modeling text as relational graphs for joint entity and relation extraction. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 1409–1418. https://doi.org/10.18653/v1/P19-1136

Gao C, Zhang X, Liu H, Yun W, Jiang J (2022) A joint extraction model of entities and relations based on relation decomposition. Int J Mach Learn Cybern 13:1833–1845. https://doi.org/10.1007/s13042-021-01491-6CrossRef

Gardent C, Shimorina A, Narayan S, Perez-Beltrachini L (2017) The WEBNLG challenge: generating text from RDF data. In: Proceedings of the 10th international conference on natural language generation, pp 124–133. https://doi.org/10.18653/v1/W17-3518

Geng Z, Zhang Y, Han Y (2021) Joint entity and relation extraction model based on rich semantics. Neurocomputing 429:132–140. https://doi.org/10.1016/j.neucom.2020.12.037CrossRef

Grohe M (2021) The logic of graph neural networks. In: Proceedings of the 36th annual ACM/IEEE symposium on logic in computer science. https://doi.org/10.1109/LICS52264.2021.9470677

Guo Z, Zhang Y, Lu W (2020) Attention guided graph convolutional networks for relation extraction. In: Proceedings of the 57th annual meeting of the association for computational linguistics, pp 241–251. https://doi.org/10.18653/v1/P19-1024

Guo M, Lu C, Liu Z, Cheng M, Hu S (2023) Visual attention network. Comput Vis Media 9(4):733–752. https://doi.org/10.1007/s41095-023-0364-2CrossRef

Hu Z, Dong Y, Wang K, Chang K-W, Sun Y (2020) Gpt-gnn: Generative pre-training of graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery and data mining, pp 1857–1867. https://doi.org/10.1145/3394486.3403237

Huang J, Shen H, Hou L, Cheng X (2021) SDGNN: learning node representation for signed directed networks. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 196–203. https://doi.org/10.1609/aaai.v35i1.16093

Ji S, Pan S, Cambria E, Marttinen P, Yu PS (2022) A survey on knowledge graphs: representation, acquisition, and applications. IEEE Trans Neural Netw Learn Syst 33(2):494–514. https://doi.org/10.1109/TNNLS.2021.3070843MathSciNetCrossRefPubMed

Lai T, Cheng L, Wang D, Ye H, Zhang W (2022) RMAN: relational multi-head attention neural network for joint extraction of entities and relations. Appl Intell 52(3):3132–3142. https://doi.org/10.1007/s10489-021-02600-2CrossRef

Li Q, Ji H (2014) Incremental joint extraction of entity mentions and relations. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (vol 1: long papers), pp 402–412. https://doi.org/10.3115/v1/p14-1038

Li X, Luo X, Dong C, Yang D, Luan B, He Z (2021) TDEER: an efficient translating decoding schema for joint extraction of entities and relations. In: Proceedings of the 2021 conference on empirical methods in natural language processing, pp 8055–8064. https://doi.org/10.18653/v1/2021.emnlp-main.635

Li J, Sun A, Han J, Li C (2022) A survey on deep learning for named entity recognition. IEEE Trans Knowl Data Eng 34(1):50–70. https://doi.org/10.1109/TKDE.2020.2981314ADSCrossRef

Liu K (2020) A survey on neural relation extraction. Sci China Technol Sci 63(10):1971–1989. https://doi.org/10.1007/s11431-020-1673-6ADSCrossRef

Liu W, Fu X, Zhang Y, Xiao W (2021) Lexicon enhanced chinese sequence labeling using Bert adapter. In: Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (vol 1: long papers), pp 5847–5858. https://doi.org/10.18653/v1/2021.acl-long.454

Liu P, Guo Y, Wang F, Li G (2022) Chinese named entity recognition: the state of the art. Neurocomputing 473:37–53. https://doi.org/10.1016/j.neucom.2021.10.101CrossRef

Miwa M, Bansal M (2016) End-to-end relation extraction using lstms on sequences and tree structures. In: Proceedings of the 54th annual meeting of the association for computational linguistics (vol 1: long papers), pp 1105–1116. https://doi.org/10.18653/v1/P16-1105

Miwa M, Sasaki Y (2014) Modeling joint entity and relation extraction with table representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1858–1869. https://doi.org/10.3115/v1/D14-1200

Nayak T, Ng HT (2020) Effective modeling of encoder–decoder architecture for joint entity and relation extraction. Proc AAAI Conf Artif Intell 34(05):8528–8535. https://doi.org/10.1609/aaai.v34i05.6374CrossRef

Nimmani P, Vodithala S, Polepally V (2021) Neural network based integrated model for information retrieval. In: 2021 5th International conference on intelligent computing and control systems, pp 1286–1289. https://doi.org/10.1109/ICICCS51141.2021.9432241

Niu Z, Zhong G, Yu H (2021) A review on the attention mechanism of deep learning. Neurocomputing 452:48–62. https://doi.org/10.1016/j.neucom.2021.03.091CrossRef

Qiao B, Zou Z, Huang Y, Fang K, Zhu X, Chen Y (2022) A joint model for entity and relation extraction based on BERT. Neural Comput Appl 34(5):3471–3481. https://doi.org/10.1007/s00521-021-05815-zCrossRef

Shang Y-M, Huang H, Mao X (2022) Onerel: Joint entity and relation extraction with one module in one step. In: Proceedings of the AAAI conference on artificial intelligence, vol 36, pp 11285–11293. https://doi.org/10.1609/aaai.v36i10.21379

Shen Y, Fu H, Du Z, Chen X, Burnaev E, Zorin D, Zhou K, Zheng Y (2022) GCN-denoiser: mesh denoising with graph convolutional networks. ACM Trans Graph. https://doi.org/10.1145/3480168CrossRef

Takanobu R, Zhang T, Liu J, Huang M (2019) A hierarchical framework for relation extraction with reinforcement learning. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 7072–7079. https://doi.org/10.1609/aaai.v33i01.33017072

Tian T, Liu Y, Yang X, Lyu Y, Zhang X, Fang B (2020) QSAN: a quantum-probability based signed attention network for explainable false information detection. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp 1445–1454. https://doi.org/10.1145/3340531.3411890

Wan Q, Wei L, Chen X, Liu J (2021) A region-based hypergraph network for joint entity-relation extraction. Knowl Based Syst 228:107298. https://doi.org/10.1016/j.knosys.2021.107298CrossRef

Wang S, Zhang Y, Che W, Liu T (2018) Joint extraction of entities and relations based on a novel graph scheme. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, pp 4461–4467. https://doi.org/10.24963/ijcai.2018/620

Wang X, Huang T, Wang D, Yuan Y, Liu Z, He X, Chua T-S (2021) Learning intents behind interactions with knowledge graph for recommendation. In: Proceedings of the web conference 2021, pp 878–887. https://doi.org/10.1145/3442381.3450133

Wei Z, Su J, Wang Y, Tian Y, Chang Y (2020a) A novel cascade binary tagging framework for relational triple extraction. https://doi.org/10.18653/v1/2020.acl-main.136

Wei Z, Su J, Wang Y, Tian Y, Chang Y (2020b) A novel cascade binary tagging framework for relational triple extraction. In: Proceedings of the 58th annual meeting of the association for computational linguistics, pp 1476–1488. https://doi.org/10.18653/v1/2020.acl-main.136

Ye Y, Ji S (2021) Sparse graph attention networks. IEEE Trans Knowl Data Eng 35:905–916. https://doi.org/10.1109/TKDE.2021.3072345

Ye D, Lin Y, Li P, Sun M (2022) Packed levitated marker for entity and relation extraction. In: Proceedings of the 60th annual meeting of the association for computational linguistics (vol 1: long papers), pp 4904–4917. https://doi.org/10.18653/v1/2022.acl-long.337

Yuan Y, Zhou X, Pan S, Zhu Q, Song Z, Guo L (2020) A relation-specific attention network for joint entity and relation extraction. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, pp 4054–4060. https://doi.org/10.24963/ijcai.2020/561

Zeng X, Zeng D, He S, Liu K, Zhao J (2018) Extracting relational facts by an end-to-end neural model with copy mechanism. In: Proceedings of the 56th annual meeting of the association for computational linguistics (vol 1: long papers), pp 506–514. https://doi.org/10.18653/v1/P18-1047

Zeng X, He S, Zeng D, Liu K, Liu S, Zhao J (2019) Learning the extraction order of multiple relational facts in a sentence with reinforcement learning. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing, pp 367–377. https://doi.org/10.18653/v1/D19-1035

Zhang Y, Qi P, Manning CD (2018) Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 2205–2215. https://doi.org/10.18653/v1/D18-1244

Zhang S, Hu Z, Zhu G, Jin M, Li K-C (2021) Sentiment classification model for Chinese micro-blog comments based on key sentences extraction. Soft Comput 25(1):463–476. https://doi.org/10.1007/s00500-020-05160-8CrossRef

Zhang S, Zhu H, Xu H, Zhu G, Li K-C (2022a) A named entity recognition method towards product reviews based on BiLSTM-attention-CRF. Int J Comput Sci Eng 25(5):479–489. https://doi.org/10.1504/IJCSE.2022.126251

Zhang S, Xu H, Zhu G, Chen X, Li K (2022b) A data processing method based on sequence labeling and syntactic analysis for extracting new sentiment words from product reviews. Soft Comput 26(2):853–866. https://doi.org/10.1007/s00500-021-06228-9

Zhang S, Wu H, Xu X, Zhu G, Hsieh M-Y (2022c) CL-ECPE: contrastive learning with adversarial samples for emotion-cause pair extraction. Connect Sci 34(1):1877–1894. https://doi.org/10.1080/09540091.2022.2082383

Zhang N, Deng S, Ye H, Zhang W, Chen H (2022d) Robust triple extraction with cascade bidirectional capsule network. Expert Syst Appl 187:115806. https://doi.org/10.1016/j.eswa.2021.115806

Zhao K, Xu H, Cheng Y, Li X, Gao K (2021) Representation iterative fusion based on heterogeneous graph neural network for joint entity and relation extraction. Knowl Based Syst 219:106888. https://doi.org/10.1016/j.knosys.2021.106888CrossRef

Zheng S, Hao Y, Lu D, Bao H, Xu J, Hao H, Xu B (2017a) Joint entity and relation extraction based on a hybrid neural network. Neurocomputing 257:59–66. https://doi.org/10.1016/j.neucom.2016.12.075

Zheng S, Wang F, Bao H, Hao Y, Zhou P, Xu B (2017b) Joint extraction of entities and relations based on a novel tagging scheme. In: Proceedings of the 55th annual meeting of the association for computational linguistics, vol 1: long papers, pp 1227–1236. https://doi.org/10.18653/v1/P17-1113

Zhong Z, Chen D (2021) A frustratingly easy approach for entity and relation extraction. In: Proceedings of the 2021 conference of the North American chapter of the association for computational linguistics: human language technologies, pp 50–61. https://doi.org/10.18653/v1/2021.naacl-main.5

Zhou W, Huang K, Ma T, Huang J (2021) Document-level relation extraction with adaptive thresholding and localized context pooling. In: Proceedings of the AAAI conference on artificial intelligence, vol 35, pp 14612–14620. https://doi.org/10.1609/aaai.v35i16.17717

Zhu G, Sun Z, Zhang S, Wei S, Li K (2022) Causality extraction model based on two-stage GCN. Soft Comput 26(24):13815–13828. https://doi.org/10.1007/s00500-022-07370-8

Titel: Joint entity and relation extraction model based on directed-relation GAT oriented to Chinese patent texts
verfasst von: Yushan Zhao
Kuan-Ching Li
Tengke Wang
Shunxiang Zhang
Publikationsdatum: 08.02.2024
Verlag: Springer Berlin Heidelberg
Erschienen in: Soft Computing
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI: https://doi.org/10.1007/s00500-024-09629-8

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Premium Partner