Skip to main content
Erschienen in: Earth Science Informatics 4/2019

16.08.2019 | Research Article

BiLSTM-CRF for geological named entity recognition from the geoscience literature

verfasst von: Qinjun Qiu, Zhong Xie, Liang Wu, Liufeng Tao, Wenjia Li

Erschienen in: Earth Science Informatics | Ausgabe 4/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Many detailed geoscience reports lie unused, offering both challenges and opportunities for information extraction. In geoscience research, geological named entity recognition (GNER) is an important task in the field of geoscience information extraction. Regarding numerical geoscience data, research on information extraction remains limited. Most conventional NER approaches are heavily dependent on feature engineering, and such sentence-level-based methods suffer from the tagging inconsistency problem. Based on the above observations, this paper proposes a neural network approach, namely, attention-based bidirectional long short-term memory with a conditional random field layer (Att-BiLSTM-CRF), for name entity recognition to extract information entities describing geoscience information from geoscience reports. This approach leverages global information learned from an attention mechanism to enforce tagging consistency across multiple instances of the same token in a document. Experiments on the constructed dataset show that our method achieves comparable performance to that of other state-of-the-art systems. Additionally, our method achieved an average F1 score of 91.47% in the NER extraction task.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Arnab A, Jayasumana S, Zheng S, & Torr, PH (2016) Higher order conditional random fields in deep neural networks. European Conference on Computer Vision 524–540 Arnab A, Jayasumana S, Zheng S, & Torr, PH (2016) Higher order conditional random fields in deep neural networks. European Conference on Computer Vision 524–540
Zurück zum Zitat Babaie HA, Davarpanah A (2018) Semantic modeling of plastic deformation of polycrystalline rock. Comput Geosci 111:213–222CrossRef Babaie HA, Davarpanah A (2018) Semantic modeling of plastic deformation of polycrystalline rock. Comput Geosci 111:213–222CrossRef
Zurück zum Zitat Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166CrossRef Bengio Y, Simard P, Frasconi P (1994) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw 5(2):157–166CrossRef
Zurück zum Zitat Cernuzzi L, Pane J (2014) Toward open government in Paraguay[J]. It Professional 16(5):62–64CrossRef Cernuzzi L, Pane J (2014) Toward open government in Paraguay[J]. It Professional 16(5):62–64CrossRef
Zurück zum Zitat Chen X, Shi Z, Qiu X, et al (2017) Adversarial multi-criteria learning for Chinese word segmentation. arXiv, arXiv:1193–1203 Chen X, Shi Z, Qiu X, et al (2017) Adversarial multi-criteria learning for Chinese word segmentation. arXiv, arXiv:1193–1203
Zurück zum Zitat Chiu J P C, Nichols E (2016) Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics 4(1):357–370CrossRef Chiu J P C, Nichols E (2016) Named entity recognition with bidirectional LSTM-CNNs. Transactions of the Association for Computational Linguistics 4(1):357–370CrossRef
Zurück zum Zitat Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa PP (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537 Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa PP (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537
Zurück zum Zitat Cracknell MJ, Reading AM (2014) Geological mapping using remote sensing data: a comparison of five machine learning algorithms, their response to variations in the spatial distribution of training data and the use of explicit spatial information. Comput Geosci 63(1):22–33CrossRef Cracknell MJ, Reading AM (2014) Geological mapping using remote sensing data: a comparison of five machine learning algorithms, their response to variations in the spatial distribution of training data and the use of explicit spatial information. Comput Geosci 63(1):22–33CrossRef
Zurück zum Zitat Eltyeb S Salim N (2014) Chemical named entities recognition: a review on approaches and applications. J Cheminform 6:17 Eltyeb S Salim N (2014) Chemical named entities recognition: a review on approaches and applications. J Cheminform 6:17
Zurück zum Zitat Finkel J, Dingare S, Nguyen H et al (2004) Exploiting context for biomedical entity recognition: from syntax to the web. In: Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications. Association for Computational Linguistics 88–91 Finkel J, Dingare S, Nguyen H et al (2004) Exploiting context for biomedical entity recognition: from syntax to the web. In: Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications. Association for Computational Linguistics 88–91
Zurück zum Zitat Gao J, Li M, Huang CN, Wu A (2005) Chinese word segmentation and named entity recognition: a pragmatic approach. Computational Linguistics 31(4):531–574CrossRef Gao J, Li M, Huang CN, Wu A (2005) Chinese word segmentation and named entity recognition: a pragmatic approach. Computational Linguistics 31(4):531–574CrossRef
Zurück zum Zitat Habibi M, Weber L, Neves M, Wiegandt DL, Leser U (2017) Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics 33(14):i37–i48CrossRef Habibi M, Weber L, Neves M, Wiegandt DL, Leser U (2017) Deep learning with word embeddings improves biomedical named entity recognition. Bioinformatics 33(14):i37–i48CrossRef
Zurück zum Zitat He H, Sun X (2017) A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In Thirty-First AAAI Conference on Artificial Intelligence 3216–3222 He H, Sun X (2017) A unified model for cross-domain and semi-supervised named entity recognition in Chinese social media. In Thirty-First AAAI Conference on Artificial Intelligence 3216–3222
Zurück zum Zitat He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Zurück zum Zitat Hettne KM et al (2009) A dictionary to identify small molecules and drugs in free text. Bioinformatics 25:2983–2991CrossRef Hettne KM et al (2009) A dictionary to identify small molecules and drugs in free text. Bioinformatics 25:2983–2991CrossRef
Zurück zum Zitat Hinton G, Deng L, Yu D, Dahl G, Mohamed AR, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath T, Kingsbury B (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97CrossRef Hinton G, Deng L, Yu D, Dahl G, Mohamed AR, Jaitly N, Senior A, Vanhoucke V, Nguyen P, Sainath T, Kingsbury B (2012) Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups. IEEE Signal Process Mag 29(6):82–97CrossRef
Zurück zum Zitat Huang L, Du Y, Chen G (2015) GeoSegmenter: a statistically learned Chinese word segmenter for the geoscience domain. Comput Geosci 76:11–17CrossRef Huang L, Du Y, Chen G (2015) GeoSegmenter: a statistically learned Chinese word segmenter for the geoscience domain. Comput Geosci 76:11–17CrossRef
Zurück zum Zitat Li L, Liu Y, Zhu H, Ying S, Luo Q, Luo H, Kuai X, Xia H, Shen H (2017) A bibliometric and visual analysis of global geo-ontology research. Comput Geosci 99:1–8CrossRef Li L, Liu Y, Zhu H, Ying S, Luo Q, Luo H, Kuai X, Xia H, Shen H (2017) A bibliometric and visual analysis of global geo-ontology research. Comput Geosci 99:1–8CrossRef
Zurück zum Zitat Lima LA, Görnitz N, Varella LE, Vellasco M, Müller KR, Nakajima S (2017) Porosity estimation by semi-supervised learning with sparsely available labeled samples. Comput Geosci 106:33–48CrossRef Lima LA, Görnitz N, Varella LE, Vellasco M, Müller KR, Nakajima S (2017) Porosity estimation by semi-supervised learning with sparsely available labeled samples. Comput Geosci 106:33–48CrossRef
Zurück zum Zitat Liu S et al (2015) Drug name recognition: approaches and resources. Information 6:790–810CrossRef Liu S et al (2015) Drug name recognition: approaches and resources. Information 6:790–810CrossRef
Zurück zum Zitat Ma X, Fox P (2013) Recent progress on geologic time ontologies and considerations for future works. Earth Sci Inf 6(1):31–46CrossRef Ma X, Fox P (2013) Recent progress on geologic time ontologies and considerations for future works. Earth Sci Inf 6(1):31–46CrossRef
Zurück zum Zitat Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354 Ma X, Hovy E (2016) End-to-end sequence labeling via bi-directional lstm-cnns-crf. arXiv preprint arXiv:1603.01354
Zurück zum Zitat Ma X, Carranza EJM, Wu C, van der Meer FD (2012) Ontology-aided annotation, visualization, and generalization of geological time-scale information from online geological map services. Comput Geosci 40:107–119CrossRef Ma X, Carranza EJM, Wu C, van der Meer FD (2012) Ontology-aided annotation, visualization, and generalization of geological time-scale information from online geological map services. Comput Geosci 40:107–119CrossRef
Zurück zum Zitat Ma X, Hummer D, Golden JJ et al (2017) Using visual exploratory data analysis to facilitate collaboration and hypothesis generation in cross-disciplinary research. ISPRS Int J Geo Inf 6(11):368CrossRef Ma X, Hummer D, Golden JJ et al (2017) Using visual exploratory data analysis to facilitate collaboration and hypothesis generation in cross-disciplinary research. ISPRS Int J Geo Inf 6(11):368CrossRef
Zurück zum Zitat Mastella LS, Abel M, De Ros LF et al (2007) Event ordering reasoning ontology applied to petrology and geological modelling. In: Theoretical Advances and applications of fuzzy logic and soft computing. Springer, Berlin, pp 465–475 Mastella LS, Abel M, De Ros LF et al (2007) Event ordering reasoning ontology applied to petrology and geological modelling. In: Theoretical Advances and applications of fuzzy logic and soft computing. Springer, Berlin, pp 465–475
Zurück zum Zitat Mikolov T, Kombrink S, Burget L, et al (2011) Extensions of recurrent neural network language model. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 5528–5531 Mikolov T, Kombrink S, Burget L, et al (2011) Extensions of recurrent neural network language model. In: 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, pp 5528–5531
Zurück zum Zitat Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781
Zurück zum Zitat Nawroth C, Schmedding M, Brocks H, Kaufmann M, Fuchs M, Hemmje M (2015) Towards cloud-based knowledge capturing based on natural language processing. Procedia Computer Science 68:206–216CrossRef Nawroth C, Schmedding M, Brocks H, Kaufmann M, Fuchs M, Hemmje M (2015) Towards cloud-based knowledge capturing based on natural language processing. Procedia Computer Science 68:206–216CrossRef
Zurück zum Zitat Pennington J, Socher R, Glove MC (2014) Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543 Pennington J, Socher R, Glove MC (2014) Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543
Zurück zum Zitat Poria S, Peng H, Hussain A, Howard N, Cambria E (2017) Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis. Neurocomputing 261:217–230CrossRef Poria S, Peng H, Hussain A, Howard N, Cambria E (2017) Ensemble application of convolutional neural networks and multiple kernel learning for multimodal sentiment analysis. Neurocomputing 261:217–230CrossRef
Zurück zum Zitat Rei M, Crichton G K, Pyysalo S (2016) Attending to characters in neural sequence labeling models. In: International Conference on Computational Linguistics, pp 309–318 Rei M, Crichton G K, Pyysalo S (2016) Attending to characters in neural sequence labeling models. In: International Conference on Computational Linguistics, pp 309–318
Zurück zum Zitat Rumelhart D, Hinton G, Williams R (1986) Learning representations by back-propagating errors. Nature 323:533–536CrossRef Rumelhart D, Hinton G, Williams R (1986) Learning representations by back-propagating errors. Nature 323:533–536CrossRef
Zurück zum Zitat Santos R, Flores PM, Calado P et al (2017) Toponym matching through deep neural networks. Int J Geogr Inf Sci (3):1–25 Santos R, Flores PM, Calado P et al (2017) Toponym matching through deep neural networks. Int J Geogr Inf Sci (3):1–25
Zurück zum Zitat Sarkar K, Shaw SK (2017) A memory-based learning approach for named entity recognition in Hindi. J Intell Syst 26(2):301–321 Sarkar K, Shaw SK (2017) A memory-based learning approach for named entity recognition in Hindi. J Intell Syst 26(2):301–321
Zurück zum Zitat Shen Y, Yun H, Lipton ZC et al (2017) Deep Active learning for named entity recognition. arXiv preprint arXiv:1707.05928 Shen Y, Yun H, Lipton ZC et al (2017) Deep Active learning for named entity recognition. arXiv preprint arXiv:1707.05928
Zurück zum Zitat Sutskever I, Vinyals O, Le Q V (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112 Sutskever I, Vinyals O, Le Q V (2014) Sequence to sequence learning with neural networks. In: Advances in neural information processing systems, pp 3104–3112
Zurück zum Zitat Tsochantaridis I, Joachims T, Hofmann T et al (2005) Large margin methods for structured and interdependent output variables. J Mach Learn Res 6(2):1453–1484 Tsochantaridis I, Joachims T, Hofmann T et al (2005) Large margin methods for structured and interdependent output variables. J Mach Learn Res 6(2):1453–1484
Zurück zum Zitat Unanue IJ, Borzeshi EZ, Piccardi M (2017) Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition. J Biomed Inform 76:102–109CrossRef Unanue IJ, Borzeshi EZ, Piccardi M (2017) Recurrent neural networks with specialized word embeddings for health-domain named-entity recognition. J Biomed Inform 76:102–109CrossRef
Zurück zum Zitat Viterbi A (1967) Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transinformattheory 13(2):260–269 Viterbi A (1967) Error bounds for convolutional codes and an asymptotically optimum decoding algorithm. IEEE Transinformattheory 13(2):260–269
Zurück zum Zitat Wang C, Chen J, Xiao F (2016a) Application of empirical model decomposition and independent component analysis to magnetic anomalies separation: a case study for Gobi Desert coverage in eastern tian Shan, China. In: Geostatistical and geospatial approaches for the characterization of natural resources in the environment. Springer, Cham, pp 593–598CrossRef Wang C, Chen J, Xiao F (2016a) Application of empirical model decomposition and independent component analysis to magnetic anomalies separation: a case study for Gobi Desert coverage in eastern tian Shan, China. In: Geostatistical and geospatial approaches for the characterization of natural resources in the environment. Springer, Cham, pp 593–598CrossRef
Zurück zum Zitat Wang C, Chen J, Xiao F, Fode T, Li L (2016b) Radioelement distributions and analysis of micro topographical influences in a shallow covered area, Inner Mongolia, China: implications for mineral exploration. J Appl Geophys 133:62–69CrossRef Wang C, Chen J, Xiao F, Fode T, Li L (2016b) Radioelement distributions and analysis of micro topographical influences in a shallow covered area, Inner Mongolia, China: implications for mineral exploration. J Appl Geophys 133:62–69CrossRef
Zurück zum Zitat Wang C, Ma X, Chen J (2018a) Ontology-driven data integration and visualization for exploring regional geologic time and paleontological information. Comput Geosci 115:12–19CrossRef Wang C, Ma X, Chen J (2018a) Ontology-driven data integration and visualization for exploring regional geologic time and paleontological information. Comput Geosci 115:12–19CrossRef
Zurück zum Zitat Wang C, Ma X, Chen J, Chen J (2018b) Information extraction and knowledge graph construction from geoscience literature. Comput Geosci 112:112–120CrossRef Wang C, Ma X, Chen J, Chen J (2018b) Information extraction and knowledge graph construction from geoscience literature. Comput Geosci 112:112–120CrossRef
Zurück zum Zitat Werbos PJ (1988) Generalization of backpropagation with application to a recurrent gas market model. Neural Netw 1(4):339–356CrossRef Werbos PJ (1988) Generalization of backpropagation with application to a recurrent gas market model. Neural Netw 1(4):339–356CrossRef
Zurück zum Zitat Wu L, Xue L, Li C, Lv X, Chen Z, Jiang B, Guo M, Xie Z (2017) A knowledge-driven geospatially enabled framework for geological big data. ISPRS Int J Geo Inf 6(6):166CrossRef Wu L, Xue L, Li C, Lv X, Chen Z, Jiang B, Guo M, Xie Z (2017) A knowledge-driven geospatially enabled framework for geological big data. ISPRS Int J Geo Inf 6(6):166CrossRef
Zurück zum Zitat Xiao F, Chen Z, Chen J, Zhou Y (2016) A batch sliding window method for local singularity mapping and its application for geochemical anomaly identification. Comput Geosci 90(PA):189–201CrossRef Xiao F, Chen Z, Chen J, Zhou Y (2016) A batch sliding window method for local singularity mapping and its application for geochemical anomaly identification. Comput Geosci 90(PA):189–201CrossRef
Zurück zum Zitat Xie S, Girshick R, Dollár P et al (2017) Aggregated residual transformations for deep neural networks. In: 2017 IEEE conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 5987–5995 Xie S, Girshick R, Dollár P et al (2017) Aggregated residual transformations for deep neural networks. In: 2017 IEEE conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 5987–5995
Zurück zum Zitat Yang Z, Salakhutdinov R, Cohen WW (2017) Transfer learning for sequence tagging with hierarchical recurrent networks. arXiv preprint arXiv:1703.06345 Yang Z, Salakhutdinov R, Cohen WW (2017) Transfer learning for sequence tagging with hierarchical recurrent networks. arXiv preprint arXiv:1703.06345
Zurück zum Zitat Zeng D, Sun C, Lin L et al (2017) Lstm-crf for drug-named entity recognition. Entropy 19(6):283CrossRef Zeng D, Sun C, Lin L et al (2017) Lstm-crf for drug-named entity recognition. Entropy 19(6):283CrossRef
Zurück zum Zitat Zhong J, Aydina A, McGuinness DL (2009) Ontology of fractures. J Struct Geol 31(3):251–259CrossRef Zhong J, Aydina A, McGuinness DL (2009) Ontology of fractures. J Struct Geol 31(3):251–259CrossRef
Zurück zum Zitat Zhu Q, Li X, Conesa A et al (2017) GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text. Bioinformatics:btx815 Zhu Q, Li X, Conesa A et al (2017) GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text. Bioinformatics:btx815
Zurück zum Zitat Zhu Q, Li X, Conesa A, Pereira C (2018) GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text. Bioinformatics 34(9):1547–1554CrossRef Zhu Q, Li X, Conesa A, Pereira C (2018) GRAM-CNN: a deep learning approach with local context for named entity recognition in biomedical text. Bioinformatics 34(9):1547–1554CrossRef
Metadaten
Titel
BiLSTM-CRF for geological named entity recognition from the geoscience literature
verfasst von
Qinjun Qiu
Zhong Xie
Liang Wu
Liufeng Tao
Wenjia Li
Publikationsdatum
16.08.2019
Verlag
Springer Berlin Heidelberg
Erschienen in
Earth Science Informatics / Ausgabe 4/2019
Print ISSN: 1865-0473
Elektronische ISSN: 1865-0481
DOI
https://doi.org/10.1007/s12145-019-00390-3

Weitere Artikel der Ausgabe 4/2019

Earth Science Informatics 4/2019 Zur Ausgabe

Premium Partner