Skip to main content
Top

2020 | OriginalPaper | Chapter

6. Erfassung der Bedeutung von geschriebenem Text

Authors : Gerhard Paaß, Dirk Hecker

Published in: Künstliche Intelligenz

Publisher: Springer Fachmedien Wiesbaden

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Zusammenfassung

 Die allermeisten Informationen in unserer Gesellschaft sind als geschriebener Text verfügbar. Diese Kapitel beschreibt daher die Extraktion von Wissen aus geschriebenem Text. In tiefen neuronalen Netzen (TNN) werden Wörter, Sätze und Dokumente meist durch Embedding-Vektoren repräsentiert. Während einfache Verfahren zur Bestimmung von Embeddings nur zur Darstellung der Bedeutung von Wörtern verwendet werden können, haben rekurrente neuronale Netze (RNN) das Potential, die Bedeutung eines Satzes zu erfassen. Das bekannteste RNN, das Long-Short Time Memory (LSTM), kann als Sprachmodell genutzt werden. Es prognostizier das jeweils nächste Wort eines Satzes und kann dadurch die syntaktische und semantische Struktur einer Sprache erfassen. Es kann unter anderem zur Übersetzung von einer Sprache in eine andere genutzt werden. Das Transformermodell berechnet die „Korrelation“ zwischen alle Wörtern eines Satzes und kann damit kontextsensitive Embeddingvektoren ableiten, welche wesentlich feinere Bedeutungsnuancen erfassen. Das BERT-Modell baut hierauf auf. Es wird auf einen großen Textdatenbestand unüberwacht vortrainiert und dann auf einen kleinen gelabelten Datenbestand an spezielle Aufgaben angepasst. Mit diesen Modellen konnte bei vielfältigen semantischen Aufgaben mittlerweile die Leistung von Menschen für nahezu erreicht oder übertroffen werden. Weitere Abschnitte widmen sich der Beschreibung von Bildern durch Text und der Erklärung von Prognosen tiefer neuronaler Netze. 

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
go back to reference Aharoni, R., Johnson, M. und Firat, O. 2019. Massively multilingual neural machine translation. [Online] 2019. [Zitat vom: 21. 10. 2019.] arXiv preprint arXiv:1903.00089. Aharoni, R., Johnson, M. und Firat, O. 2019. Massively multilingual neural machine translation. [Online] 2019. [Zitat vom: 21. 10. 2019.] arXiv preprint arXiv:1903.00089.
go back to reference Bahdanau, D., Cho, K. und Bengio, Y. 2015. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. ICLR. 2015. Bahdanau, D., Cho, K. und Bengio, Y. 2015. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. ICLR. 2015.
go back to reference Bar-Hillel, Y. 1962. The future of machine translation. Times Literary Supplement. Times Newspapers, London. April 20th. 1962. (1969) Bar-Hillel, Y. 1962. The future of machine translation. Times Literary Supplement. Times Newspapers, London. April 20th. 1962. (1969)
go back to reference Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35(8), 1798–1828 (2013) Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives. IEEE transactions on pattern analysis and machine intelligence 35(8), 1798–1828 (2013)
go back to reference Bojanowski, P., et al. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135–146. 2017. Bojanowski, P., et al. 2017. Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135–146. 2017.
go back to reference Bojar, O., et al. 2017. Findings of the 2017 Conference on Machine Translation (WMT17). In Proceedings of the Second Conference on Machine Translation (pp. 169–214). 2017. Bojar, O., et al. 2017. Findings of the 2017 Conference on Machine Translation (WMT17). In Proceedings of the Second Conference on Machine Translation (pp. 169–214). 2017.
go back to reference Britz, D., et al. 2017. Massive Exploration of Neural Machine Translation Architectures. EMNLP. 2017. Britz, D., et al. 2017. Massive Exploration of Neural Machine Translation Architectures. EMNLP. 2017.
go back to reference Clark, P., et al. 2019. From ‘F’ to ‘A’ on the N.Y. Regents Science Exams: An Overview of the Aristo Project. [Online] 2019. [Zitat vom: 09. 09. 2019.] Clark, P., et al. 2019. From ‘F’ to ‘A’ on the N.Y. Regents Science Exams: An Overview of the Aristo Project. [Online] 2019. [Zitat vom: 09. 09. 2019.]
go back to reference Devlin, J., et al. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint. [Online] 2018. arXiv:1810.04805. Devlin, J., et al. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint. [Online] 2018. arXiv:1810.04805.
go back to reference Diaz, F., Mitra, B. und Craswell, N. 2016. Query Expansion with Locally-Trained Word Embeddings. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol. 1, pp. 367–377). 2016. Diaz, F., Mitra, B. und Craswell, N. 2016. Query Expansion with Locally-Trained Word Embeddings. Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Vol. 1, pp. 367–377). 2016.
go back to reference Dyer, C. 2014. Notes on noise contrastive estimation and negative sampling. [Online] 2014. arXiv preprint arXiv:1410.8251. Dyer, C. 2014. Notes on noise contrastive estimation and negative sampling. [Online] 2014. arXiv preprint arXiv:1410.8251.
go back to reference Ebrahimi, J., Lowd, D., Dou, D. On adversarial examples for character-level neural machine translation. COLING 2018, 653–663 (2018) Ebrahimi, J., Lowd, D., Dou, D. On adversarial examples for character-level neural machine translation. COLING 2018, 653–663 (2018)
go back to reference Firth, J.R. A synopsis of linguistic theory 1930–1955. Studies in linguistic analysis, S. 1–32. Reprinted in F.R. Palmer, ed. (1968). Selected papers of J.R. Firth 1952–1959. Longman, London (1957) Firth, J.R. A synopsis of linguistic theory 1930–1955. Studies in linguistic analysis, S. 1–32. Reprinted in F.R. Palmer, ed. (1968). Selected papers of J.R. Firth 1952–1959. Longman, London (1957)
go back to reference Guidotti, R., et al.:A survey of methods for explaining black. box models. ACM computing surveys (CSUR) 51(5), 93 (2018) Guidotti, R., et al.:A survey of methods for explaining black. box models. ACM computing surveys (CSUR) 51(5), 93 (2018)
go back to reference Hall, P. 2019.On the Art and Science of Explainable Machine Learning: Techniques, Recommendations, and Responsibilities. [Online] 2019. [Zitat vom: 24. 08. 2019.] arXiv:1810.02909v3. Hall, P. 2019.On the Art and Science of Explainable Machine Learning: Techniques, Recommendations, and Responsibilities. [Online] 2019. [Zitat vom: 24. 08. 2019.] arXiv:1810.02909v3.
go back to reference Hassan, H., et al. 2018.Achieving human parity on automatic Chinese to English news translation. arXiv preprint. [Online] 2018. arXiv:1803.05567. (2018) Hassan, H., et al. 2018.Achieving human parity on automatic Chinese to English news translation. arXiv preprint. [Online] 2018. arXiv:1803.05567. (2018)
go back to reference Hendrycks, D. und Gimpel, K. 2016. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. [Online] 2016. arXiv preprint arXiv:1606.08415. Hendrycks, D. und Gimpel, K. 2016. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. [Online] 2016. arXiv preprint arXiv:1606.08415.
go back to reference Johnson, M., et al. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5, 339–351. 2017. Johnson, M., et al. 2017. Google’s multilingual neural machine translation system: Enabling zero-shot translation. Transactions of the Association for Computational Linguistics, 5, 339–351. 2017.
go back to reference Karpukhin, V., et al.: Training on synthetic noise improves robustness to natural noise in machine translation. [Online] 2019. [Zitat vom: 15. 06. 2019.] arXiv preprint arXiv:1902.01509. (2019) Karpukhin, V., et al.: Training on synthetic noise improves robustness to natural noise in machine translation. [Online] 2019. [Zitat vom: 15. 06. 2019.] arXiv preprint arXiv:1902.01509. (2019)
go back to reference Kudugunta, S. R., et al. 2019. Investigating Multilingual NMT Representations at Scale. [Online] 2019. [Zitat vom: 21. 10. 2019.] arXiv preprint arXiv:1909.02197. Kudugunta, S. R., et al. 2019. Investigating Multilingual NMT Representations at Scale. [Online] 2019. [Zitat vom: 21. 10. 2019.] arXiv preprint arXiv:1909.02197.
go back to reference Lample, G., et al.: Neural architectures for named entity recognition. Proceedings of NAACL-HLT, S. 260–270. (2016) Lample, G., et al.: Neural architectures for named entity recognition. Proceedings of NAACL-HLT, S. 260–270. (2016)
go back to reference Lample, G., et al. 2018. Phrase-based & neural unsupervised machine translation. [Online] 2018. arXiv preprint arXiv:1804.07755. Lample, G., et al. 2018. Phrase-based & neural unsupervised machine translation. [Online] 2018. arXiv preprint arXiv:1804.07755.
go back to reference Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., Müller, K. R. Unmasking clever hans predictors and assessing what machines really learn. Nature communications, 10(1), 1–8 (2019) Lapuschkin, S., Wäldchen, S., Binder, A., Montavon, G., Samek, W., Müller, K. R. Unmasking clever hans predictors and assessing what machines really learn. Nature communications, 10(1), 1–8 (2019)
go back to reference Lan, Z., et al. 2019. Albert: A lite bert for self-supervised learning of language representations. [Online] 2019. [Zitat vom: 09. 01. 2020.] arXiv preprint arXiv:1909.11942. Lan, Z., et al. 2019. Albert: A lite bert for self-supervised learning of language representations. [Online] 2019. [Zitat vom: 09. 01. 2020.] arXiv preprint arXiv:1909.11942.
go back to reference Lee, J., et al. 2019. Biobert: pre-trained biomedical language representation model for biomedical text mining. [Online] 2019. [Zitat vom: 16. 07. 2019.] arXiv preprint arXiv:1901.08746. Lee, J., et al. 2019. Biobert: pre-trained biomedical language representation model for biomedical text mining. [Online] 2019. [Zitat vom: 16. 07. 2019.] arXiv preprint arXiv:1901.08746.
go back to reference Lenat, D. B. 1995. CYC: A large-scale investment in knowledge infrastructure. Communications of the ACM, 38(11), 33–38. 1995. Lenat, D. B. 1995. CYC: A large-scale investment in knowledge infrastructure. Communications of the ACM, 38(11), 33–38. 1995.
go back to reference Levy, O., Goldberg, Y. und Dagan, I. 2015. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3, 211–225. 2015. Levy, O., Goldberg, Y. und Dagan, I. 2015. Improving distributional similarity with lessons learned from word embeddings. Transactions of the Association for Computational Linguistics, 3, 211–225. 2015.
go back to reference Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., ... Zettlemoyer, L.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. (2019) Lewis, M., Liu, Y., Goyal, N., Ghazvininejad, M., Mohamed, A., Levy, O., ... Zettlemoyer, L.: Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461. (2019)
go back to reference Li, X., et al.: Oscar: Object-semantics aligned pre-training for vision-language tasks. European Conference on Computer Vision. Springer, Cham (2020) Li, X., et al.: Oscar: Object-semantics aligned pre-training for vision-language tasks. European Conference on Computer Vision. Springer, Cham (2020)
go back to reference Lin, T., et al.: Microsoft COCO: common objects in context. In European conference on computer vision, S. 740–755. Springer, Cham (2014) Lin, T., et al.: Microsoft COCO: common objects in context. In European conference on computer vision, S. 740–755. Springer, Cham (2014)
go back to reference Ma, M., et al.: STACL: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, S. 3025–3036. (2019) Ma, M., et al.: STACL: Simultaneous translation with implicit anticipation and controllable latency using prefix-to-prefix framework. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, S. 3025–3036. (2019)
go back to reference Melis, G., et al.: On the state of the art of evaluation in neural language models. [Online] 2017. arXiv preprint arXiv:1707.05589 (2017) Melis, G., et al.: On the state of the art of evaluation in neural language models. [Online] 2017. arXiv preprint arXiv:1707.05589 (2017)
go back to reference Montavon, G., Samek, W., Müller, K. R.: Methods for interpreting and understanding deep neural networks. Digital Signal Processing 73, 1–15 (2018) Montavon, G., Samek, W., Müller, K. R.: Methods for interpreting and understanding deep neural networks. Digital Signal Processing 73, 1–15 (2018)
go back to reference Mushtaq, M. F., et al.: Neural network techniques for time series prediction: A review. Int. J. Engineering Information Computing and Appl. (IJEAIS), 1(1). (2019) Mushtaq, M. F., et al.: Neural network techniques for time series prediction: A review. Int. J. Engineering Information Computing and Appl. (IJEAIS), 1(1). (2019)
go back to reference Petroni, F., et al.: Language models as knowledge bases? arXiv preprint arXiv:1909.01066. (2019) Petroni, F., et al.: Language models as knowledge bases? arXiv preprint arXiv:1909.01066. (2019)
go back to reference Popel, M., Bojar, O.: Training tips for the transformer model. Karls-Universität, Prag (2018) Popel, M., Bojar, O.: Training tips for the transformer model. Karls-Universität, Prag (2018)
go back to reference Quine, Willard Van Orman.: Ontological relativity. Chapter 2. Ontological relativity and other essays. Columbia University Press. pp. 26–68. ISBN 0231083572. 1969. Quine, Willard Van Orman.: Ontological relativity. Chapter 2. Ontological relativity and other essays. Columbia University Press. pp. 26–68. ISBN 0231083572. 1969.
go back to reference Rajpurkar, P., et al. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint. [Online] 2016. arXiv:1606.05250. Rajpurkar, P., et al. 2016. Squad: 100,000+ questions for machine comprehension of text. arXiv preprint. [Online] 2016. arXiv:1606.05250.
go back to reference Ribeiro, M. T., Singh, S. und Guestrin, C.: Model-agnostic interpretability of machine learning. [Online] 2016. [Zitat vom: 24. 08. 2019.] arXiv preprint arXiv:1606.05386. (2016) Ribeiro, M. T., Singh, S. und Guestrin, C.: Model-agnostic interpretability of machine learning. [Online] 2016. [Zitat vom: 24. 08. 2019.] arXiv preprint arXiv:1606.05386. (2016)
go back to reference Ritter, A., Clark, S. und Etzioni, O.: Named entity recognition in tweets: an experimental study. EMNLP (pp. 1524–1534). Association for Computational Linguistics (2011) Ritter, A., Clark, S. und Etzioni, O.: Named entity recognition in tweets: an experimental study. EMNLP (pp. 1524–1534). Association for Computational Linguistics (2011)
go back to reference Rumelhart, D.E, Hinton, G.E. und Williams, R.J. 1986. Learning Representations by back-proagating errors. Nature 329 (9), pp. 533–536. 1986. Rumelhart, D.E, Hinton, G.E. und Williams, R.J. 1986. Learning Representations by back-proagating errors. Nature 329 (9), pp. 533–536. 1986.
go back to reference Sap, M., et al. 2019. ATOMIC: an atlas of machine commonsense for if-then reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, pp. 3027–30). 2019. Sap, M., et al. 2019. ATOMIC: an atlas of machine commonsense for if-then reasoning. In Proceedings of the AAAI Conference on Artificial Intelligence (Vol. 33, pp. 3027–30). 2019.
go back to reference Schmidhuber, Jürgen und Hochreiter, Sepp. 1997. Long short-term memory. Neural Comput 9.8: 1735–1780. 1997. Schmidhuber, Jürgen und Hochreiter, Sepp. 1997. Long short-term memory. Neural Comput 9.8: 1735–1780. 1997.
go back to reference Sharma, P., et al. 2018. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (pp. 2556–2565). 2018. Sharma, P., et al. 2018. Conceptual captions: A cleaned, hypernymed, image alt-text dataset for automatic image captioning. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (pp. 2556–2565). 2018.
go back to reference Shazeer, N., et al. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. [Online] 2017. [Zitat vom: 21. 10. 2019.] arXiv preprint arXiv:1701.06538. Shazeer, N., et al. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. [Online] 2017. [Zitat vom: 21. 10. 2019.] arXiv preprint arXiv:1701.06538.
go back to reference Strobelt, H., et al.: Visual analysis of hidden state dynamics in recurrent neural networks. [Online] 2016. arXiv preprint arXiv:1606.07461 (2016) Strobelt, H., et al.: Visual analysis of hidden state dynamics in recurrent neural networks. [Online] 2016. arXiv preprint arXiv:1606.07461 (2016)
go back to reference Sutskever, I., Vinyals, O. und Le, Q. V.: Sequence to sequence learning with neural networks. Advances in neural information processing systems , S. 3104–3112. (2014) Sutskever, I., Vinyals, O. und Le, Q. V.: Sequence to sequence learning with neural networks. Advances in neural information processing systems , S. 3104–3112. (2014)
go back to reference Thrun, S. (1998).Lifelong learning algorithms. In Learning to learn. (pp. 181–209). [Buchverf.] S. Thrun und L. Pratt. Learning to Learn. s.l. : Springer, Boston, MA., 1998. Thrun, S. (1998).Lifelong learning algorithms. In Learning to learn. (pp. 181–209). [Buchverf.] S. Thrun und L. Pratt. Learning to Learn. s.l. : Springer, Boston, MA., 1998.
go back to reference van der Maaten, L.J.P. und Hinton, G.E. 2008. Visualizing Data Using t-SNE. Journal of Machine Learning Research. 9: 2579–2605. 2008. van der Maaten, L.J.P. und Hinton, G.E. 2008. Visualizing Data Using t-SNE. Journal of Machine Learning Research. 9: 2579–2605. 2008.
go back to reference Vaswani, Ashish, et al. 2017. Attention is All you Need. NIPS 2017: 6000–6010. 2017. Vaswani, Ashish, et al. 2017. Attention is All you Need. NIPS 2017: 6000–6010. 2017.
go back to reference Wu, L. Y., et al. 2018. Starspace: Embed all the things! Thirty-Second AAAI Conference on Artificial Intelligence. 2018. Wu, L. Y., et al. 2018. Starspace: Embed all the things! Thirty-Second AAAI Conference on Artificial Intelligence. 2018.
go back to reference Yang, Z., et al. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. [Online] 2019. [Zitat vom: 09. 01. 2020.] arXiv preprint arXiv:1906.08237. Yang, Z., et al. 2019. XLNet: Generalized Autoregressive Pretraining for Language Understanding. [Online] 2019. [Zitat vom: 09. 01. 2020.] arXiv preprint arXiv:1906.08237.
go back to reference Zellers, R., et al. 2018. Swag: A large-scale adversarial dataset for grounded commonsense inference. [Online] 2018. arXiv preprint arXiv:1808.05326. Zellers, R., et al. 2018. Swag: A large-scale adversarial dataset for grounded commonsense inference. [Online] 2018. arXiv preprint arXiv:1808.05326.
go back to reference Zhu, Y., et al. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. Proceedings of the IEEE international conference on computer vision (pp. 19–27). 2015. Zhu, Y., et al. 2015. Aligning books and movies: Towards story-like visual explanations by watching movies and reading books. Proceedings of the IEEE international conference on computer vision (pp. 19–27). 2015.
Metadata
Title
Erfassung der Bedeutung von geschriebenem Text
Authors
Gerhard Paaß
Dirk Hecker
Copyright Year
2020
DOI
https://doi.org/10.1007/978-3-658-30211-5_6

Premium Partner