Skip to main content
EXTENDED SEARCH
Log in
Top
Published in:
Read chapter Read first chapter

2020 | OriginalPaper | Chapter

Understanding Word Embeddings and Language Models

Authors : Jose Manuel Gomez-Perez, Ronald Denaux, Andres Garcia-Silva

Published in: A Practical Guide to Hybrid Natural Language Processing

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Early word embeddings algorithms like word2vec and GloVe generate static distributional representations for words regardless of the context and the sense in which the word is used in a given sentence, offering poor modeling of ambiguous words and lacking coverage for out-of-vocabulary words. Hence a new wave of algorithms based on training language models such as Open AI GPT and BERT has been proposed to generate contextual word embeddings that use as input word constituents allowing them to generate representations for out-of-vocabulary words by combining the word pieces. Recently, fine-tuning pre-trained language models that have been trained on large corpora have constantly advanced the state of the art for many NLP tasks.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Business + Economics & Engineering + Technology"

Online-Abonnement

Springer Professional "Business + Economics & Engineering + Technology" gives you access to:

  • more than 102.000 books
  • more than 537 journals

from the following subject areas:

  • Automotive
  • Construction + Real Estate
  • Business IT + Informatics
  • Electrical Engineering + Electronics
  • Energy + Sustainability
  • Finance + Banking
  • Management + Leadership
  • Marketing + Sales
  • Mechanical Engineering + Materials
  • Insurance + Risk


Secure your knowledge advantage now!

inform now

Springer Professional "Engineering + Technology"

Online-Abonnement

Springer Professional "Engineering + Technology" gives you access to:

  • more than 67.000 books
  • more than 390 journals

from the following specialised fileds:

  • Automotive
  • Business IT + Informatics
  • Construction + Real Estate
  • Electrical Engineering + Electronics
  • Energy + Sustainability
  • Mechanical Engineering + Materials





 

Secure your knowledge advantage now!

inform now

Springer Professional "Business + Economics"

Online-Abonnement

Springer Professional "Business + Economics" gives you access to:

  • more than 67.000 books
  • more than 340 journals

from the following specialised fileds:

  • Construction + Real Estate
  • Business IT + Informatics
  • Finance + Banking
  • Management + Leadership
  • Marketing + Sales
  • Insurance + Risk



Secure your knowledge advantage now!

inform now
previous chapter Word, Sense, and Graph Embeddings
next chapter Capturing Meaning from Text as Word Embeddings
Metadata
Title
Understanding Word Embeddings and Language Models
Authors
Jose Manuel Gomez-Perez
Ronald Denaux
Andres Garcia-Silva
Copyright Year
2020
Book
A Practical Guide to Hybrid Natural Language Processing

Print ISBN: 978-3-030-44829-5

Electronic ISBN: 978-3-030-44830-1

Copyright Year: 2020

DOI
https://doi.org/10.1007/978-3-030-44830-1_3

Premium Partner

    Image Credits