nach oben

2018 | Buch

Deep Learning in Natural Language Processing

herausgegeben von: Dr. Li Deng, Yang Liu

Verlag: Springer Singapore

Enthalten in: Springer Professional "Wirtschaft+Technik" , Springer Professional "Technik" , Springer Professional "Wirtschaft"

Einloggen, um Zugang zu erhalten

Über dieses Buch

In recent years, deep learning has fundamentally changed the landscapes of a number of areas in artificial intelligence, including speech, vision, natural language, robotics, and game playing. In particular, the striking success of deep learning in a wide variety of natural language processing (NLP) applications has served as a benchmark for the advances in one of the most important tasks in artificial intelligence.

This book reviews the state of the art of deep learning research and its successful applications to major NLP tasks, including speech recognition and understanding, dialogue systems, lexical analysis, parsing, knowledge graphs, machine translation, question answering, sentiment analysis, social computing, and natural language generation from images. Outlining and analyzing various research frontiers of NLP in the deep learning era, it features self-contained, comprehensive chapters written by leading researchers in the field. A glossary of technical terms and commonly used acronyms in the intersection of deep learning and NLP is also provided.

The book appeals to advanced undergraduate and graduate students, post-doctoral researchers, lecturers and industrial researchers, as well as anyone interested in deep learning and natural language processing.

Inhaltsverzeichnis

Frontmatter

Chapter 1. A Joint Introduction to Natural Language Processing and to Deep Learning

Abstract

In this chapter, we set up the fundamental framework for the book. We first provide an introduction to the basics of natural language processing (NLP) as an integral part of artificial intelligence. We then survey the historical development of NLP, spanning over five decades, in terms of three waves. The first two waves arose as rationalism and empiricism, paving ways to the current deep learning wave. The key pillars underlying the deep learning revolution for NLP consist of (1) distributed representations of linguistic entities via embedding, (2) semantic generalization due to the embedding, (3) long-span deep sequence modeling of natural language, (4) hierarchical networks effective for representing linguistic levels from low to high, and (5) end-to-end deep learning methods to jointly solve many NLP tasks. After the survey, several key limitations of current deep learning technology for NLP are analyzed. This analysis leads to five research directions for future advances in NLP.

Li Deng, Yang Liu

Chapter 2. Deep Learning in Conversational Language Understanding

Abstract

Recent advancements in AI resulted in increased availability of conversational assistants that can help with tasks such as seeking times to schedule an event and creating a calendar entry at that time, finding a restaurant and booking a table there at a certain time. However, creating automated agents with human-level intelligence still remains one of the most challenging problems of AI. One key component of such systems is conversational language understanding, which is a holy grail area of research for decades, as it is not a clearly defined task but relies heavily on the AI application it is used for. Nevertheless, this chapter attempts to compile the recent deep learning based literature on such goal-oriented conversational language understanding studies, starting with a historical perspective, pre-deep learning era work, moving toward most recent advances in this field.

Gokhan Tur, Asli Celikyilmaz, Xiaodong He, Dilek Hakkani-Tür, Li Deng

Chapter 3. Deep Learning in Spoken and Text-Based Dialog Systems

Abstract

Last few decades have witnessed substantial breakthroughs on several areas of speech and language understanding research, specifically for building human to machine conversational dialog systems. Dialog systems, also known as interactive conversational agents, virtual agents or sometimes chatbots, are useful in a wide range of applications ranging from technical support services to language learning tools and entertainment. Recent success in deep neural networks has spurred the research in building data-driven dialog models. In this chapter, we present state-of-the-art neural network architectures and details on each of the components of building a successful dialog system using deep learning. Task-oriented dialog systems would be the focus of this chapter, and later different networks are provided for building open-ended non-task-oriented dialog systems. Furthermore, to facilitate research in this area, we have a survey of publicly available datasets and software tools suitable for data-driven learning of dialog systems. Finally, appropriate choice of evaluation metrics are discussed for the learning objective.

Asli Celikyilmaz, Li Deng, Dilek Hakkani-Tür

Chapter 4. Deep Learning in Lexical Analysis and Parsing

Abstract

Lexical analysis and parsing tasks model the deeper properties of the words and their relationships to each other. The commonly used techniques involve word segmentation, part-of-speech tagging and parsing. A typical characteristic of such tasks is that the outputs are structured. Two types of methods are usually used to solve these structured prediction tasks: graph-based methods and transition-based methods. Graph-based methods differentiate output structures based on their characteristics directly, while transition-based methods transform output construction processes into state transition processes, differentiating sequences of transition actions. Neural network models have been successfully used for both graph-based and transition-based structured prediction. In this chapter, we give a review of applying deep learning in lexical analysis and parsing, and compare with traditional statistical methods.

Wanxiang Che, Yue Zhang

Chapter 5. Deep Learning in Knowledge Graph

Abstract

Knowledge Graph (KG) is a fundamental resource for human-like commonsense reasoning and natural language understanding, which contains rich knowledge about the world’s entities, entities’ attributes, and semantic relations between different entities. Recent years have witnessed the remarkable success of deep learning techniques in KG. In this chapter, we introduce three broad categories of deep learning-based KG techniques: (1) knowledge representation learning techniques which embed entities and relations in a KG into a dense, low-dimensional, and real-valued semantic space; (2) neural relation extraction techniques which extract facts/relations from text, which can then be used to construct/complete KG; (3) deep learning-based entity linking techniques which bridge Knowledge Graph with textual data, which can facilitate many different tasks.

Zhiyuan Liu, Xianpei Han

Chapter 6. Deep Learning in Machine Translation

Abstract

Machine translation (MT) is an important natural language processing task that investigates the use of computers to translate human languages automatically. Deep learning-based methods have made significant progress in recent years and quickly become the new de facto paradigm of MT in both academia and industry. This chapter introduces two broad categories of deep learning-based MT methods: (1) component-wise deep learning for machine translation that leverages deep learning to improve the capacity of the main components of SMT such as translation models, reordering models, and language models; and (2) end-to-end deep learning for machine translation that uses neural networks to directly map between source and target languages based on the encoder–decoder framework. The chapter closes with a discussion on challenges and future directions of deep learning-based MT.

Yang Liu, Jiajun Zhang

Chapter 7. Deep Learning in Question Answering

Abstract

Question answering (QA) is a challenging task in natural language processing. Recently, with the remarkable success of deep learning on many natural language processing tasks, including semantic and syntactic analysis, machine translation, relation extraction, etc., more and more efforts have also been devoted to the task of question answering. This chapter briefly introduces the recent advances in deep learning methods on two typical and popular question answering tasks. (1) Deep learning in question answering over knowledge base (KBQA) which mainly employs deep neural networks to understand the meaning of the questions and try to translate them into structured queries, or directly translate them into distributional semantic representations compared with candidate answers in the knowledge base. (2) Deep learning in machine comprehension (MC) which manages to construct an end-to-end paradigm based on novel neural networks for directly computing the deep semantic matching among question, answers and the given passage.

Kang Liu, Yansong Feng

Chapter 8. Deep Learning in Sentiment Analysis

Abstract

Sentiment analysis (also known as opinion mining) is an active research area in natural language processing. The task aims at identifying, extracting, and organizing sentiments from user-generated texts in social networks, blogs, or product reviews. Over the past two decades, many studies in the literature exploit machine learning approaches to solve sentiment analysis tasks from different perspectives. Since the performance of a machine learner heavily depends on the choices of data representation, many studies devote to building powerful feature extractor with domain expertise and careful engineering. Recently, deep learning approaches emerge as powerful computational models that discover intricate semantic representations of texts automatically from data without feature engineering. These approaches have improved the state of the art in many sentiment analysis tasks, including sentiment classification, opinion extraction, fine-grained sentiment analysis, etc. In this paper, we give an overview of the successful deep learning approaches sentiment analysis tasks at different levels.

Duyu Tang, Meishan Zhang

Chapter 9. Deep Learning in Social Computing

Abstract

The goal of social computing is to devise computational systems to learn mechanisms and principles to explain and understand the behaviors of each individual and collective teams, communities, and organizations. The unprecedented online data in social media provides a fruitful resource for this purpose. However, traditional techniques have a hard time in handling the complex and heterogeneous nature of social media for social computing. Fortunately, the recent revival and success of deep learning brings new opportunities and solutions to address these challenges. This chapter introduces the recent progress of deep learning on social computing in three aspects, namely user-generated content, social connections, and recommendation, which have covered most of the core elements and applications in social computing. Our focus lies in the discussions on how to adapt deep learning techniques to mainstream social computing tasks.

Xin Zhao, Chenliang Li

Chapter 10. Deep Learning in Natural Language Generation from Images

Abstract

Natural language generation from images, referred to as image or visual captioning also, is an emerging deep learning application that is in the intersection between computer vision and natural language processing. Image captioning also forms the technical foundation for many practical applications. The advances in deep learning technologies have created significant progress in this area in recent years. In this chapter, we review the key developments in image captioning and their impact in both research and industry deployment. Two major schemes developed for image captioning, both based on deep learning, are presented in detail. A number of examples of natural language descriptions of images produced by two state-of-the-art captioning systems are provided to illustrate the high quality of the systems’ outputs. Finally, recent research on generating stylistic natural language from images is reviewed.

Xiaodong He, Li Deng

Chapter 11. Epilogue: Frontiers of NLP in the Deep Learning Era

Abstract

In the first part of this epilogue, we summarize the book holistically from two perspectives. The first, task-centric perspective ties together and categories a wide range of NLP techniques discussed in book in terms of general machine learning paradigms. In this way, the majority of sections and chapters of the book can be naturally clustered into four classes: classification, sequence-based prediction, higher-order structured prediction, and sequential decision-making. The second, representation-centric perspective distills insight from holistically analyzed book chapters from cognitive science viewpoints and in terms of two basic types of natural language representations: symbolic and distributed representations. In the second part of the epilogue, we update the most recent progress on deep learning in NLP (mainly during the later part of 2017, not surveyed in earlier chapters). Based on our reviews of these rapid recent advances, we then enrich our earlier writing on the research frontiers of NLP in Chap. 1 by addressing future directions of exploiting compositionality of natural language for generalization, unsupervised and reinforcement learning for NLP and their intricate connections, meta-learning for NLP, and weak-sense and strong-sense interpretability for NLP systems based on deep learning.

Li Deng, Yang Liu

Backmatter

Titel: Deep Learning in Natural Language Processing
herausgegeben von: Dr. Li Deng
Yang Liu
Verlag: Springer Singapore
Electronic ISBN: 978-981-10-5209-5
Print ISBN: 978-981-10-5208-8
DOI: https://doi.org/10.1007/978-981-10-5209-5

Springer Professional

Über dieses Buch

Inhaltsverzeichnis

Frontmatter

Chapter 1. A Joint Introduction to Natural Language Processing and to Deep Learning

Chapter 2. Deep Learning in Conversational Language Understanding

Chapter 3. Deep Learning in Spoken and Text-Based Dialog Systems

Chapter 4. Deep Learning in Lexical Analysis and Parsing

Chapter 5. Deep Learning in Knowledge Graph

Chapter 6. Deep Learning in Machine Translation

Chapter 7. Deep Learning in Question Answering

Chapter 8. Deep Learning in Sentiment Analysis

Chapter 9. Deep Learning in Social Computing

Chapter 10. Deep Learning in Natural Language Generation from Images

Chapter 11. Epilogue: Frontiers of NLP in the Deep Learning Era

Backmatter

Premium Partner