Advertisement
Published in:
2019 | OriginalPaper | Chapter
Enhancing Unsupervised Pretraining with External Knowledge for Natural Language Inference
Authors : Xiaoyu Yang, Xiaodan Zhu, Huasha Zhao, Qiong Zhang, Yufei Feng
Published in: Advances in Artificial Intelligence
Publisher: Springer International Publishing
Activate our intelligent search to find suitable subject content or patents.
Select sections of text to find matching patents with Artificial Intelligence. powered by
Select sections of text to find additional relevant content using AI-assisted search. powered by
Abstract
Unsupervised pretraining such as BERT (Bidirectional Encoder Representations from Transformers) [2] represents the most recent advance on learning representation for natural language, which has helped achieve leading performance on many natural language processing problems. Although BERT can leverage large corpora, we assume it cannot learn all needed semantics and knowledge for natural language inference (NLI). In this paper, we leverage human-authorized external knowledge to further improve BERT, and our results show that BERT, the current state-of-the-art pretraining framework, can benefit from external knowledge.