nach oben

Discover Computing

Erschienen in:

25.10.2018 | Knowledge Graphs and Semantics in Text Analysis and Retrieval

Neural variational entity set expansion for automatically populated knowledge graphs

verfasst von: Pushpendre Rastogi, Adam Poliak, Vince Lyzinski, Benjamin Van Durme

Erschienen in: Discover Computing | Ausgabe 3-4/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

We propose Neural variational set expansion to extract actionable information from a noisy knowledge graph (KG) and propose a general approach for increasing the interpretability of recommendation systems. We demonstrate the usefulness of applying a variational autoencoder to the Entity set expansion task based on a realistic automatically generated KG.

Vorheriger Artikel Special issue on knowledge graphs and semantics in text analysis and retrieval

Nächster Artikel Automated assessment of knowledge hierarchy evolution: comparing directed acyclic graphs

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Nur mit Berechtigung zugänglich

We refer to the items in the seed as entities but they can also be referred to as items or elements.

https://nyti.ms/2hR1S15.

We ignore confidence scores that entity linking systems often assign to a link because confidence scores will prevent us from using a multinomial distribution to model a document as a bag-of-words.

Lucene replaced tf-idf with BM25 as its default algorithm: https://issues.apache.org/jira/browse/LUCENE-6789.

Converting entity mentions to entity IDs allows us to overcome issues related to embedding multi-word expressions as explained in Poliak et al. (2017).

Our generative model is inspired by Miao et al. (2016)’s NVDM. They assume that a single latent variable generates only one observation, but we posit that the same latent variable z generates all observations in \({\mathcal{Q}}\).

This is a generalization of Bouchacourt et al. (2017) combining variational approximations of posterior distributions since the product of gaussians is a Gaussian distribution.

Also notice that the POE approach recommends adding the outputs of the neural networks which is different than concatenating the features for all x in \({\mathcal{Q}}\) or naively adding the inputs of the neural network. (Appendix 2) gives more details.

Recently, Zaheer et al. (2017) gave a theorem that any permutation invariant function of sets must be representable as the function of a sum of features of elements of the set. We note that our POE approximation also has a similar form and is permutation invariant.

More informally, we remove the plates from Fig. 2.

Tinkerbell constructed a KG from LDC2017E25 that contains 30K English documents. Half of them are from online forums and the other half from Reuters and NYT. We focused on the 77,845 entities from English documents appearing in 344,735 sentences. 25,149 entities were also linked to DBPedia.

https://tac.nist.gov/2017/KGP/data.html.

The mean is 4.43, the standard deviation is 29.19, the minimum number of sentences for an entity is 1, the maximum number of sentences is 4638, and the median is 1 (44,317 entities).

https://bitbucket.org/yoavgo/word2vecf, https://github.com/mickeystroller/SetExpan.

Training NVSE on 1 Tesla K80 using the Adam optimizer with learning rate \(5e^{{-}5}\) and minibatch size 64 took 12 h.

Figure 3 in (Appendix 5) illustrates the AMT interface.

Al-Badrashiny, M., Bolton, J., Tejavsi Chaganty, A., Clark, K., Harman, C., Huang, L., Lamm, M., Lei, J., Lu, D., Pan, X., Paranjape, A., Pavlick, E., Peng, H., Qi, P., Rastogi, P., See, A., Sun, K., Thomas, M., Tsai, C. T., Wu, H., Zhang, B., Callison-Burch, C., Cardie, C., Ji, H., Manning, C., Muresan, S., Rambow, O. C., Roth, D., Sammons, M., & Van Durme, B. (2017). Tinkerbell: Cross-lingual cold-start knowledge base construction. In Text analysis conference (TAC).

Altszyler, E., Sigman, M., & Slezak, D. F. (2016). Comparative study of LSA vs word2vec embeddings in small corpora: a case study in dreams database. CoRR http://arxiv.org/abs/1610.01520.

Auer, S., Bizer, C., Kobilarov, G., Lehmann, J., Cyganiak, R., & Ives, Z. (2007). Dbpedia: A nucleus for a web of open data. The semantic web pp. 722–735.

Balog K. (2009). Overview of the trec 2009 entity track. In Proc. TREC2009.

Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of machine Learning research, 3, 993–1022.MATH

Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008). Freebase: A collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on management of data, New York, NY: ACM.

Bouchacourt, D., Tomioka, R., & Nowozin, S. (2017). Multi-level variational autoencoder: Learning disentangled representations from grouped observations. arXiv preprint http://arxiv.org/abs/1705.08841.

Demartini, G., Iofciu, T., & De Vries, A. P. (2010). Overview of the inex 2009 entity ranking track. In Proceedings of the focused retrieval and evaluation, and 8th international conference on initiative for the evaluation of XML retrieval. INEX’09, (pp. 254–264). Berlin, Heidelberg: Springer.

Ghahramani, Z., & Heller, K. A. (2005). Bayesian sets. In Proceedings of the 18th international conference on neural information processing systems. NIPS’05, (pp. 435–442). Cambridge, MA, USA: MIT Press.

Gottipati, S., & Jiang, J. (2011). Linking entities to a knowledge base with query expansion. In Proceedings of the conference on empirical methods in natural language processing, association for computational linguistics,(pp. 804–813).

He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T. S. (2017). Neural collaborative filtering. In Proceedings of the 26th international conference on World Wide Web. WWW ’17, Republic and Canton of Geneva, Switzerland, international World Wide Web conferences steering Committee, (pp. 173–182).

He, Y., & Grishman, R. (2015). Ice: Rapid information extraction customization for nlp novices. In Proceedings of the 2015 conference of the North American chapter of the association for computational linguistics: Demonstrations, Denver, Colorado, Association for computational linguistics, (pp. 31–35).

He, Y., & Xin, D. (2011). Seisa: set expansion by iterative similarity aggregation. In Proceedings of the 20th international conference on World wide web, (pp. 427–436). ACM.

Hinton, G. E. (1999). Products of experts. In 1999 ninth international conference on artificial neural networks ICANN 99. (Conf. Publ. No. 470), (Vol 1, pp. 1–6).

Jebara, T., Kondor, R., & Howard, A. (2004). Probability product kernels. Journal of Machine Learning Research, 5, 819–844.MathSciNetMATH

Kingma, D. P., & Welling, M. (2013). Auto-encoding variational bayes. arXiv preprint http://arxiv.org/abs/1312.6114.

Kipf, T. N., & Welling, M. (2017). Semi-supervised classification with graph convolutional networks. In Proceedings of ICLR.

Lang, J., & Henderson, J. (2013). Graph-based seed set expansion for relation extraction using random walk hitting times. In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, Atlanta, Georgia, Association for computational linguistics, (pp. 772–776).

Lee, W., Song, K., & Moon, I. C. (2017). Augmented variational autoencoders for collaborative filtering with auxiliary information. In ACM conference on information and knowledge management, Number 6. ACM. https://doi.org/10.475/1234.

Levy, O., & Goldberg, Y. (2014). Dependency-based word embeddings. In Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: short papers), Baltimore, Maryland: Association for computational linguistics, (pp. 302–308).

Lin, D. (1998). Automatic retrieval and clustering of similar words. In Proceedings of the 17th international conference on computational linguistics-Volume 2, Association for computational linguistics, (pp. 768–774).

Manning, C. D., Raghavan, P., & Schtze, H. (2008). Introduction to information retrieval. Cambridge: Cambridge University Press.CrossRefMATH

Metzger, S., Schenkel, R., & Sydow, M. (2014). Aspect-based similar entity search in semantic knowledge graphs with diversity-awareness and relaxation. In Web Intelligence (WI) and intelligent agent technologies (IAT), 2014 IEEE/WIC/ACM international joint conferences on, (pp. 60–69). IEEE.

Miao, Y., Yu, L., & Blunsom, P. (2016). Neural variational inference for text processing. In International conference on machine learning, (pp. 1727–1736).

Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Proceedings of the 26th international conference on neural information processing systems—Volume 2. NIPS’13, (pp. 3111–3119). USA: Curran Associates Inc.

Mitra, B., & Craswell, N. (2017). Neural models for information retrieval. ArXiv e-prints (May 2017).

Pantel, P., Crestan, E., Borkovsky, A., Popescu, A. M., & Vyas, V. (2009). Web-scale distributional similarity and entity set expansion. In Proceedings of the 2009 conference on empirical methods in natural language processing, Singapore, Association for Computational Linguistics, (pp. 938–947).

Paşca, M., & Van Durme, B. (2007). What you seek is what you get: Extraction of class attributes from query logs. In IJCAI.

Paşca, M., & Van Durme, B. (2008). Weakly-supervised acquisition of open-domain classes and class attributes from web documents and query logs. In Proceedings of ACL-08: HLT, (pp. 19–27).

Poliak, A., Rastogi, P., Martin, M. P., & Van Durme, B. (2017). Efficient, compositional, order-sensitive n-gram embeddings. In Proceedings of the 15th conference of the European chapter of the association for computational linguistics: Volume 2, short papers, Valencia, Spain, Association for Computational Linguistics, (pp. 503–508).

Pujara, J., Augustine, E., & Getoor, L. (2017). Sparsity and noise: Where knowledge graph embeddings fall short. In Proceedings of the 2017 conference on empirical methods in natural language processing, Copenhagen, Denmark, Association for computational linguistics, (pp. 1751–1756).

Rastogi, P., Lyzinski, V., & Van Durme, B. (2017). Vertex nomination on the cold start knowledge graph. Human Language Technology Center of Excellence: Technical report.

Řehůřek, R., & Sojka, P. (2010). Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 workshop on new challenges for NLP frameworks, Valletta, Malta, ELRA, (pp. 45–50).

Rezende, D. J., Mohamed, S., & Wierstra, D. (2014). Stochastic backpropagation and approximate inference in deep generative models. arXiv preprint arXiv:1401.4082.

Robertson, S., & Zaragoza, H. (2009). The probabilistic relevance framework: Bm25 and beyond. Foundations Trends in Information Retrieval, 3(4), 333–389.CrossRef

Sadamitsu, K., Saito, K., Imamura, K., & Kikui, G. (2011). Entity set expansion using topic information. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, Portland, Oregon, USA, Association for computational linguistics, (pp. 726–731).

Shen, J., Wu, Z., Lei, D., Shang, J., Ren, X., & Han, J. (2017). Setexpan: Corpus-based set expansion via context feature selection and rank ensemble. In M. Ceci, J. Hollmén, L. Todorovski, C. Vens, & S. Džeroski (Eds.), Machine learning and knowledge discovery in databases (pp. 288–304). Cham: Springer International Publishing.CrossRef

Sønderby, C. K., Raiko, T., Maaløe, L., Sønderby, S. K., & Winther, O. (2016). Ladder variational autoencoders. In Proceedings of the 30th international conference on neural information processing systems. NIPS’16, (pp. 3738–3746). USA: Curran Associates Inc.

Tong, S., & Dean, J. (2008). System and methods for automatically creating lists (March 25 2008) US Patent 7,350,187.

Vartak, M., Thiagarajan, A., Miranda, C., Bratman, J., & Larochelle, H. (2017). A meta-learning perspective on cold-start recommendations for items. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems (Vol. 30, pp. 6907–6917). Red Hook: Curran Associates Inc.

Wang, R. C., & Cohen, W. W. (2007). Language-independent set expansion of named entities using the web. In Seventh IEEE international conference on data mining (ICDM 2007), (pp. 342–350).

Wang, R. C., & Cohen, W. W. (2008). Iterative set expansion of named entities using the web. In 2008 eighth IEEE international conference on data mining, (pp. 1091–1096).

Wang, R. C., & Cohen, W. W. (2009). Automatic set instance extraction using the web. In Proceedings of the joint conference of the 47th annual meeting of the ACL and the 4th international joint conference on natural language processing of the AFNLP: Volume 1-Volume 1, Association for computational linguistics, )(pp. 441–449).

Wang, R. C., Schlaefer, N., Cohen, W. W., & Nyberg, E. (2008). Automatic set expansion for list question answering. In Proceedings of the 2008 conference on empirical methods in natural language processing, Honolulu, Hawaii, Association for computational linguistics, (pp. 947–954).

Zaheer, M., Kottur, S., Ravanbakhsh, S., Poczos, B., Salakhutdinov, R. R., & Smola, A. J. (2017). Deep sets. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, & R. Garnett (Eds.), Advances in neural information processing systems (Vol. 30, pp. 3394–3404). Red Hook: Curran Associates Inc.

Zaidan, O., Eisner, J., & Piatko, C. (2007). Using annotator rationales to improve machine learning for text categorization. In Human language technologies 2007: The conference of the North American chapter of the association for computational linguistics; proceedings of the main conference, (pp. 260–267).

Zhang, X., Chen, Y., Chen, J., Du, X., Wang, K., & Wen, J. R. (2017). Entity set expansion via knowledge graphs. In Proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval. SIGIR ’17, (pp. 1101–1104). New York, NY: ACM.

Zheng, Y., Shi, C., Cao, X., Li, X., & Wu, B. (2017). Entity set expansion with meta path in knowledge graph. In Pacific-Asia conference on knowledge discovery and data mining, (pp. 317–329). Springer.

Titel: Neural variational entity set expansion for automatically populated knowledge graphs
verfasst von: Pushpendre Rastogi
Adam Poliak
Vince Lyzinski
Benjamin Van Durme
Publikationsdatum: 25.10.2018
Verlag: Springer Netherlands
Erschienen in: Discover Computing / Ausgabe 3-4/2019
Print ISSN: 2948-2984
Elektronische ISSN: 2948-2992
DOI: https://doi.org/10.1007/s10791-018-9342-1

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 3-4/2019

Special issue on knowledge graphs and semantics in text analysis and retrieval

Overcoming low-utility facets for complex answer retrieval

Automated assessment of knowledge hierarchy evolution: comparing directed acyclic graphs

Identifying and exploiting target entity type information for ad hoc entity retrieval

Neural architecture for question answering using a knowledge graph and web corpus

Payoffs and pitfalls in using knowledge-bases for consumer health search