Top

Journal of Intelligent Information Systems

Published in:

27-01-2020

Improve the translational distance models for knowledge graph embedding

Authors: Siheng Zhang, Zhengya Sun, Wensheng Zhang

Published in: Journal of Intelligent Information Systems | Issue 3/2020

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Knowledge graph embedding techniques can be roughly divided into two mainstream, translational distance models and semantic matching models. Though intuitive, translational distance models fail to deal with the circle structure and hierarchical structure in knowledge graphs. In this paper, we propose a general learning framework named TransX-pa, which takes various models (TransE, TransR, TransH and TransD) into consideration. From this unified viewpoint, we analyse the learning bottlenecks are: (i) the common assumption that the inverse of a relation r is modelled as its opposite − r; and (ii) the failure to capture the rich interactions between entities and relations. Correspondingly, we introduce position-aware embeddings and self-attention blocks, and show that they can be adapted to various translational distance models. Experiments are conducted on different datasets extracted from real-world knowledge graphs Freebase and WordNet in the tasks of both triplet classification and link prediction. The results show that our approach makes a great improvement, showing a better, or comparable, performance with state-of-the-art methods.

previous article A survey on intention analysis: successful approaches and open challenges

next article Machine learning for music genre: multifaceted review and experimentation with audioset

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

https://github.com/quark0/ANALOGY

https://github.com/ttrouill/complex

https://github.com/TimDettmers/ConvE

https://github.com/thunlp/OpenKE

https://www.cs.princeton.edu/∼danqic/data/nips13-dataset.tar.bz2

Nickel, M., Murphy, K., Tresp, V., Gabrilovich, E. (2016). A review of relational machine learning for knowledge graphs. Proceedings of the IEEE, 104, 11–33.CrossRef

Wang, Q., Mao, Z., Wang, B., Guo, L. (2017). Knowledge graph embedding: a survey of approaches and applications. IEEE Transactions on Knowledge and Data Engineering, 29, 2724–2743.CrossRef

Toutanova, K., & Chen, D. (2015). Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality (pp. 57–66).

Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. Advances in Neural Information Processing Systems, 26, 2787–2795.

Wang, Z., Zhang, J., Feng, J., Chen, Z. (2014). Knowledge graph embedding by translating on hyperplanes. In Proceedings of the twenty-eighth AAAI conference on artificial intelligence (pp. 1112–1119).

Lin, Y., Liu, Z., Zhu, X., Zhu, X., Zhu, X. (2015). Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the twenty-ninth AAAI conference on artificial intelligence (pp. 2181–2187).

Ji, G., He, S., Xu, L., Liu, K., Zhao, J. (2015). Knowledge graph embedding via dynamic mapping matrix. In Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing, (Vol. 1: Long Papers pp. 687–696).

Ji, G., Liu, K., He, S., Zhao, J. (2016). Knowledge graph completion with adaptive sparse transfer matrix. In Proceedings of the thirtieth AAAI conference on artificial intelligence (pp. 985–991).

Fan, M., Zhou, Q., Chang, E., Zheng, T.F. (2014). Transition-based knowledge graph embedding with relational mapping properties. In Proceedings of the 28th Pacific Asia Conference on Language, Information and Computing (pp. 328–337).

Xiao, H., Huang, M., Zhu, X. (2016). From one point to a manifold: knowledge graph embedding for precise link prediction. In Proceedings of the 25th international joint conference on artificial intelligence (pp. 1315–1321).

He, S., Liu, K., Ji, G., Zhao, J. (2015). Learning to represent knowledge graphs with Gaussian embedding. In Proceedings of the 24th ACM international on conference on information and knowledge management (pp. 623–632).

Zhang, W. (2017). Knowledge graph embedding with diversity of structures. In Proceedings of the 26th international conference on world wide web companion (pp. 747–753).

Bordes, A., Weston, J., Collobert, R., Bengio, Y. (2011). Learning structured embeddings of knowledge bases. In Proceedings of the twenty-fifth AAAI conference on artificial intelligence (pp. 301–306).

Bordes, A., Glorot, X., Weston, J., Bengio, Y. (2014). A semantic matching energy function for learning with multi-relational data: application to word-sense disambiguation. Machine Learning, 94(2), 233–259.MathSciNetCrossRef

Yang, B., Yih, W. T., He, X., Gao, J., Deng, L. (2015). Embedding entities and relations for learning and inference in knowledge bases. In Proceedings of the International Conference on Learning Representations.

Nickel, M., Rosasco, L., Poggio, T. (2016). Holographic embeddings of knowledge graphs. In Proceedings of the thirtieth AAAI conference on artificial intelligence (pp. 1955–1961).

Théo, T., Johannes, W., Sebastian, R., Eric, G., Guillaume, B. (2016). Complex embeddings for simple link prediction. In International Conference on Machine Learning (pp. 2071–2080).

Hayashi, K., & Shimbo, M. (2017). On the equivalence of holographic and complex embeddings for link prediction. In Proceedings of the 55th annual meeting of the association for computational linguistics (pp. 554–559).

Dettmers, T., Minervini, P., Stenetorp, P., Ridedel, S. (2018). Convolutional 2D knowledge graph embeddings. In Proceedings of the thirty-second AAAI conference on artificial intelligence.

Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., Liu, S. (2015). Modeling relation paths for representation learning of knowledge bases. In Proceedings of the conference on empirical methods in natural language processing (pp. 705–714).

Socher, R., Chen, D., Manning, C., Chen, D., Ng, A. (2013). Reasoning with neural tensor networks for knowledge base completion. In Proceedings of the 26th international conference on neural information processing systems (pp. 926–934).

Wang, Z., & Li, J. (2016). Text-enhanced representation learning for knowledge graph. In Proceedings of the 25th international joint conferences on artificial intelligence (pp. 1293–1299).

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., Polosukhin, I. (2017). Attention is all you need. In Proceedings of the 30th international conference on neural information processing systems (pp. 5998–6008).

Bengio, Y., Courville, A., Vincent, P. (2013). Representation learning: a review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35, 1798–1828.CrossRef

Glorot, X., & Bengio, Y. (2010). Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics (pp. 249–256).

Miller, G. A. (2005). Wordnet: a lexical database for english. Communications of the Association for Computing Machinery, 38, 39–41.CrossRef

Bollacker, K., Evans, C., Paritosh, P., Sturge, T., Taylor, J. (2008). Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 ACM SIGMOD international conference on Management of data (pp. 1247–1250).

Liu, H., Wu, Y., Yang, Y. (2017). Analogical inference for multi-relational embeddings. In Proceedings of the 34th international conference on machine learning (pp. 2168–2178).

Han, X., Cao, S., Lv, X., Lin, Y., Liu, Z., Sun, M., Li, J. (2018). OpenKE: an open toolkit for knowledge embedding. In Proceedings of the 2018 conference on empirical methods in natural language processing: system demonstrations (pp. 139–144).

Akrami, F., Guo, L., Hu, W., Li, C. (2018). Re-evaluating embedding-based knowledge graph completion methods. In Proceedings of the 27th ACM international conference on information and knowledge management (pp. 1779–1782).

Toutanova, K., Chen, D., Pantel, P., Poon, H., Choudhury, P., Gamon, M. (2015). Representing text for joint embedding of text and knowledge bases. In Proceedings of the 2015 conference on empirical methods in natural language processing (pp. 1499–1509).

Nguyen, D. Q., Nguyen, T. D., Nguyen, D. Q., Phung, D. (2018). A novel embedding model for knowledge base completion based on convolutional neural network. In Proceedings of Annual Conference of the North American Chapter of the Association for Computational Linguistics.

Duchi, J., Hazan, E., Singer, Y. (2011). Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12, 2121–2159.MathSciNetMATH

Demsar, J. (2006). Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7, 1–30.MathSciNetMATH

Wang, Z., Zhang J., Feng J., Chen Z. (2014). Knowledge graph and text jointly embedding. In Proceedings of the 2014 conference on empirical methods in natural language processing (pp. 1591–1601).

Title: Improve the translational distance models for knowledge graph embedding
Authors: Siheng Zhang
Zhengya Sun
Wensheng Zhang
Publication date: 27-01-2020
Publisher: Springer US
Published in: Journal of Intelligent Information Systems / Issue 3/2020
Print ISSN: 0925-9902
Electronic ISSN: 1573-7675
DOI: https://doi.org/10.1007/s10844-019-00592-7

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 3/2020

System for monitoring road slippery based on CCTV cameras and convolutional neural networks

Automated conversion from natural language query to SPARQL query

Machine learning for music genre: multifaceted review and experimentation with audioset

A minimum spanning tree based partitioning and merging technique for clustering heterogeneous data sets

Compacting frequent star patterns in RDF graphs

A survey on intention analysis: successful approaches and open challenges

Premium Partner