Knowledge graph embedding with shared latent semantic units. (English) Zbl 1521.68177
Summary: Knowledge graph embedding (KGE) aims to project both entities and relations into a continuous low-dimensional space. However, for a given knowledge graph (KG), only a small number of entities and relations occur many times, while the vast majority of entities and relations occur less frequently. This data sparsity problem has largely been ignored by most of the existing KGE models. To this end, in this paper, we propose a general technique to enable knowledge transfer among semantically similar entities or relations. Specifically, we define latent semantic units (LSUs), which are the sub-components of entity and relation embeddings. Semantically similar entities or relations are supposed to share the same LSUs, and thus knowledge can be transferred among entities or relations. Finally, extensive experiments show that the proposed technique is able to enhance existing KGE models and can provide better representations of KGs.
References:
| [1] | Bollacker, K., Evans, C., Paritosh, P., Sturge, T., & Taylor, J. (2008). Freebase: a collaboratively created graph database for structuring human knowledge. In SIGMOD (pp. 1247-1250). |
| [2] | Bordes, A., Usunier, N., Garcia-Duran, A., Weston, J., & Yakhnenko, O. (2013). Translating embeddings for modeling multi-relational data. In NIPS (pp. 2787-2795). |
| [3] | Dalton, J.; Dietz, L.; Allan, J., Entity query feature expansion using knowledge base links, (Proceedings of the 37th international ACM SIGIR conference on research & development in information retrieval (2014), ACM), 365-374 |
| [4] | Dettmers, T., Minervini, P., Stenetorp, P., & Riedel, S. (2018). Convolutional 2d knowledge graph embeddings. In AAAI. |
| [5] | Feng, J., Huang, M., Zhao, L., Yang, Y., & Zhu, X. (2018). Reinforcement learning for relation classification from noisy data. In AAAI. |
| [6] | Ferrucci, D.; Brown, E.; Chu-Carroll, J.; Fan, J.; Gondek, D.; Kalyanpur, A. A., Building Watson: An overview of the DeepQA project, AI Magazine, 31, 3, 59-79 (2010) |
| [7] | Ji, G., Liu, K., He, S., & Zhao, J. (2016). Knowledge graph completion with adaptive sparse transfer matrix. In AAAI (pp. 985-991). |
| [8] | Jiang, J. (2009). Multi-task transfer learning for weakly-supervised relation extraction. In ACL (pp. 1012-1020). |
| [9] | Kazemi, S. M., & Poole, D. (2018). SimplE embedding for link prediction in knowledge graphs. In NIPS (pp. 4289-4300). |
| [10] | Kingma, D.; Ba, J., Adam: A method for stochastic optimization, Computer Science (2014) |
| [11] | Lee, H., Battle, A., Raina, R., & Ng, A. Y. (2007). Efficient sparse coding algorithms. In NIPS (pp. 801-808). |
| [12] | Lin, Y., Liu, Z., Luan, H., Sun, M., Rao, S., & Liu, S. (2015). Modeling relation paths for representation learning of knowledge bases. In EMNLP. |
| [13] | Lin, Y., Liu, Z., Sun, M., Liu, Y., & Zhu, X. (2015) Learning entity and relation embeddings for knowledge graph completion. In AAAI (pp. 2181-2187). |
| [14] | Mikolov, T., Sutskever, I., Chen, K., Corrado, G. S., & Dean, J. Distributed representations of words and phrases and their compositionality. In NIPS (pp. 3111-3119). |
| [15] | Miller, G. A., Wordnet: a lexical database for english, Communications of the ACM, 38, 11, 39-41 (1995) |
| [16] | Mintz, M., Bills, S., Snow, R., & Jurafsky, D. (2009). Distant supervision for relation extraction without labeled data. In ACL-IJCNLP (pp. 1003-1011). |
| [17] | Nathani, D., Chauhan, J., Sharma, C., & Kaul, M. (2019). Learning attention-based embeddings for relation prediction in knowledge graphs. In ACL. |
| [18] | Nguyen, K., Daumé III., H., & Boyd-Graber, J. (2017). Reinforcement learning for bandit neural machine translation with simulated human feedback. In Proceedings of the 2017 conference on empirical methods in natural language processing (pp. 1464-1474). |
| [19] | Nguyen, D. Q., Nguyen, T. D., Nguyen, D. Q., & Phung, D. (2018). A novel embedding model for knowledge base completion based on convolutional neural network. In NAACL-HLT. (pp. 327-333). |
| [20] | Qin, P., Weiran, X. U., & Wang, W. Y. (2018). Robust distant supervision relation extraction via deep reinforcement learning. In ACL (pp. 2137-2147). |
| [21] | Schlichtkrull, M.; Kipf, T. N.; Bloem, P.; Van Den Berg, R.; Titov, I., Modeling relational data with graph convolutional networks, (European semantic web conference (2018), Springer), 593-607 |
| [22] | Socher, R., Chen, D., Manning, C. D., & Ng, A. (2013). Reasoning with neural tensor networks for knowledge base completion. In NIPS (pp. 926-934). |
| [23] | Suchanek, F. M.; Kasneci, G.; Weikum, G., Yago: a core of semantic knowledge, (Proceedings of the 16th international conference on world wide web (2007), ACM), 697-706 |
| [24] | Toutanova, K., & Chen, D. (2015). Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality (pp. 57-66). |
| [25] | Trouillon, T., Welbl, J., Riedel, S., Gaussier, É., & Bouchard, G. (2016). Complex embeddings for simple link prediction. In ICML (pp. 2071-2080). |
| [26] | Wang, Z., Zhang, J., Feng, J., & Chen, Z. (2014). Knowledge graph embedding by translating on hyperplanes. In AAAI (pp. 1112-1119). |
| [27] | Williams, R. J., Simple statistical gradient-following algorithms for connectionist reinforcement learning, Machine Learning, 8, 3-4, 229-256 (1992) · Zbl 0772.68076 |
| [28] | Xiao, H., Huang, M., Meng, L., & Zhu, X. (2017). SSP: semantic space projection for knowledge graph embedding with text descriptions. In AAAI (pp. 3104-3110). |
| [29] | Xie, R., Liu, Z., Jia, J., Luan, H., & Sun, M. (2016). Representation learning of knowledge graphs with entity descriptions. In AAAI (pp. 2659-2665). |
| [30] | Xie, Q., Ma, X., Dai, Z., & Hovy, E. (2017). An interpretable knowledge transfer model for knowledge base completion. In ACL (pp. 50-962). |
| [31] | Xiong, W., Hoang, T., & Wang, W. Y. (2017). DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning. In EMNLP. |
| [32] | Xiong, C., Zhong, V., & Socher, R. (2018). Dcn+: Mixed objective and deep residual coattention for question answering. In ICLR. |
| [33] | Yang, Z., Salakhutdinov, R., & Cohen, W. W. (2017). Transfer learning for sequence tagging with hierarchical recurrent networks. In ICLR. |
| [34] | Yang, B., Yih, W.-t., He, X., Gao, J., & Deng, L. (2015). Embedding entities and relations for learning and inference in knowledge bases. In ICLR. |
| [35] | Zhang, Z., Zhuang, F., Qu, M., Lin, F., & He, Q. (2018). Knowledge graph embedding with hierarchical relation structure. In EMNLP (pp. 3198-3207). |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
