Distributed representation learning for knowledge graphs with entity descriptions

Miao Fan, Qiang Zhou, Thomas Fang Zheng, Ralph Grishman

Research output: Contribution to journalArticle

Abstract

Recent studies of knowledge representation attempt to project both entities and relations, which originally compose a high-dimensional and sparse knowledge graph, into a continuous low-dimensional space. One canonical approach . TransE [2] which represents entities and relations with vectors (embeddings), achieves leading performances solely with triplets, i.e. (head_entity, relation, tail_entity), in a knowledge base. The cutting-edge method . DKRL [23] extends . TransE via enhancing the embeddings with entity descriptions by means of deep neural network models. However, . DKRL requires extra space to store parameters of inner layers, and relies on more hyperparameters to be tuned. Therefore, we create a single-layer model which requests much fewer parameters. The model measures the probability of each triplet along with corresponding entity descriptions, and learns contextual embeddings of entities, relations and words in descriptions simultaneously, via maximizing the loglikelihood of the observed knowledge. We evaluate our model in the tasks of knowledge graph completion and entity type classification with two benchmark datasets: . FB500K and . EN15K, respectively. Experimental results demonstrate that the proposed model outperforms both . TransE and . DKRL, indicating that it is both efficient and effective in learning better distributed representations for knowledge bases.

Original languageEnglish (US)
JournalPattern Recognition Letters
DOIs
StateAccepted/In press - Apr 15 2016

Fingerprint

Knowledge representation
Deep neural networks

Keywords

  • Entity description
  • Entity type classification
  • Knowledge graph
  • Knowledge graph completion
  • Representation learning

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Artificial Intelligence

Cite this

Distributed representation learning for knowledge graphs with entity descriptions. / Fan, Miao; Zhou, Qiang; Zheng, Thomas Fang; Grishman, Ralph.

In: Pattern Recognition Letters, 15.04.2016.

Research output: Contribution to journalArticle

@article{647acfdb4ce0473fb5750973cc35ec64,
title = "Distributed representation learning for knowledge graphs with entity descriptions",
abstract = "Recent studies of knowledge representation attempt to project both entities and relations, which originally compose a high-dimensional and sparse knowledge graph, into a continuous low-dimensional space. One canonical approach . TransE [2] which represents entities and relations with vectors (embeddings), achieves leading performances solely with triplets, i.e. (head_entity, relation, tail_entity), in a knowledge base. The cutting-edge method . DKRL [23] extends . TransE via enhancing the embeddings with entity descriptions by means of deep neural network models. However, . DKRL requires extra space to store parameters of inner layers, and relies on more hyperparameters to be tuned. Therefore, we create a single-layer model which requests much fewer parameters. The model measures the probability of each triplet along with corresponding entity descriptions, and learns contextual embeddings of entities, relations and words in descriptions simultaneously, via maximizing the loglikelihood of the observed knowledge. We evaluate our model in the tasks of knowledge graph completion and entity type classification with two benchmark datasets: . FB500K and . EN15K, respectively. Experimental results demonstrate that the proposed model outperforms both . TransE and . DKRL, indicating that it is both efficient and effective in learning better distributed representations for knowledge bases.",
keywords = "Entity description, Entity type classification, Knowledge graph, Knowledge graph completion, Representation learning",
author = "Miao Fan and Qiang Zhou and Zheng, {Thomas Fang} and Ralph Grishman",
year = "2016",
month = "4",
day = "15",
doi = "10.1016/j.patrec.2016.09.005",
language = "English (US)",
journal = "Pattern Recognition Letters",
issn = "0167-8655",
publisher = "Elsevier",

}

TY - JOUR

T1 - Distributed representation learning for knowledge graphs with entity descriptions

AU - Fan, Miao

AU - Zhou, Qiang

AU - Zheng, Thomas Fang

AU - Grishman, Ralph

PY - 2016/4/15

Y1 - 2016/4/15

N2 - Recent studies of knowledge representation attempt to project both entities and relations, which originally compose a high-dimensional and sparse knowledge graph, into a continuous low-dimensional space. One canonical approach . TransE [2] which represents entities and relations with vectors (embeddings), achieves leading performances solely with triplets, i.e. (head_entity, relation, tail_entity), in a knowledge base. The cutting-edge method . DKRL [23] extends . TransE via enhancing the embeddings with entity descriptions by means of deep neural network models. However, . DKRL requires extra space to store parameters of inner layers, and relies on more hyperparameters to be tuned. Therefore, we create a single-layer model which requests much fewer parameters. The model measures the probability of each triplet along with corresponding entity descriptions, and learns contextual embeddings of entities, relations and words in descriptions simultaneously, via maximizing the loglikelihood of the observed knowledge. We evaluate our model in the tasks of knowledge graph completion and entity type classification with two benchmark datasets: . FB500K and . EN15K, respectively. Experimental results demonstrate that the proposed model outperforms both . TransE and . DKRL, indicating that it is both efficient and effective in learning better distributed representations for knowledge bases.

AB - Recent studies of knowledge representation attempt to project both entities and relations, which originally compose a high-dimensional and sparse knowledge graph, into a continuous low-dimensional space. One canonical approach . TransE [2] which represents entities and relations with vectors (embeddings), achieves leading performances solely with triplets, i.e. (head_entity, relation, tail_entity), in a knowledge base. The cutting-edge method . DKRL [23] extends . TransE via enhancing the embeddings with entity descriptions by means of deep neural network models. However, . DKRL requires extra space to store parameters of inner layers, and relies on more hyperparameters to be tuned. Therefore, we create a single-layer model which requests much fewer parameters. The model measures the probability of each triplet along with corresponding entity descriptions, and learns contextual embeddings of entities, relations and words in descriptions simultaneously, via maximizing the loglikelihood of the observed knowledge. We evaluate our model in the tasks of knowledge graph completion and entity type classification with two benchmark datasets: . FB500K and . EN15K, respectively. Experimental results demonstrate that the proposed model outperforms both . TransE and . DKRL, indicating that it is both efficient and effective in learning better distributed representations for knowledge bases.

KW - Entity description

KW - Entity type classification

KW - Knowledge graph

KW - Knowledge graph completion

KW - Representation learning

UR - http://www.scopus.com/inward/record.url?scp=85000839169&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85000839169&partnerID=8YFLogxK

U2 - 10.1016/j.patrec.2016.09.005

DO - 10.1016/j.patrec.2016.09.005

M3 - Article

AN - SCOPUS:85000839169

JO - Pattern Recognition Letters

JF - Pattern Recognition Letters

SN - 0167-8655

ER -