Holographic Embeddings of Knowledge Graphs

Abstract

Learning embeddings of entities and relations is an efficient and versatile method to perform machine learning on relational data such as knowledge graphs. In this work, we propose holographic embeddings (HOLE) to learn compositional vector space representations of entire knowledge graphs. The proposed method is related to holographic models of associative memory in that it employs circular correlation to create compositional representations. By using correlation as the compositional operator, HOLE can capture rich interactions but simultaneously remains efficient to compute, easy to train, and scalable to very large datasets. Experimentally, we show that holographic embeddings are able to outperform state-ofthe-art methods for link prediction on knowledge graphs and relational learning benchmark datasets. Introduction Relations are a key concept in artificial intelligence and cognitive science. Many of the structures that humans impose on the world, such as logical reasoning, analogies, or taxonomies, are based on entities, concepts and their relationships. Hence, learning from and with relational knowledge representations has long been considered an important task in artificial intelligence (see e.g., Getoor and Taskar (2007); Muggleton (1991); Gentner (1983); Kemp et al. (2006); Xu et al. (2006); Richardson and Domingos (2006)). In this work we are concerned with learning from knowledge graphs (KGs), i.e., knowledge bases which model facts as instances of binary relations (e.g., bornIn(BarackObama, Hawaii)). This form of knowledge representation can be interpreted as a multigraph, where entities correspond to nodes, facts correspond to typed edges, and the type of an edge indicates the kind of the relation. Modern knowledge graphs such as YAGO (Suchanek, Kasneci, and Weikum, 2007), DBpedia (Auer et al., 2007), and Freebase (Bollacker et al., 2008) contain billions of facts about millions of entities and have found important applications in question answering, structured search, and digital assistants. Recently, vector space embeddings of knowledge graphs have received considerable attention, as they can be used to create statistical models of entire KGs, i.e., to predict the probability of any possible relation instance (edge) in the graph. Such models can be used to derive new knowledge from known facts (link prediction), to disambiguate entities (entity resolution), to extract taxonomies, and for probabilistic question answering (see e.g., (Nickel, Tresp, and Kriegel, 2011; Bordes et al., 2013; Krompaß, Nickel, and Tresp, 2014)). Furthermore, embeddings of KGs have been used to support machine reading and to assess the trustworthiness of web sites (Dong et al., 2014, 2015). However, existing embedding models that can capture rich interactions in relational data are often limited in their scalability. Vice versa, models that can be computed efficiently are often considerably less expressive. In this work, we approach learning from KGs within the framework of compositional vector space models. We introduce holographic embeddings (HOLE) which use the circular correlation of entity embeddings (vector representations) to create compositional representations of binary relational data. By using correlation as the compositional operator HOLE can capture rich interactions but simultaneously remains efficient to compute, easy to train, and scalable to very large datasets. As we will show experimentally, HOLE is able to outperform state-of-the-art embedding models on various benchmark datasets for learning from KGs. Compositional vector space models have also been considered in cognitive science and natural language processing, e.g., to model symbolic structures, to represent the semantic meaning of phrases, and as models for associative memory (see e.g., Smolensky (1990); Plate (1995); Mitchell and Lapata (2008); Socher et al. (2012)). In this work, we do not only draw inspiration from these models, but we will also highlight the connections of HOLE to holographic models of associative memory. Compositional Representations In this section we introduce compositional vector space models for KGs, the general learning setting, and related work. Let E denote the set of all entities and P the set of all relation types (predicates) in a domain. A binary relation is a subsetRp ⊆ E ×E of all pairs of entities (i.e., those pairs which are in a relation of type p). Higher-arity relations are defined analogously. The characteristic function φp : E × E → {±1} of a relation Rp indicates for each possible pair of entities whether they are part of Rp. We will denote (possible) relation instances as Rp(s, o), where s, o ∈ E denote the first and second argument of the asymmetric relationRp. We will refer to s, o as subject and object and toRp(s, o) as triples. Compositional vector space models provide an elegant way to learn the characteristic functions of the relations in a ar X iv :1 51 0. 04 93 5v 2 [ cs .A I] 7 D ec 2 01 5 knowledge graph, as they allow to cast the learning task as a problem of supervised representation learning. Here, we discuss models of the form Pr(φp(s, o) = 1|Θ) = σ(ηspo) = σ(rp (es ◦ eo)) (1) where rp ∈ Rr , ei ∈ Re are vector representations of relations and entities; σ(x) = 1/(1 + exp(−x)) denotes the logistic function; Θ = {ei} i=1 ∪ {rk} nr k=1 denotes the set of all embeddings; ◦ : Re × Re → Rp denotes the compositional operator which creates a composite vector representation for the pair (s, o) from the embeddings es, eo. We will discuss possible compositional operators below. Let xi ∈ P × E × E denote a triple, and yi ∈ {±1} denote its label. Given a dataset D = {(xi, yi)}i=1 of true and false relation instances, we then want to learn representations of entities and relations Θ that best explain D according to eq. (1). This can, for instance, be done by minimizing the (regularized) logistic loss

Extracted Key Phrases

5 Figures and Tables

0501001502014201520162017
Citations per Year

121 Citations

Semantic Scholar estimates that this publication has 121 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@inproceedings{Nickel2016HolographicEO, title={Holographic Embeddings of Knowledge Graphs}, author={Maximilian Nickel and Lorenzo Rosasco and Tomaso A. Poggio}, booktitle={AAAI}, year={2016} }