Mitigating social bias in knowledge graph embeddings

{"value":"Question-answering systems frequently rely on knowledge graphs, large collections of facts about real-world entities (people, organizations, countries, etc.). To make use of the information in knowledge graphs, machine learning models often employ knowledge graph embeddings, vector representations of the entities in the graphs. \n\nA potential problem with this approach is that the distributions of data in knowledge graphs reflect current and historical social biases. For instance, most knowledge graphs include more male entities than female with the profession “banker”, or more “white American” entities than “African-American” entities with the profession “ballet dancer”. \n\nIf knowledge graph embeddings end up encoding these biases, so will the question-answering systems that use them. If a little girl talking to a chatbot asks, “What should I be when I grow up?”, a biased embedding might rule out possible answers that are predominantly associated with men in the knowledge graph. For some professions — “baritone”, for instance — that may be fine. But in other cases, the biases may be relics of a less egalitarian past.\n\n\n\nA two-dimensional representation of our method for measuring bias in knowledge graph embeddings. In each diagram, the blue dots labeled person1 indicate the shift in an embedding as we tune its parameters. The orange arrows represent relation vectors and the orange dots the resulting sums. As we shift the gender relation toward maleness, the profession relation shifts away from nurse and closer to doctor.\n\nCREDIT: GLYNIS CONDON\n\nEarlier this year, at the AKBC Workshop on Bias in Knowledge Graphs, we presented a ++[paper](https://www.amazon.science/publications/measuring-social-bias-in-knowledge-graph-embeddings)++ that examines this problem. Using a standard embedding technique, we looked for correlations between the professions of people listed in Wikidata and demographic factors, such as gender, ethnicity, and religion, to see whether the embeddings do indeed encode harmful social biases. \n\nFollowing on from this, at last week’s Conference on Empirical Methods in Natural Language Processing (++[EMNLP](https://www.amazon.science/conferences-and-events/emnlp-2020)++), we presented “++[Debiasing knowledge graph embeddings](https://www.amazon.science/publications/debiasing-knowledge-graph-embeddings)++”, in which we attempt to address this problem by developing a lightweight alteration to the standard method of training graph embeddings that reduces bias. \n\nAs knowledge graph embeddings become more widely used within the machine learning community, we hope this work raises awareness of the biases they may encode and moves us closer to the goal of effective debiasing.\n\n\n#### **Knowledge graph embedding**\n\n\nA standard knowledge graph represents data using triples, each of which consists of two entities and the relationship between them: for instance, the entities emmanuelle_charpentier and germany are related by the relation lives_in.\n\n\n\nKnowledge graphs generally store facts as triples consisting of a left entity, a relation, and a right entity.\n\nCREDIT: JOSEPH FISHER\n\nKnowledge graph embeddings represent the entities in a knowledge graph as points in a multidimensional space. The idea is that spatial relationships between the points encode the relationships captured by the graph. \n\nWith the common embedding framework TransE, for instance, adding the vector representing the relationship lives_in to the point representing emmanuelle_charpentier should bring us close to the location of the point representing germany. \n\nDuring training, the embedding model learns to maximize the accuracy of these spatial relationships across all the triples captured in the knowledge graph. Among other applications, embeddings can be used for link prediction, or inferring relationships between entities that do not yet feature in the graph.\n\n\n#### **Do trained knowledge graph embeddings encode social biases?**\n\n\nTo see why knowledge graph embeddings might encode social biases, let’s look at the counts of male and female entities in ++[Wikidata](https://www.wikidata.org/)++, the most extensive open-source knowledge graph.\n\n\n\nCounts of male and female entities in Wikidata\n\nCREDIT: JOSEPH FISHER\n\nThere are more than four times as many male entities in Wikidata as there are female, a reflection of long-persisting social biases in the real world. \n\nIn our paper “++[Measuring social bias in knowledge graph embeddings](https://www.amazon.science/publications/measuring-social-bias-in-knowledge-graph-embeddings)++”, we determine whether such differences in counts become encoded in embeddings. To do this, we take the embedding of a human entity and tune it so that the addition of a relation vector — such has has_religion or has_gender — edges closer to the embedding for some particular right-hand attribute — such as “Catholic” or “female”.\n\n\n\nThe top 20 “most female” professions in Wikidata according to TransE embeddings. ***B_p*** denotes the bias score, ***C_fem*** the counts of female entities in the knowledge graph with these professions, and ***C_male*** the counts of male entities with these professions.\n\nFROM “MEASURING SOCIAL BIAS IN KNOWLEDGE GRAPH EMBEDDINGS”\n\nAs we tune the embedding, we observe how the result of adding the has_profession vector changes. That is, for each potential profession, we determine whether the model assigns it to the person with greater or lesser probability as the embedding changes. \n\nRunning this calculation across all humans and professions, we are able to identify the professions that the embeddings encode as the “most male” and the “most female”. The table at right shows the top 20 “most female” professions according to our measure. (The number of entities in Wikipedia with non-binary genders is comparatively negligible; although this represents another bias in the data, it also means that the resulting embeddings would be too noisy to yield meaningful results in our study.)\n\n\n\nTop 20 \"most male\" professions in Wikidata according to TransE embeddings.\n\nFROM “MEASURING SOCIAL BIAS IN KNOWLEDGE GRAPH EMBEDDINGS”\n\nThe differences in the counts of entities in the knowledge graph with these professions appear to translate to biases in the embeddings. There are some professions, such as “homekeeper”, that we would prefer were not associated with a particular gender; others, such as “woman of letters”, may be less controversial. \n\nWe also calculate the top 20 “male” professions, where the conclusions are similar.\n\n\n#### **Can we adjust the training of knowledge graph embeddings to reduce encoded biases?**\n\n\nIn “++[Debiasing knowledge graph embeddings](https://assets.amazon.science/ad/82/c9bf75a44dde82f48c08804d9230/debiasing-knowledge-graph-embeddings.pdf)++”, we turn our attention to reducing such biases and their potentially harmful consequences for downstream applications, such as chatbots. To do this, we train the embedding model not only on how faithfully it reconstructs triples but also on how well it approximates even distributions for gender and other sensitive characteristics, such as religion. \n\nPut another way, we update the embedding of **person1** so that it becomes impossible for the model to predict gender. If this is done precisely, it should also break correlations between gender and profession.\n\nA potential drawback is that this approach prevents the model from using gender, religion, nationality, or ethnicity to predict noncontroversial triples. For instance, we may like the embeddings to reflect that a **nun** is more likely to be female than male.\n\n\n\nOur debiasing approach uses Kullback-Leibler (KL) divergence to measure how well a knowledge graph embedding scheme matches our target distributions.\n\nCREDIT: JOSEPH FISHER\n\nTo allow this, we introduce **attribute embeddings**. In cases where we wish to make use of sensitive information, we can simply add these attribute vectors back in to the embeddings.\n\n\n\nHere, we add the male attribute embedding back into the model for the profession nun but not for the profession doctor.\n\nCREDIT: JOSEPH FISHER\n\nWe evaluate our model against a **Basic** TransE model with no debiasing and against the debiasing approach adopted by **Bose et al.**, which uses neural-network filters proposed in the literature previously. We measure the usefulness of the embeddings for link prediction (according to mean reciprocal rank, or MRR), their bias, and training time.\n\nDuring training, embeddings are scored on their accuracy — the degree to which they reproduce the corresponding triples in the knowledge graph. We measure bias as the difference between those scores for entities that fall into one category or another — religion, gender, and so on. We find that our model incurs a slight (roughly 3%) dropoff in link prediction accuracy in exchange for a dramatic reduction in bias.\n\n\n\nABOUT THE AUTHOR\n\n\n#### **[Joseph Fisher](https://www.amazon.science/author/joseph-fisher)**\n\n\nJoseph Fisher is an applied scientist in the Alexa AI organization.","render":"<p>Question-answering systems frequently rely on knowledge graphs, large collections of facts about real-world entities (people, organizations, countries, etc.). To make use of the information in knowledge graphs, machine learning models often employ knowledge graph embeddings, vector representations of the entities in the graphs.</p>\n<p>A potential problem with this approach is that the distributions of data in knowledge graphs reflect current and historical social biases. For instance, most knowledge graphs include more male entities than female with the profession “banker”, or more “white American” entities than “African-American” entities with the profession “ballet dancer”.</p>\n<p>If knowledge graph embeddings end up encoding these biases, so will the question-answering systems that use them. If a little girl talking to a chatbot asks, “What should I be when I grow up?”, a biased embedding might rule out possible answers that are predominantly associated with men in the knowledge graph. For some professions — “baritone”, for instance — that may be fine. But in other cases, the biases may be relics of a less egalitarian past.</p>\n<p><img src=\"https://dev-media.amazoncloud.cn/eafd80dbfa094a5caa016a46680dd047_image.png\" alt=\"image.png\" /></p>\n<p>A two-dimensional representation of our method for measuring bias in knowledge graph embeddings. In each diagram, the blue dots labeled person1 indicate the shift in an embedding as we tune its parameters. The orange arrows represent relation vectors and the orange dots the resulting sums. As we shift the gender relation toward maleness, the profession relation shifts away from nurse and closer to doctor.</p>\n<p>CREDIT: GLYNIS CONDON</p>\n<p>Earlier this year, at the AKBC Workshop on Bias in Knowledge Graphs, we presented a <ins><a href=\"https://www.amazon.science/publications/measuring-social-bias-in-knowledge-graph-embeddings\" target=\"_blank\">paper</a></ins> that examines this problem. Using a standard embedding technique, we looked for correlations between the professions of people listed in Wikidata and demographic factors, such as gender, ethnicity, and religion, to see whether the embeddings do indeed encode harmful social biases.</p>\n<p>Following on from this, at last week’s Conference on Empirical Methods in Natural Language Processing (<ins><a href=\"https://www.amazon.science/conferences-and-events/emnlp-2020\" target=\"_blank\">EMNLP</a></ins>), we presented “<ins><a href=\"https://www.amazon.science/publications/debiasing-knowledge-graph-embeddings\" target=\"_blank\">Debiasing knowledge graph embeddings</a></ins>”, in which we attempt to address this problem by developing a lightweight alteration to the standard method of training graph embeddings that reduces bias.</p>\n<p>As knowledge graph embeddings become more widely used within the machine learning community, we hope this work raises awareness of the biases they may encode and moves us closer to the goal of effective debiasing.</p>\n<h4><a id=\"Knowledge_graph_embedding_19\"></a><strong>Knowledge graph embedding</strong></h4>\n<p>A standard knowledge graph represents data using triples, each of which consists of two entities and the relationship between them: for instance, the entities emmanuelle_charpentier and germany are related by the relation lives_in.</p>\n<p><img src=\"https://dev-media.amazoncloud.cn/05a793a3ff7c4f7291cf410da902d648_image.png\" alt=\"image.png\" /></p>\n<p>Knowledge graphs generally store facts as triples consisting of a left entity, a relation, and a right entity.</p>\n<p>CREDIT: JOSEPH FISHER</p>\n<p>Knowledge graph embeddings represent the entities in a knowledge graph as points in a multidimensional space. The idea is that spatial relationships between the points encode the relationships captured by the graph.</p>\n<p>With the common embedding framework TransE, for instance, adding the vector representing the relationship lives_in to the point representing emmanuelle_charpentier should bring us close to the location of the point representing germany.</p>\n<p>During training, the embedding model learns to maximize the accuracy of these spatial relationships across all the triples captured in the knowledge graph. Among other applications, embeddings can be used for link prediction, or inferring relationships between entities that do not yet feature in the graph.</p>\n<h4><a id=\"Do_trained_knowledge_graph_embeddings_encode_social_biases_37\"></a><strong>Do trained knowledge graph embeddings encode social biases?</strong></h4>\n<p>To see why knowledge graph embeddings might encode social biases, let’s look at the counts of male and female entities in <ins><a href=\"https://www.wikidata.org/\" target=\"_blank\">Wikidata</a></ins>, the most extensive open-source knowledge graph.</p>\n<p><img src=\"https://dev-media.amazoncloud.cn/cd6f327d33ca4c0e8028da11b2653dd6_image.png\" alt=\"image.png\" /></p>\n<p>Counts of male and female entities in Wikidata</p>\n<p>CREDIT: JOSEPH FISHER</p>\n<p>There are more than four times as many male entities in Wikidata as there are female, a reflection of long-persisting social biases in the real world.</p>\n<p>In our paper “<ins><a href=\"https://www.amazon.science/publications/measuring-social-bias-in-knowledge-graph-embeddings\" target=\"_blank\">Measuring social bias in knowledge graph embeddings</a></ins>”, we determine whether such differences in counts become encoded in embeddings. To do this, we take the embedding of a human entity and tune it so that the addition of a relation vector — such has has_religion or has_gender — edges closer to the embedding for some particular right-hand attribute — such as “Catholic” or “female”.</p>\n<p><img src=\"https://dev-media.amazoncloud.cn/5eb9655d399a47f0aa21706c2bbc0c9d_image.png\" alt=\"image.png\" /></p>\n<p>The top 20 “most female” professions in Wikidata according to TransE embeddings. <em><strong>B_p</strong></em> denotes the bias score, <em><strong>C_fem</strong></em> the counts of female entities in the knowledge graph with these professions, and <em><strong>C_male</strong></em> the counts of male entities with these professions.</p>\n<p>FROM “MEASURING SOCIAL BIAS IN KNOWLEDGE GRAPH EMBEDDINGS”</p>\n<p>As we tune the embedding, we observe how the result of adding the has_profession vector changes. That is, for each potential profession, we determine whether the model assigns it to the person with greater or lesser probability as the embedding changes.</p>\n<p>Running this calculation across all humans and professions, we are able to identify the professions that the embeddings encode as the “most male” and the “most female”. The table at right shows the top 20 “most female” professions according to our measure. (The number of entities in Wikipedia with non-binary genders is comparatively negligible; although this represents another bias in the data, it also means that the resulting embeddings would be too noisy to yield meaningful results in our study.)</p>\n<p><img src=\"https://dev-media.amazoncloud.cn/63a1cadfaa2b433bb01b2440fd67e540_image.png\" alt=\"image.png\" /></p>\n<p>Top 20 “most male” professions in Wikidata according to TransE embeddings.</p>\n<p>FROM “MEASURING SOCIAL BIAS IN KNOWLEDGE GRAPH EMBEDDINGS”</p>\n<p>The differences in the counts of entities in the knowledge graph with these professions appear to translate to biases in the embeddings. There are some professions, such as “homekeeper”, that we would prefer were not associated with a particular gender; others, such as “woman of letters”, may be less controversial.</p>\n<p>We also calculate the top 20 “male” professions, where the conclusions are similar.</p>\n<h4><a id=\"Can_we_adjust_the_training_of_knowledge_graph_embeddings_to_reduce_encoded_biases_73\"></a><strong>Can we adjust the training of knowledge graph embeddings to reduce encoded biases?</strong></h4>\n<p>In “<ins><a href=\"https://assets.amazon.science/ad/82/c9bf75a44dde82f48c08804d9230/debiasing-knowledge-graph-embeddings.pdf\" target=\"_blank\">Debiasing knowledge graph embeddings</a></ins>”, we turn our attention to reducing such biases and their potentially harmful consequences for downstream applications, such as chatbots. To do this, we train the embedding model not only on how faithfully it reconstructs triples but also on how well it approximates even distributions for gender and other sensitive characteristics, such as religion.</p>\n<p>Put another way, we update the embedding of <strong>person1</strong> so that it becomes impossible for the model to predict gender. If this is done precisely, it should also break correlations between gender and profession.</p>\n<p>A potential drawback is that this approach prevents the model from using gender, religion, nationality, or ethnicity to predict noncontroversial triples. For instance, we may like the embeddings to reflect that a <strong>nun</strong> is more likely to be female than male.</p>\n<p><img src=\"https://dev-media.amazoncloud.cn/57a483995ffb4f7385d0259dcb280448_image.png\" alt=\"image.png\" /></p>\n<p>Our debiasing approach uses Kullback-Leibler (KL) divergence to measure how well a knowledge graph embedding scheme matches our target distributions.</p>\n<p>CREDIT: JOSEPH FISHER</p>\n<p>To allow this, we introduce <strong>attribute embeddings</strong>. In cases where we wish to make use of sensitive information, we can simply add these attribute vectors back in to the embeddings.</p>\n<p><img src=\"https://dev-media.amazoncloud.cn/2d10a8be3e744baaa593b97b4c517ec0_image.png\" alt=\"image.png\" /></p>\n<p>Here, we add the male attribute embedding back into the model for the profession nun but not for the profession doctor.</p>\n<p>CREDIT: JOSEPH FISHER</p>\n<p>We evaluate our model against a <strong>Basic</strong> TransE model with no debiasing and against the debiasing approach adopted by <strong>Bose et al.</strong>, which uses neural-network filters proposed in the literature previously. We measure the usefulness of the embeddings for link prediction (according to mean reciprocal rank, or MRR), their bias, and training time.</p>\n<p>During training, embeddings are scored on their accuracy — the degree to which they reproduce the corresponding triples in the knowledge graph. We measure bias as the difference between those scores for entities that fall into one category or another — religion, gender, and so on. We find that our model incurs a slight (roughly 3%) dropoff in link prediction accuracy in exchange for a dramatic reduction in bias.</p>\n<p><img src=\"https://dev-media.amazoncloud.cn/236c42050f10490fb321201960e10ade_e00ea4641762dff57deaddc1ae0efcd.png\" alt=\"e00ea4641762dff57deaddc1ae0efcd.png\" /></p>\n<p>ABOUT THE AUTHOR</p>\n<h4><a id=\"Joseph_Fisherhttpswwwamazonscienceauthorjosephfisher_105\"></a><strong><a href=\"https://www.amazon.science/author/joseph-fisher\" target=\"_blank\">Joseph Fisher</a></strong></h4>\n<p>Joseph Fisher is an applied scientist in the Alexa AI organization.</p>\n"}