CHAPTER 13

⚡ Graph Embeddings & Vector Representations

Bridging Structured Knowledge and Neural Space

Learning Objectives:
Understand graph embeddings, learn Node2Vec and TransE algorithms, explore how to preserve network structure in vector space, and apply embeddings to similarity matching and link prediction.

What Are Graph Embeddings?

Graph embeddings are learned vector representations of graph objects (nodes, edges, subgraphs). The goal: map "who is related to whom" (graph structure) into distances and directions in vector space, so that structure is preserved numerically.

Core Insight:
Convert: "Drug A is similar to Drug B in the knowledge graph"
Into: Vector(Drug A) is close to Vector(Drug B) in R^d space

Then: Use fast vector operations (cosine similarity, nearest neighbor) instead of graph traversal

Why Embeddings Matter for Neuro-Symbolic AI

Knowledge graphs are powerful but slow: multi-hop queries require traversal. Embeddings trade structure for speed:

Node2Vec: Random Walk Based Embeddings

How Node2Vec Works

Idea: Embed nodes so that nodes visited together in random walks are close in vector space.

  1. Random Walk: Start at node, randomly traverse edges (like drunk person walking)
  2. Collect Walks: Perform many random walks from each node
  3. Learn Embeddings: Use skip-gram model (like Word2Vec) to learn vectors
  4. Result: Nodes "similar" in structure are close in vector space

Example in Drug Discovery:

# Drug knowledge graph edges:
drug_A -- similar_to -- drug_B
drug_A -- treats -- disease_X
drug_B -- treats -- disease_X
disease_X -- causes -- symptom_Y

# Random walk starting at drug_A:
drug_A → drug_B → disease_X → symptom_Y → disease_X → drug_B → drug_A

# Node2Vec learns:
# - drug_A and drug_B vectors are close (visited together often)
# - Both close to disease_X (related by same walks)
# - All related to symptom_Y (multi-hop neighbor)

# Use case: Find similar drugs
embedding(drug_A) ≈ [0.2, 0.5, 0.8, ...]
embedding(drug_B) ≈ [0.2, 0.5, 0.8, ...]
similarity = cosine(embedding_A, embedding_B) ≈ 0.95 (very similar!)
                
# Python: Node2Vec implementation
from node2vec import Node2Vec
import networkx as nx

# Create graph
G = nx.Graph()
G.add_edges_from([
    ('drug_A', 'drug_B'),
    ('drug_A', 'disease_X'),
    ('drug_B', 'disease_X'),
    ('disease_X', 'symptom_Y')
])

# Learn embeddings
node2vec = Node2Vec(G, dimensions=64, walks_per_node=10, walk_length=80)
model = node2vec.fit(window=10, min_count=1)

# Get embeddings
drug_A_vec = model.wv['drug_A']  # 64-dimensional vector
drug_B_vec = model.wv['drug_B']

# Similarity
similarity = model.wv.similarity('drug_A', 'drug_B')  # 0.0 to 1.0

# Find similar drugs
similar_to_A = model.wv.most_similar('drug_A', topn=5)

TransE: Relation-Based Embeddings

TransE models knowledge graph triples (head, relation, tail) as vector equations: head + relation ≈ tail

TransE for Medical Knowledge

# Knowledge graph facts:
(drug_A, treats, disease_X)
(disease_X, caused_by, bacteria_Y)
(bacteria_Y, resistant_to, antibiotic_Z)

# TransE learns embeddings where:
embedding(drug_A) + embedding(treats) ≈ embedding(disease_X)
embedding(disease_X) + embedding(caused_by) ≈ embedding(bacteria_Y)

# This means:
# If disease_X is caused by bacteria_Y
# And drug_A treats disease_X
# Then maybe drug_A is effective against bacteria_Y (link prediction!)

# Scoring a triple:
score(drug_A, treats, disease_X) = ||embedding(drug_A) + rel_treats - embedding(disease_X)||
# Low score = true fact (vectors align)
# High score = false fact (vectors misaligned)

Medical Embeddings: ClinVec and Beyond

Domain-specific embeddings capture medical semantics:

Embedding Type What It Captures Use Case
ClinVec Clinical concepts from EHR data Disease similarity, treatment prediction
Protein Embeddings Protein sequences, structures Drug-target prediction, protein function
Drug Embeddings Chemical structure, pharmacophore Drug discovery, side effect prediction

Practical Application: Drug-Disease Linking

End-to-End Example

# Step 1: Load drug and disease embeddings
drug_embeddings = load_pretrained_embeddings('drug_vectors.bin')
disease_embeddings = load_pretrained_embeddings('disease_vectors.bin')

# Step 2: For each disease, find most similar drugs
disease = 'hypertension'
disease_vec = disease_embeddings[disease]

# Fast vector search (instead of graph traversal!)
similar_drugs = find_nearest_neighbors(disease_vec, drug_embeddings, k=10)

# Step 3: Rank and validate
for drug, similarity_score in similar_drugs:
    # Only use if:
    # 1. Similarity high enough (>0.7)
    # 2. No contraindications in knowledge base
    # 3. FDA approval or clinical evidence
    if similarity_score > 0.7:
        if not has_contraindication(drug, disease):
            print(f"{drug} for {disease}: {similarity_score:.3f}")

# Output:
# lisinopril for hypertension: 0.892
# enalapril for hypertension: 0.881
# metoprolol for hypertension: 0.765

Limitations and Considerations

When Embeddings Work Well:
✓ Similarity matching
✓ Link prediction
✓ Clustering
✓ As neural network input
When Embeddings Fall Short:
✗ Logical inference (relationships ≠ geometry)
✗ Hard constraints (vectors are soft)
✗ Interpretability (why are these drugs similar?)
✗ Out-of-distribution nodes (new drugs with no history)

Key Takeaways

  • Graph embeddings map graph structure into vector space for fast computation
  • Node2Vec uses random walks to learn node similarity
  • TransE models relations as vector operations (head + rel ≈ tail)
  • Medical embeddings capture clinical similarity (ClinVec, protein vecs, drug vecs)
  • Embeddings enable neural integration of structured knowledge
  • Trade-off: Speed for interpretability. Use with symbolic constraints!
Related Papers:
  • Grover, A., & Leskovec, J. (2016). "node2vec: Scalable Feature Learning for Networks"
  • Bordes, A., et al. (2013). "Translating Embeddings for Modeling Multi-relational Data" (TransE)

🎯 Ready for Assessment?

10 Questions
~10 mins
70% to Pass
Take Quiz Now →