Chapter 14: Lifting & Lowering Methods

                Learning Objectives:

                Understand the two main integration strategies in neuro-symbolic AI: Lowering (compress symbols into neural patterns) and Lifting (extract symbols from neural outputs). Learn attention rewiring and KG-RAG approaches.

The Two Integration Strategies

The core challenge of neuro-symbolic AI: how to combine neural perception (learning patterns) with symbolic reasoning (applying rules)? There are two fundamental approaches:

Approach	What It Does	Pros	Cons
Lowering	Compress symbolic knowledge into neural patterns	Fast, end-to-end differentiable	Less transparent, hard constraints not enforced
Lifting	Extract symbolic structure from neural outputs	Interpretable, enforces hard rules	Requires careful pipeline design, slower

Lowering: Compress Symbols into Neural Patterns

Lowering means embedding symbolic knowledge (facts, rules, graphs) directly into neural network representations, so the network can reason using learned patterns.

Attention Rewiring: Knowledge Fusion with LLMs

Instead of just passing text to an LLM, you can inject knowledge graph information into the neural computation itself.

How it works:

User asks: "What drug treats diabetes and hypertension?"
Retrieve relevant KG triples: (drug_X, treats, diabetes), (drug_X, treats, hypertension)
Convert KG triples to attention tensors (K,V pairs)
Prepend these tensors to LLM's attention mechanism
LLM now has direct access to KG facts during token prediction
Output: More grounded, factually correct answer

# Attention Rewiring pseudocode
def rewire_attention_with_kg(question, llm_model, knowledge_graph):
    # Step 1: Query KG for relevant facts
    relevant_triples = kg.query(question)
    # E.g., [(diabetes, treats, metformin), (hypertension, treats, metformin)]

    # Step 2: Convert triples to attention tensors
    kg_keys = encode_triples_as_keys(relevant_triples)      # Shape: [num_triples, d]
    kg_values = encode_triples_as_values(relevant_triples)  # Shape: [num_triples, d]

    # Step 3: Prepend KG tensors to LLM's attention
    # Normal LLM attention: Q @ K.T where K is word tokens
    # Rewired: Q @ [kg_keys | word_keys].T
    # Now LLM attends to both KG facts AND text context

    for layer in llm_model.transformer_layers[selected_layers]:
        original_K = layer.attention.key(question_tokens)
        original_V = layer.attention.value(question_tokens)

        # Prepend KG information
        new_K = concat([kg_keys, original_K])
        new_V = concat([kg_values, original_V])

        layer.attention.K = new_K
        layer.attention.V = new_V

    # Step 4: Generate output (now informed by KG)
    output = llm_model.generate(question)
    return output

Result: LLM generates answers grounded in knowledge graph facts, not hallucinations.

KG-RAG: Knowledge Graph Retrieval-Augmented Generation

KG-RAG combines knowledge graphs with RAG: retrieve relevant evidence subgraphs, then generate grounded answers.

Medical Diagnosis with KG-RAG

# Question: "Why might aspirin help this patient?"
# Patient: 75-year-old with heart disease, high blood pressure, no aspirin allergy

# Step 1: Retrieve relevant subgraph
query = """
MATCH (drug:Drug)-[r1:treats]->(condition:Condition),
      (drug)-[r2:has_mechanism]->(mechanism:Mechanism),
      (drug)-[r3:has_side_effect]->(effect:Effect)
WHERE drug.name = 'aspirin'
RETURN drug, condition, mechanism, effect, r1, r2, r3
"""

evidence_subgraph = kg.execute(query)
# Returns: aspirin → treats → cardiovascular disease
#          aspirin → mechanism → inhibits platelet aggregation
#          aspirin → side_effect → GI bleeding (CAUTION for this patient!)

# Step 2: Feed to LLM with context
prompt = f"""
Based on this knowledge graph evidence:
{format_evidence(evidence_subgraph)}

Patient: 75-year-old, heart disease, high BP, no aspirin allergy

Question: Why might aspirin help this patient?
Be sure to mention:
- How it works (mechanism)
- Any relevant risks (side effects)
- Why it's suitable for THIS patient
"""

answer = llm.generate(prompt)
# Output: "Aspirin works by inhibiting platelet aggregation, reducing
#          heart attack risk. The main concern would be GI bleeding in
#          elderly patients, but this patient has no documented GI issues..."

Lifting: Extract Symbols from Neural Outputs

Lifting means taking neural network outputs (which are implicit, continuous) and converting them into explicit symbolic structures (facts, rules, graphs) that can be reasoned over.

Clinical Lifting Pipeline

The pipeline:

# Step 1: Neural Perception (implicit)
patient_text = "Patient has fever 101F, cough for 2 days, no trouble breathing"
nlp_output = nlp_model(patient_text)
# Output: Dense embeddings, attention weights (implicit knowledge)

# Step 2: Entity & Relation Extraction (lifting to symbolic)
entities = extract_entities(patient_text)
# [SYMPTOM: fever, SEVERITY: 101F, SYMPTOM: cough, DURATION: 2days, ...]

relations = extract_relations(patient_text, entities)
# [has_symptom(patient, fever), severity(fever, 101F), has_symptom(patient, cough), ...]

# Step 3: Build Symbolic Graph (now explicit, auditable)
symbolic_facts = {
    'has_symptom(patient, fever)': True,
    'severity(fever, 101F)': True,
    'has_symptom(patient, cough)': True,
    'duration(cough, 2days)': True,
    'has_difficulty_breathing': False
}

# Step 4: Apply Symbolic Rules (explicit inference)
if symbolic_facts['has_symptom(patient, fever)'] and \
   symbolic_facts['has_symptom(patient, cough)'] and \
   not symbolic_facts['has_difficulty_breathing']:
    diagnosis = 'Probable viral URI'
    severity = 'mild'
else:
    escalate_to_expert()

# Step 5: Generate Explanation (traceable)
explanation = f"""
Diagnosis: {diagnosis}
Based on:
- Patient has fever (101F) ✓
- Patient has cough (2 days) ✓
- No respiratory distress ✓
Rule fired: fever + cough + no_distress → viral_URI
Confidence: 0.85
Recommendation: Monitor for 48 hours, escalate if worsens
"""

Lifting Components: Entity and Relation Extraction

The key lifting operations are:

Entity Extraction (NER): Identify what things are mentioned (diseases, drugs, symptoms, etc.)
Relation Extraction: Identify relationships between entities (treats, causes, contraindicates)
Coreference Resolution: Link pronouns and aliases to entities ("it" → disease, "the drug" → aspirin)
Fact Assembly: Convert extracted info into symbolic facts/triples
Consistency Checking: Verify facts don't contradict knowledge base

Lowering vs Lifting: When to Use Each

Use Lowering When:
✓ You want end-to-end learning (knowledge and neural weights optimize together)
✓ Speed is critical (no intermediate extraction step)
✓ Knowledge is dense (large KG, many triples)
✗ But: Hard constraints not guaranteed, less interpretable

Use Lifting When:
✓ You need interpretability and auditability
✓ You need to enforce hard constraints (safety rules, regulations)
✓ You need to check for contradictions
✗ But: Requires good NLP models, slower, error-prone extraction

Hybrid Approaches

Many systems combine both:

Lifting + Symbolic Reasoning + Lowering (3-Step Pipeline)

# Step 1: LIFTING (Neural → Symbolic)
patient_description = "70-year-old with chest pain and shortness of breath"
extracted_facts = lift_text_to_facts(patient_description)
# Output: [age(patient, 70), symptom(patient, chest_pain), symptom(patient, SOB)]

# Step 2: SYMBOLIC REASONING (Symbolic → Symbolic)
inferred_facts = apply_rules(extracted_facts)
# Rules fired: symptom(X, chest_pain) ∧ symptom(X, SOB) → urgent(X)
# Output: [age(patient, 70), ..., urgent(patient)]

# Step 3: VERIFICATION & GROUNDING (Symbolic → Neural)
# Check facts against learned embeddings
fact_confidence = verify_with_embedding_space(inferred_facts)
# If confidence < threshold, return to neural for clarification

if fact_confidence > 0.8:
    recommendation = "Send to ER immediately"
else:
    recommendation = "Request clarification from clinician"

Key Takeaways

Lowering = compress symbols into neural (fast, learned, less transparent)
Lifting = extract symbols from neural (interpretable, verifiable, slower)
Attention rewiring injects KG facts directly into LLM computation
KG-RAG retrieves evidence subgraphs to ground generation
Entity/relation extraction is the core lifting operation
Hybrid systems combine both for transparency + performance

🎯 Ready for Assessment?

10 Questions

~10 mins

70% to Pass

Take Quiz Now →