Learning Objectives:
Learn how to enhance LLM reasoning by integrating symbolic solvers. Understand the 3-module Logic-LM architecture, implement medical drug logic, and see how formal reasoning guarantees correctness.
Learn how to enhance LLM reasoning by integrating symbolic solvers. Understand the 3-module Logic-LM architecture, implement medical drug logic, and see how formal reasoning guarantees correctness.
Original Paper
Citation:
Pan, L., Albalak, A., Wang, X., & Wang, W. (2023). "Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning." In Findings of the Association for Computational Linguistics: EMNLP 2023 (pp. 3806-3824).
ArXiv: https://arxiv.org/abs/2305.12295
Conference: EMNLP 2023 (Empirical Methods in Natural Language Processing)
Authors: Lifeng Pan, Abraham Albalak, Xin Wang, William Yang Wang
Pan, L., Albalak, A., Wang, X., & Wang, W. (2023). "Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning." In Findings of the Association for Computational Linguistics: EMNLP 2023 (pp. 3806-3824).
ArXiv: https://arxiv.org/abs/2305.12295
Conference: EMNLP 2023 (Empirical Methods in Natural Language Processing)
Authors: Lifeng Pan, Abraham Albalak, Xin Wang, William Yang Wang
Why Pure LLM Reasoning Fails on Logic
LLMs struggle with formal logic and constraint satisfaction. They may understand concepts but can't reliably solve logic puzzles or guarantee solutions satisfy hard constraints.
The Core Problem (Pan et al., 2023):
"While LLMs excel at pattern matching, they struggle with formal logical reasoning that requires considering all constraints simultaneously. Logic-LM bridges this gap by combining neural understanding with symbolic solvers."
"While LLMs excel at pattern matching, they struggle with formal logical reasoning that requires considering all constraints simultaneously. Logic-LM bridges this gap by combining neural understanding with symbolic solvers."
The 3-Module Logic-LM Architecture
Figure 1 (Pan et al., 2023): Logic-LM 3-module architecture applied to drug interactions
How Each Module Works
| Module | Input | Technology | Output |
|---|---|---|---|
| 1. Problem Formulator | Natural language question | LLM (neural) | Logical formulas (predicate logic, SAT) |
| 2. Symbolic Solver | Logical formulas + constraints | Z3, SMT solver, SAT solver | Solution (satisfying assignment) or UNSAT |
| 3. Result Interpreter | Solver output (solution) | LLM (neural) | Natural language explanation |
Medical Implementation: Drug Interaction Checker
# Logic-LM for drug interaction checking (simplified)
from anthropic import Anthropic
client = Anthropic()
def logic_lm_drug_checker(current_drugs, candidate_drug):
"""Check drug interactions using Logic-LM approach"""
# Step 1: Problem Formulation (NL → Logic)
formulation_prompt = f"""
Convert this drug interaction problem to logical constraints:
Current drugs: {', '.join(current_drugs)}
Candidate drug: {candidate_drug}
Output logical predicates:
- Drug(X) for each drug
- Contraindication(DrugA, DrugB) for known interactions
- Query: Can_add({candidate_drug})?
Formal constraint:
Can_add(D) :- NOT (exists C in CurrentDrugs where Contraindication(C, D))"""
formulation = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=300,
messages=[{"role": "user", "content": formulation_prompt}]
)
logical_formulas = formulation.content[0].text
# Step 2: Symbolic Solver
# (In practice, would use Z3 or similar solver)
solver_prompt = f"""
Using these logical constraints:
{logical_formulas}
Apply the constraint and determine:
- Is the query satisfiable? (Can we add the drug?)
- If not satisfiable, what contradictions exist?
- Provide the result as: SATISFIABLE / UNSATISFIABLE"""
solver = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=300,
messages=[{"role": "user", "content": solver_prompt}]
)
solver_result = solver.content[0].text
# Step 3: Result Interpretation (Logic → NL)
interpretation_prompt = f"""
Given this solver result:
{solver_result}
Generate a clear clinical recommendation:
- Is it safe to add {candidate_drug}?
- Why or why not? (cite specific contraindications)
- What are the risks?
- What alternatives exist if contraindicated?
Format as a clinical summary for a physician."""
interpretation = client.messages.create(
model="claude-3-5-sonnet-20241022",
max_tokens=400,
messages=[{"role": "user", "content": interpretation_prompt}]
)
final_output = interpretation.content[0].text
return {
"formulation": logical_formulas,
"solver_result": solver_result,
"clinical_recommendation": final_output
}
# Example usage
result = logic_lm_drug_checker(
current_drugs=["warfarin", "aspirin"],
candidate_drug="ibuprofen"
)
print("DRUG INTERACTION CHECK")
print("=" * 50)
print(result["clinical_recommendation"])
Key Advantages of Logic-LM
Why Logic-LM Works
- Guarantees correctness: Solver proves the answer is correct or impossible
- Handles constraints: Hard constraints (contraindications) are enforced
- Combines strengths: Neural language + symbolic reasoning
- Explainable: Can show exactly why a drug is contraindicated
- Safe by design: Can't accidentally output unsafe recommendations
When to Use Logic-LM vs Other Methods
| Scenario | Best Approach | Why |
|---|---|---|
| Drug interactions, constraints | Logic-LM | Hard constraints need formal verification |
| Complex reasoning, no constraints | Chain-of-Thought | Simpler, sufficient for reasoning |
| Hallucination-prone tasks | Chain-of-Verification | Fact-checking catches errors |
| Multi-path reasoning | Graph-of-Logic | Explore multiple hypotheses |
Key Takeaways
- Logic-LM = 3-module system combining neural and symbolic AI
- Module 1: LLM translates NL to logical formulas
- Module 2: Solver (Z3, SAT) finds solution guaranteeing correctness
- Module 3: LLM translates solution back to natural language
- Guarantees safety: Hard constraints can't be violated
- Perfect for medical AI: Drug interactions, dosing rules, contraindications
Original Paper:
Pan, L., Albalak, A., Wang, X., & Wang, W. (2023). "Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning." EMNLP 2023 Findings.
Available on ArXiv: https://arxiv.org/abs/2305.12295
Pan, L., Albalak, A., Wang, X., & Wang, W. (2023). "Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning." EMNLP 2023 Findings.
Available on ArXiv: https://arxiv.org/abs/2305.12295
🎯 Ready for Assessment?
10 Questions
~10 mins
70% to Pass