[Atlas] Extract causal models from debate transcripts and wiki content
Goal
Extract causal relationships from SciDEX debate transcripts and wiki content to build a directed causal model layer in the Atlas knowledge graph. This enables causal reasoning, identifies causal contradictions between Skeptic debates and established wiki knowledge, and provides causal DAG visualizations on entity pages showing upstream causes and downstream effects. The system should automatically deduplicate and merge causal edges from multiple sources, increasing confidence when the same causal claim appears across multiple debates or wiki pages.
Acceptance Criteria
☐ Create causal_edges table in PostgreSQL
☐ Implement causal extraction from debate transcripts (Theorist/Expert rounds)
☐ Implement causal extraction from wiki content
☐ Build deduplication and confidence merging logic
☐ Add causal DAG visualization to entity pages
☐ Implement causal contradiction detection (Skeptic vs wiki)
☐ Test: curl entity pages show causal models
☐ All affected pages load (200 status)
☐ No broken links introduced
☐ Code follows existing patterns
Approach
Read current database schema and debate/wiki data structures
Design and create causal_edges table with proper fields
Extract causal patterns from debate_rounds table (Theorist/Expert personas)
Extract causal patterns from wiki pages (when Atlas integration exists)
Implement deduplication and confidence scoring
Add causal DAG endpoints/visualization to api.py
Integrate causal model display into entity pages
Test extraction on existing debates
Commit and push changesWork Log
2026-04-26 12:57 PT — Slot 76
- Verified task is still valid: causal_edges table didn't exist in DB yet
- Applied migration:
migrations/20260426_add_causal_edges.sql creates causal_edges + history mirror + triggers
- Added
store_causal_edge() with deduplication + confidence merging (same sig → merge avg)
- Added
extract_causal_edges_from_debate() using LLM extraction from Theorist/Expert rounds
- Added pattern-based
extract_causal_from_wiki.py script (14 causal verb patterns)
- Added
/api/causal/edges and /api/causal/contradictions endpoints to api.py
- Added
causal section to /api/entity/{name} response (upstream_causes + downstream_effects)
- Backfilled 19,738 causal edges from debates + wiki pages
- Committed and pushed as
174abde29
Work Log
2026-04-01 23:56 PT — Slot 6
- Started task: causal model extraction from debates and wiki
- Creating spec file following AGENTS.md format
- Read database schema and existing code
2026-04-02 00:15 PT — Slot 6
- Created
causal_edges table with proper schema
- Built
extract_causal_edges.py - pattern-based causal extraction from debate transcripts
- Extracted 251 causal edges from 36 debate rounds (Theorist + Expert personas)
- Added API endpoints:
/api/causal/edges, /api/causal/contradictions
- Enhanced
/api/atlas/entity/{name} to include causal relationships
- Added causal model visualization to entity detail pages
- Updated
/api/status to include causal_edges count
- Tested: 251 causal edges extracted, 0 contradictions found (debates are generally consistent)
- Result: Done - causal model extraction complete and integrated into Atlas