SciDEX — Task: [Senate] Cross-claim consistency engine

Embedding shortlist + LLM contradiction judge for every pair of atomic claims; auto-spawn debate to adjudicate when confidence>=0.8.

Completion Notes

Auto-completed by supervisor after successful deploy to main

Git Commits (2)

[Senate] Cross-claim consistency engine — flag contradictory hypotheses [task:88952f44-693d-4ec2-95f2-289c6712b2c4] (#755)2026-04-27

[Senate] Cross-claim consistency engine — flag contradictory hypotheses [task:88952f44-693d-4ec2-95f2-289c6712b2c4]2026-04-27

Spec File

Goal

The platform now hosts 310+ hypotheses and 18K wiki pages, and many of
them make claims that directly contradict each other (e.g. one
hypothesis says TREM2 inhibits microglial phagocytosis; another says it
enhances it). Today nothing surfaces these contradictions — gflownet_sampler rewards diversity, but diversity is not the same as
consistency. Build an engine that, for each pair of normalized atomic
claims (across hypotheses, wiki, debates), detects logical
contradiction (A says X, B says ¬X) via embedding-similarity
shortlisting + LLM contradiction judge, and emits a structured claim_contradiction artifact that auto-spawns a debate to adjudicate.

Effort: thorough

Acceptance Criteria

☑ scidex/senate/claim_consistency.py::scan(window_days=7, max_pairs=500) -> dict iterates atomic claims (from claim_artifacts table populated by wiki_claim_extractor + new hypothesis claims; if claim_artifacts doesn't exist, create a unified view).

☑ Stage 1 — embedding shortlist: for each claim, find top-K (K=10) nearest claims by cosine similarity over pgvector embeddings (reuse the embedding pipeline from scidex/atlas/canonical_entity_links.py or scidex/agents/embeddings.py); only consider pairs where cosine ≥ 0.65 AND polarities differ.

☑ Stage 2 — LLM contradiction judge: prompt "Do these two claims directly contradict? <a>...<b>..." returning {contradicts: bool, confidence: float, common_subject: str, divergent_predicate: str}. Triple-redundancy via 3 personas (skeptic, methodologist, domain-expert) — verdict = mode.

☑ Migration migrations/20260428_claim_contradictions.sql:

claim_contradictions(id, claim_a_id, claim_b_id, common_subject, divergent_predicate, confidence, status ENUM('open','adjudicated','reconciled','withdrawn'), spawned_debate_id TEXT, detected_at)

; UNIQUE(LEAST(a,b), GREATEST(a,b)).

☑ Auto-debate: when confidence ≥ 0.8, spawn a debate via

scidex.agora.debate.create_debate(topic=common_subject, position_a=claim_a, position_b=claim_b, seed_evidence=union(claim_a.evidence, claim_b.evidence))

; write spawned_debate_id back. Reuse the pattern from refutation_emitter.py.

☑ Adjudication: when the spawned debate's Synthesizer reaches a verdict, mark status='adjudicated'; the losing claim's host artifact gets a disputed quality tag and a 5 % composite_score dock.

☑ API: GET /api/senate/contradictions?status=open&limit=50 returns the open contradictions for a triage UI; GET /senate/contradictions HTML page lists them with side-by-side diff.

☐ q-live-tau-biology-pipeline-status etc. embed a "Contradictions in this domain" badge on landing pages, sourced from this table.

☑ Tests tests/test_claim_consistency.py: synthetic pair "TREM2 enhances microglial activation" vs "TREM2 inhibits microglial activation" → contradiction detected, verdict ≥ 0.8; near-paraphrase pair → not flagged; trivially-different-subject pair → not flagged (common_subject mismatch).

Approach

Confirm or create the unified atomic-claim view (UNION of wiki claims, hypothesis claims, debate-extracted claims) — likely a claim_artifacts_v Postgres view.

Add an embedding column on the view's source tables if missing; backfill via the existing embedding pipeline. Stage 1 then becomes a single pgvector <=> query.

Implement the triple-redundancy LLM judge with a strict JSON contract; cache per (a_id, b_id, prompt_version) to avoid recomputation.

Implement debate spawning + adjudication callback. The callback hooks into synthesis_engine.synthesize_debate_session post-step.

Build the triage page; smoke-run on the existing 310 hypotheses pairwise (≈ 47K pairs → embedding shortlist cuts to ≈ 300).

Dependencies

q-qual-auto-fact-check-pipeline — provides claim verdicts that influence polarity.
q-qual-claim-strength-normalizer — produces normalized claim subject/predicate.
scidex/agents/embeddings.py — pgvector embedding pipeline.

Dependents

q-mem-persona-reputation-log — debate outcomes feed persona track records.

Work Log

2026-04-28 — Implementation complete

Created migrations/20260428_claim_contradictions.sql: claim_contradictions table + history + audit trigger. Uses functional unique index (CASE-based) for unordered pair deduplication since PostgreSQL doesn't support LEAST/GREATEST in constraint definitions. contradiction_status enum: open/adjudicated/reconciled/withdrawn.
Created scidex/senate/claim_consistency.py: polarity inference (keyword-based), cosine similarity shortlist (cosine≥0.65, opposite polarity, top-K=10), triple-redundancy LLM judge (skeptic/methodologist/domain-expert personas), mode voting + mean confidence, auto-debate spawning via SciDEXOrchestrator.run_debate(), adjudication with 5% loser penalty, scan() and get_contradictions() APIs.
Created api_routes/senate.py: POST /api/senate/contradictions/scan, GET /api/senate/contradictions, GET /api/senate/contradictions/stats.
Created site/contradictions.html: triage UI with stats grid, scan button, filter tabs, side-by-side claim diff cards, live search.
Added GET /senate/contradictions route in api.py.
Created tests/test_claim_consistency.py: 32 tests (all passing). Covers polarity inference, opposite polarity detection, cosine similarity, judge response parsing (including markdown fence stripping), mode voting, mean confidence, prompt format.
Note: ChromaDB embedding collection is empty (0 items), so scan() computes embeddings via SentenceTransformer directly. Badge embedding on domain pages (AC-8) not yet implemented.