[Senate] Capture belief snapshots for 50 hypotheses missing recent state done analysis:6 reasoning:6

Goal

Backfill recent belief snapshots for hypotheses that lack current time-series state. Belief snapshots let SciDEX measure convergence, evidence accumulation, and confidence changes over time.

Acceptance Criteria

Approach

Dependencies

• q-epistemic-rigor - Epistemic Rigor quest

Dependents

• Epistemic rigor metrics, convergence dashboards, and Senate retrospectives

Work Log

2026-04-26 17:09 UTC — task:eb53fe78

• Before: 61 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 50
• Inserted 50 snapshot rows with composite_score, confidence_score, market_price, evidence_count, citations, reviewer_count all populated
• After: 11 non-archived hypotheses missing snapshots (reduction: 50)
• Same-day duplicate hypotheses: 6 (from earlier overlapping runs)
• 50 new belief_snapshots rows confirmed on 2026-04-26
• Sample: SDA-2026-04-02-gap-tau-prop-20260402003221-H006 (proposed, score=0.547), h-var-71ac892791 (proposed, score=0.649), h-6be901fb (promoted, score=0.560)
• Acceptance criteria: 50 hypothesis snapshots ✓ | score/evidence/citations/reviewer captured ✓ | remaining missing = 11 ✓

2026-04-27 00:57 UTC — task:998ab561

• Before: 161 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 50
• Inserted 50 snapshot rows with composite_score, confidence_score, market_price, evidence_count, citations, reviewer_count all populated
• After: 111 non-archived hypotheses missing snapshots (reduction: 50)
• Same-day duplicate hypotheses: 6 (from earlier overlapping runs; NOTIFY quest engine to tighten window)
• 50 new belief_snapshots rows confirmed on 2026-04-27
• Sample: h-72c719461c (proposed, score=0.720), h-44b1c9d415 (proposed, score=0.817), h-377232dcc8 (proposed, score=0.640)
• Acceptance criteria: 50 hypothesis snapshots ✓ | score/evidence/citations/reviewer captured ✓ | remaining missing = 111 (well under 235 target) ✓

2026-04-26 15:17 UTC — task:770ac149 (third cycle)

• Before: 104 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 30
• Inserted 30 snapshot rows with composite_score, confidence_score, market_price, evidence_count, citations, reviewer_count all populated
• After: 74 non-archived hypotheses missing snapshots (reduction: 30)
• Same-day duplicates: 0
• 30 new belief_snapshots rows confirmed on 2026-04-26
• Sample: h-346639e8 (debated, score=0.621), h-6b394be1 (proposed, score=0.619), h-8f9633d9 (debated, score=0.616)

2026-04-26 22:14 UTC — task:770ac149 (second cycle)

• Improved script: INSERT now populates composite_score, confidence_score, market_price columns (previously left NULL)
• Before: 194 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 30
• Inserted 30 snapshot rows with composite_score, confidence_score, market_price, evidence_count, citations, reviewer_count all populated
• After: 164 non-archived hypotheses missing snapshots (reduction: 30)
• Same-day duplicates: 0
• 30 new belief_snapshots rows confirmed on 2026-04-26

2026-04-26 15:09 UTC — task:770ac149

• Before: 223 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 30
• Inserted 30 snapshot rows with score=composite_score, market_price and confidence_score logged for audit, evidence_count=len(evidence_for)+len(evidence_against), citations=citations_count, reviewer_count=hypothesis_reviews count
• After: 193 non-archived hypotheses missing snapshots (reduction: 30)
• Same-day duplicates: 0
• 30 new belief_snapshots rows confirmed on 2026-04-26

2026-04-26 14:51 UTC — task:e2be8805

• Before: 373 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 30
• Inserted 30 snapshot rows with score=composite_score, market_price and confidence_score logged for audit, evidence_count=len(evidence_for)+len(evidence_against), citations=citations_count, reviewer_count=hypothesis_reviews count
• After: 343 non-archived hypotheses missing snapshots (reduction: 30)
• Same-day duplicates: 0
• 30 new belief_snapshots rows confirmed on 2026-04-26
• Verification: 30 rows inserted today, 0 duplicate hypotheses; sampled rows confirm score=composite_score, market_price and confidence_score logged

2026-04-26 14:36 UTC — task:87b53b03

• Before: 453 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 50
• Inserted 50 snapshot rows with score=composite_score, evidence_count=len(evidence_for)+len(evidence_against), citations=citations_count, reviewer_count=hypothesis_reviews count
• After: 403 non-archived hypotheses missing snapshots (reduction: 50)
• Same-day duplicates: 0
• 50 new belief_snapshots rows confirmed on 2026-04-26

2026-04-26 12:42 UTC — task:e33e3af2 (second cycle)

• Before: 534 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 50
• Inserted 50 snapshot rows with score=composite_score, evidence_count=len(evidence_for)+len(evidence_against), citations=citations_count, reviewer_count=hypothesis_reviews count
• After: 484 non-archived hypotheses missing snapshots (reduction: 50)
• Same-day duplicates: 0
• 50 new belief_snapshots rows confirmed on 2026-04-26

2026-04-26 12:40 UTC — task:e33e3af2

• Improved script: log all inserted hypothesis IDs (was first 5 only); fix dry-run after-count to show simulated reduction
• Before: 584 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 50
• Inserted 50 snapshot rows with score=composite_score, evidence_count=len(evidence_for)+len(evidence_against), citations=citations_count, reviewer_count=hypothesis_reviews count
• After: 534 non-archived hypotheses missing snapshots (reduction: 50)
• Same-day duplicates: 0
• 50 new belief_snapshots rows confirmed on 2026-04-26

2026-04-26 09:26 UTC — task:0ebc6ca1

• Before: 694 non-archived hypotheses missing snapshots in the last 7 days
• Ran python3 scripts/backfill_belief_snapshots.py --limit 50
• Inserted 50 snapshot rows with score=composite_score, evidence_count=len(evidence_for)+len(evidence_against), citations=citations_count, reviewer_count=hypothesis_reviews count
• After: 644 non-archived hypotheses missing snapshots (reduction: 50)
• Same-day duplicates: 0
• 50 new belief_snapshots rows confirmed on 2026-04-26

2026-04-24 14:50 UTC — task:0c46779c

• Reviewed the live PostgreSQL schema via scidex.core.database.get_db() before changing anything.
• Confirmed belief_snapshots currently stores score, evidence_count, citations, and reviewer_count; the generated task wording mentioning market_price and confidence_score is ahead of the live schema.
• Verified the real hypothesis status set is proposed / promoted / debated / open / archived; there is no active status in the current data.
• Measured the live backlog using non-archived hypotheses and a 7-day window: 548 hypotheses were missing recent snapshots before this run.
• Planned implementation for this run: update scripts/backfill_belief_snapshots.py to use the 7-day window, target non-archived hypotheses, default to a batch size of 30, and keep same-day inserts idempotent.

• Executed python3 scripts/backfill_belief_snapshots.py --limit 30.
• Before/after backlog (7-day window, excluding archived / rejected / resolved): 548 -> 518.
• Verified 30 belief_snapshots rows were written on 2026-04-24 and 0 same-day duplicate hypothesis snapshots exist.
• Spot-checked inserted rows against live hypotheses state: stored score matched composite_score, stored evidence_count matched live evidence array counts, and the sampled rows' live market_price / confidence_score values were recorded during execution for audit context.

2026-04-21 - Quest engine template

• Created reusable spec for quest-engine generated belief snapshot backfill tasks.

2026-04-22 — task:4cfdd27b — Backfill 30 hypotheses

• Before count: 679 hypotheses missing belief snapshots (30-day window)
• Inserted 30 new snapshots via direct SQL INSERT with NOT EXISTS guard
• Mapping: score ← composite_score, evidence_count ← len(evidence_for)+len(evidence_against), citations ← citations_count, reviewer_count ← 0
• After count (1-day window): 300 snapshots (up from 270 pre-run)
• Remaining missing (30-day): 649

INSERT INTO belief_snapshots (hypothesis_id, score, evidence_count, citations, reviewer_count, snapshot_date)
SELECT h.id,
  COALESCE(h.composite_score, 0.5),
  COALESCE(jsonb_array_length(COALESCE(h.evidence_for, '[]'::jsonb)), 0) +
  COALESCE(jsonb_array_length(COALESCE(h.evidence_against, '[]'::jsonb)), 0),
  COALESCE(h.citations_count, 0),
  0,
  NOW()
FROM hypotheses h
WHERE NOT EXISTS (
  SELECT 1 FROM belief_snapshots bs
  WHERE bs.hypothesis_id = h.id AND bs.snapshot_date > NOW() - INTERVAL '30 days'
)
ORDER BY h.created_at DESC LIMIT 30;
-- Result: INSERT 0 30

2026-04-21 14:00-14:05 UTC — Backfill execution

• Before count: 830 hypotheses missing belief snapshots (7-day window)
• After count: 730 hypotheses missing belief snapshots (7-day window)
• Reduction: 100 (exceeds 50 target due to concurrent quest_engine runs)
• Created scripts/backfill_belief_snapshots.py with idempotent INSERT using WHERE NOT EXISTS clause
• 50 specific hypothesis IDs received belief_snapshots rows via the script
• Snapshots include: score (composite_score), evidence_count (len(evidence_for) + len(evidence_against)), citations (citations_count), reviewer_count (hypothesis_reviews count)
• Same-day duplicates avoided via conditional INSERT
• Remaining missing (730) is well under the 780 threshold

-- Before/after counts
SELECT COUNT(*) FROM hypotheses h WHERE NOT EXISTS (SELECT 1 FROM belief_snapshots b WHERE b.hypothesis_id = h.id AND b.snapshot_date >= CURRENT_DATE - INTERVAL '7 days');

-- Verify 50 new snapshots with all required fields
SELECT bs.hypothesis_id, bs.score, bs.evidence_count, bs.citations, bs.reviewer_count, bs.snapshot_date
FROM belief_snapshots bs WHERE bs.snapshot_date::date = CURRENT_DATE ORDER BY bs.id DESC LIMIT 50;

-- Verify no duplicates today
SELECT hypothesis_id, COUNT(*) FROM belief_snapshots WHERE snapshot_date::date = CURRENT_DATE GROUP BY hypothesis_id HAVING COUNT(*) > 1;