[Exchange] Back-fill Resource Metrics for Hypotheses

[Exchange] Back-fill Resource Metrics for Hypotheses

Task ID: q02-e2-E3B0F52A Priority: P88 Status: In Progress

Goal

Calculate and populate resource usage metrics (tokens_used, kg_edges_generated, citations_count) for all 139 existing hypotheses. These metrics are essential for the Resource Economics quest and enable cost-benefit analysis of hypothesis generation.

Acceptance Criteria

Approach

Check current database schema for hypotheses table

Create migration if columns are missing

Query all hypotheses and their associated analyses

For each hypothesis:

- Sum tokens_used from debate_rounds for that analysis_id
- Count knowledge_edges where analysis_id matches
- Parse and count PMIDs in evidence_for and evidence_against

Update hypotheses table with calculated values

Verify results with sample queries

Technical Details

Data Sources:

• debate_rounds.tokens_used → aggregate by session_id → map to analysis_id
• knowledge_edges.analysis_id → count edges per analysis
• hypotheses.evidence_for/against → parse PMID references

SQL Approach:

-- Tokens used per analysis
SELECT analysis_id, SUM(tokens_used) 
FROM debate_sessions ds 
JOIN debate_rounds dr ON ds.session_id = dr.session_id 
GROUP BY analysis_id

-- KG edges per analysis
SELECT analysis_id, COUNT(*) 
FROM knowledge_edges 
GROUP BY analysis_id

-- Citations: parse JSON/text fields for PMIDs

Work Log

2026-04-02 03:00 PT — Slot 0

• Started task: Backfill resource metrics for 142 hypotheses
• Created spec file with approach
• Checked schema: columns already exist (tokens_used, kg_edges_generated, citations_count)
• Created backfill_hypothesis_metrics.py script
• Fixed schema references (debate_sessions.id vs session_id)
• Fixed PMID extraction to handle JSON format in evidence fields
• Ran backfill successfully
• Results:

- 142 total hypotheses
- 118 (83%) with tokens_used data, totaling ~997K tokens
- 7 (4%) with kg_edges_generated data, totaling 301 edges
- 130 (91%) with citations_count data, totaling 549 citations

• Updated 138 hypotheses in final run
• Task complete: All resource metrics populated

File: q02-e2-E3B0F52A_spec.md

Modified: 2026-04-25 17:55

Size: 2.5 KB