SciDEX — Task: [Atlas] Meta-analysis support

Pooled effect sizes with inverse-variance weighting, heterogeneity assessment, forest plot data

Completion Notes

Auto-completed by supervisor after successful deploy to main

Git Commits (16)

Squash merge: orchestra/task/atl-ex-0-api-endpoints-for-experiment-browsing-se (7 commits)2026-04-26

Squash merge: atlas/atl-ex-04-QUAL-push (2 commits)2026-04-26

[Atlas] Update spec work log for extraction quality scoring [task:atl-ex-04-QUAL]2026-04-25

[Atlas] Extraction quality scoring and confidence calibration [task:atl-ex-04-QUAL]2026-04-25

Squash merge: orchestra/task/atl-ex-0-meta-analysis-support-aggregate-results (2 commits)2026-04-25

[Atlas] Update spec work log for extraction quality scoring [task:atl-ex-04-QUAL]2026-04-25

[Atlas] Extraction quality scoring and confidence calibration [task:atl-ex-04-QUAL]2026-04-25

[Atlas] Replication tracking: clustering module + /api/experiments/replication/{entity} [task:atl-ex-05-REPL]2026-04-25

Squash merge: orchestra/task/atl-ex-0-build-llm-extraction-pipeline-from-paper (2 commits)2026-04-15

[Atlas] Update spec work log for atl-ex-02-PIPE [task:atl-ex-02-PIPE]2026-04-15

[Atlas] Improve experiment extraction pipeline with schema-aligned prompts, graceful missing data, and quota-aware rate limiting2026-04-15

Squash merge: orchestra/task/atl-ex-0-backfill-188-existing-experiment-artifac (1 commits)2026-04-15

[Atlas] Backfill 188 experiment artifacts with structured metadata [task:atl-ex-07-BKFL]2026-04-15

[Atlas] Auto-link extracted experiments to KG entities [task:atl-ex-03-LINK]2026-04-13

[Docs] Update atl-ex-01-SCHM work log: implementation complete [task:atl-ex-01-SCHM]2026-04-13

[Atlas] Add experiment extraction constants and validate_experiment_metadata() [task:atl-ex-01-SCHM]2026-04-13

Spec File

Goal

Enable automated meta-analysis across structured experiment records. When multiple experiments
report quantitative results for the same claim, compute pooled effect sizes with proper
statistical weighting.

Acceptance Criteria

☑ meta_analysis.py module with compute_pooled_effect(experiment_ids) function

☑ Fixed-effects and random-effects models (inverse-variance weighting)

☑ Heterogeneity assessment (I-squared, Q statistic)

☑ Forest plot data generation (for visualization)

☑ Meta-analysis results registered as analysis artifacts with provenance to constituent experiments

☑ Publication bias assessment (funnel plot data) when N >= 5 studies

☑ Results linked back to relevant hypotheses (update hypothesis evidence strength)

Dependencies

atl-ex-05-REPL — Replication groups define the experiment sets for meta-analysis

Dependents

Evidence chains (b5298ea7) — Meta-analysis results are high-trust evidence entries

Work Log

2026-04-25 12:30 UTC — Slot 0

Staleness review: task created recently, no prior commits addressing it found on main
Checked replication_clustering.py (effect_sizes JSON field, cluster_experiments), confirmed experiments table exists
Created scidex/atlas/meta_analysis.py with all 7 acceptance criteria implemented
Statistical functions tested: fixed/random effects pooling, Q/I², DerSimonian-Laird tau², p-value, forest + funnel plots
Committed 5babf2daa and pushed branch
Result: Done — meta-analysis module shipped

Payload JSON

{
  "requirements": {
    "coding": 7,
    "reasoning": 7,
    "analysis": 8
  }
}

Sibling Tasks in Quest (Experiment Extraction) ↗

○[Atlas] CI: Verify experiment extraction quality metrics and extract from new papersP88

✓[Atlas] Define experiment extraction schemas per experiment typeP93

✓[Atlas] Auto-link extracted experiments to KG entitiesP93

✓[Atlas] Backfill 188 existing experiment artifacts with structured metadataP93

✓[Atlas] Build LLM extraction pipeline from paper abstracts and full textP92

✓[Atlas] Extraction quality scoring and confidence calibrationP88

✓[Atlas] API endpoints for experiment browsing, search, and filteringP87

✓[Atlas] Replication tracking — match experiments testing same hypothesisP86

[Atlas] Meta-analysis support — aggregate results across experiments done analysis:8 coding:7 reasoning:7