[Forge] Define reproducibility capsule manifest and verification schema

[Forge] Define reproducibility capsule manifest and verification schema

Goal

Introduce a canonical SciDEX capsule manifest for analyses and registered artifacts. The manifest should describe code revision, environment lock, runtime image digest, input artifacts, output artifacts, command entrypoints, checksums, and verification status in a machine-readable form that can be versioned and compared over time.

This is the core contract that later execution, export, and credit systems depend on.

Acceptance Criteria

Approach

Design a repro_capsule.json schema inspired by OCI digests, RO-Crate metadata, and Code Ocean-style capsule semantics.

Include fields for git commit, tree hash, environment lockfile hash, runtime image digest, artifact digests, command entrypoints, and verification outcomes.

Add a small validation/helper module so manifests are generated consistently.

Backfill one real analysis as a reference example.

Dependencies

• docs/planning/specs/quest_reproducible_analysis_capsules_spec.md
• docs/planning/specs/a17-18-VERS0001_version_tracking_schema_spec.md

Dependents

• Runtime capture and verification
• Artifact registry version pinning
• RO-Crate export
• ad284a00-a73c-44bc-aaee-e902586bd313 — git-backed artifact catalog
• c98d7e1e-cf0a-4807-a7bc-291855141d3a — runtime capture and verification
• 3ac441ed-e3fc-494e-96b9-c0df1f415881 — RO-Crate export

Work Log

2026-04-10 18:06 PT — Slot 57 (Minimax)

• Re-ran backfill script to update example_capsule_manifest.json (timestamps refresh on each run)
• Re-committed example_capsule_manifest.json with fresh capsule ID, timestamps, and verification signature
• Verified: schema validates, module loads, backfill script runs cleanly
• orchestra sync push blocked by SQLite DB error (unrelated to code changes — Orchestrator DB access issue)
• All acceptance criteria met, push deferred

2026-04-10 19:50 PT — Slot 58 (Minimax)

• Committed refreshed manifest (re-run backfill produced new timestamps/signatures)
• Direct git push to origin succeeded
• Task complete: pushed 6 commits for task 65ac9e7d to origin

2026-04-10 11:00 PT — Slot 56 (Minimax)

• Task ID adf2fe4e-f8e8-4eef-a939-e341ee0b278e not found in Orchestra database (worktree was previously used by task 8a4bcadf)
• Picked up this task (65ac9e7d-eb54-4243-818c-2193162a6c45) as open work
• Created repro_capsule_schema.json — formal JSON schema for reproducibility capsule manifests

- Defines all required/optional fields per the task acceptance criteria
- Includes artifact_ref and verification_result sub-schemas
- Schema ID: https://sciodex.ai/schema/repro-capsule/v1

• Created capsule_validator.py — validation helper module

- validate_manifest() — validates manifests against the JSON schema
- validate_environment_digest() — computes deterministic SHA256 for environments
- validate_capsule_creation_params() — validates params before capsule creation
- generate_example_manifest() — generates example capsules for testing
- Uses jsonschema library for schema validation

• Created backfill_capsule_example.py — backfill reference script

- Demonstrates capsule creation for real analysis SDA-2026-04-01-gap-001
- Validates and saves example manifest to example_capsule_manifest.json

• Created example_capsule_manifest.json — reference capsule manifest

- Demonstrates full manifest structure with real analysis data
- Capsule ID: capsule-c28a01fc-f10c-414e-8a2b-d4141639eaab
- Environment digest: 670214570e4420056ca98c03a78cdbd5a9d8c40cdd5a1667c7e1f29f3a5b0eee

• Validation tested: All manifests pass schema validation
• Files created: repro_capsule_schema.json, capsule_validator.py, backfill_capsule_example.py, example_capsule_manifest.json

2026-04-10 09:25 PT — Codex

• Created initial spec for the capsule manifest and verification schema.

2026-04-10 09:27 PT — Codex

• Created the live Orchestra task and attached the real task id.

2026-04-12 — Sonnet 4.6 (task closure)

• Verified all deliverables present on origin/main: repro_capsule_schema.json, capsule_validator.py, example_capsule_manifest.json, archive/oneoff_scripts/backfill_capsule_example.py
• Confirmed all acceptance criteria met: schema validates, module imports cleanly, example manifest covers TREM2 DAM microglia analysis with code/env/artifact digests, parent versioning, and verification metadata
• Closing task via Orchestra

Verification — 2026-04-16T09:00:00Z

Result: PARTIAL Verified by: claude-haiku-4-5 via task 65ac9e7d-eb54-4243-818c-2193162a6c45

Tests run

Target	Command	Expected	Actual	Pass?
Schema exists at root	ls repro_capsule_schema.json	file exists	file exists	✓
Schema ID	JSON parse + check $id	https://sciodex.ai/schema/repro-capsule/v1	https://sciodex.ai/schema/repro-capsule/v1	✓
validate_environment_digest	Python import + call	returns 64-char hex	returns 64-char hex	✓
validate_manifest via direct jsonschema	Load schema + manifest, validate	valid=True	valid=True	✓
validate_manifest via module	from scidex.forge.capsule_validator import validate_manifest	no error	FileNotFoundError (before fix)	✗
Backfill script on main	git ls-tree origin/main -- scripts/archive/oneoff_scripts/backfill_capsule_example.py	file exists	file exists (archived)	✓
Example manifest references real analysis	Check inputs[0].id	SDA-2026-04-01-gap-001	SDA-2026-04-01-gap-001	✓

Issues Found

Schema path broken after Senate package refactor (75f3726af)

- The Senate refactor moved capsule_validator.py to scidex/forge/ but left repro_capsule_schema.json at repo root
- scidex/forge/capsule_validator.py:_SCHEMA_PATH = Path(__file__).parent / "repro_capsule_schema.json" resolves to scidex/forge/repro_capsule_schema.json which didn't exist
- Fix applied: Copied schema to scidex/forge/repro_capsule_schema.json and committed (a08257b80)
- After fix: validate_manifest() works correctly, returns (True, [])

Attribution

The current passing state is produced by:

• 173489e86 — Original Forge commit: repro_capsule schema, validator, backfill example
• 75f3726af — Senate package refactor (introduced schema path bug)
• a08257b80 — Fix: move schema to forge package [task:65ac9e7d]

Notes

• The backfill script (scripts/archive/oneoff_scripts/backfill_capsule_example.py) was archived by the Senate code health sweep but remains on main
• The root-level capsule_validator.py is a backward-compat shim that delegates to scidex.forge.capsule_validator
• All acceptance criteria are met: schema validates, functions work, example manifest is complete
• The PARTIAL result reflects that a fix was needed after the Senate package refactor broke the original implementation

Tasks using this spec (1)

[Forge] Define reproducibility capsule manifest and verifica

File: repro_capsule_manifest_schema_spec.md

Modified: 2026-04-28 01:39

Size: 7.5 KB