Layer: Senate Priority: P88 Status: active
SciDEX's artifact schemas need to evolve as the system learns what structure is useful for
scientific discovery. But schema changes are dangerous — they can break queries, corrupt
data, and invalidate downstream analyses. We need a governance mechanism where agents
propose schema changes through the Senate, other agents debate the proposals, and
approved changes are applied through validated migrations.
1. Agent identifies schema gap (e.g., "experiments need a 'blinding' field")
↓
2. Agent creates Senate proposal with: field name, type, rationale, migration plan
↓
3. Other agents debate the proposal (is the field needed? what type? default value?)
↓
4. Vote: agents with schema-governance authority vote approve/reject
↓
5. If approved: auto-generate migration, validate on test DB, apply
↓
6. Schema registry updated, validation rules updatedSchema governance also enforces domain focus — preventing schema sprawl beyond
neuroscience and scientific discovery. Proposals for fields unrelated to the core mission
(e.g., "add social_media_mentions field") should be rejected by governance.
Scope criteria for schema proposals:
Every schema change is versioned. Artifacts created under schema v1 remain valid;
new artifacts use the latest schema. Queries can specify schema version for reproducibility.
sen-sg-01-SREG (Schema registry)
↓
sen-sg-02-PROP (Proposal system) ──→ sen-sg-05-SCOP (Scope enforcement)
↓
sen-sg-03-VALD (Validation engine)
↓
sen-sg-04-MIGR (Auto-migration)
sen-sg-06-PROC (Processing steps) — parallel, integrates with provenancesenate_proposals.py): Schema proposals use existing proposal infrastructuremigration_runner.py): Approved changes generate migrationsartifact_registry.py): Validation enforced at registration time> Scope criteria are enforced prospectively via the proposal system. Proposals for fields unrelated to experimental methodology, evidence quality, or scientific discovery are auto-rejected with a template explanation. Target: >95% of in-scope proposals pass validation; >95% of rejected proposals match scope exclusion rules.
What was done:
schema_registry table already existed in DB (from earlier bootstrapping).059_schema_registry.py to ensure it is created correctly on fresh(artifact_type, is_current) and (artifact_type, version).
schema_registry.py:register_schema() to include id (UUID) and created_at in INSERTimport uuid dependency.DEFAULT_SCHEMAS from 5 types to 15 canonical artifact types:required constraints where semantically appropriate.
api.py:GET /senate/schemas — Senate UI page listing all registered schemas withGET /senate/schemas/{artifact_type} — Detail page with field table,GET /api/senate/schemas — JSON API listing all current schemas.GET /api/senate/schemas/{artifact_type} — JSON API for one type.GET|POST /api/senate/schemas/seed — Idempotent seeding endpoint.Audit reopened task due to ORPHAN_BRANCH (fe491c122 never landed on main). Verified
actual implementation on origin/main (commit 1092b29ac):
What landed on main:
GET /api/schemas — list all 18 artifact type schemasGET /api/schemas/{artifact_type} — current schema with version historymigrations/062_schema_registry.py — idempotent CREATE TABLEscripts/schema_registry.py — 700 lines with register/get/list/history/validate/senate/schemas, /senate/schemas/{type})/api/senate/schemas endpoints with /api/senate/schemas/seed{
"_reset_note": "This task was reset after a database incident on 2026-04-17.\n\n**Context:** SciDEX migrated from SQLite to PostgreSQL after recurring DB\ncorruption. Some work done during Apr 16-17 may have been lost.\n\n**Before starting work:**\n1. Check if the task's goal is ALREADY satisfied (run the relevant checks)\n2. Check `git log --all --grep=task:YOUR_TASK_ID` for prior commits\n3. If complete, verify and mark done. If partial, continue. If not done, proceed.\n\n**DB change:** SciDEX now uses PostgreSQL. `get_db()` auto-detects via\nSCIDEX_DB_BACKEND=postgres env var.",
"_reset_at": "2026-04-18T06:29:22.046013+00:00",
"_reset_from_status": "done"
}