[Senate] Quality gates for all artifact types (not just hypotheses) done coding:8 safety:9

← Artifact Quality Markets
Type-specific gates: experiment (methods, results, source), KG edge (evidence, entity, relation), model, dataset, paper

Completion Notes

Auto-completed by supervisor after successful deploy to main

Git Commits (14)

[Senate] Update quality dashboard spec work log [task:exch-qm-07-DASH]2026-04-26
[Senate] Add unified quality dashboard routes in api.py [task:exch-qm-07-DASH]2026-04-25
Squash merge: orchestra/task/exch-qm--debate-to-market-feedback-loop-debate-ou (3 commits)2026-04-26
Squash merge: orchestra/task/exch-qm--artifact-lifecycle-state-machine-draft-t (16 commits)2026-04-26
[Exchange] Merge retry: re-trigger after transient gate lock [task:exch-qm-04-STAK]2026-04-25
[Exchange] Reputation staking — agents stake believability on quality claims [task:exch-qm-04-STAK]2026-04-25
[Verify] Quality gates task already complete on main [task:exch-qm-05-GATE]2026-04-25
Squash merge: orchestra/task/exch-qm--quality-gates-for-all-artifact-types-not (1 commits)2026-04-25
[Exchange] Add GET /api/market/participants leaderboard route [task:exch-qm-02-PART]2026-04-25
Squash merge: orchestra/task/exch-qm--market-participant-agents-with-evaluatio (1 commits)2026-04-25
[Exchange] Extend market pricing to all artifact types — type-specific price factors + compute/update functions [task:exch-qm-01-MEXT]2026-04-24
[Specs] SciDEX economy — holistic design for artifact generation, valuation, and showcase2026-04-24
[Senate] Holistic prioritization run 2: quest fixes + 3 new CI tasks [task:b4c60959-0fe9-4cba-8893-c88013e85104]2026-04-06
[Senate] Holistic prioritization: 6 tasks created for uncovered P88-P95 quests [task:b4c60959-0fe9-4cba-8893-c88013e85104]2026-04-06
Spec File

Goal

Extend quality gates from hypothesis-only to ALL artifact types. Each type has gates
appropriate to its nature. Gates are the minimum bar for market listing — they prevent
garbage from entering the quality market in the first place.

Current State

  • quality_gates.py: Code quality gates (compilation, undefined vars, DDL safety)
  • post_process.py: 3 hypothesis gates (evidence, score, specificity)
  • data_quality_snapshots: 10 metrics tracking hypothesis quality
  • No gates for experiments, KG edges, models, datasets, figures, notebooks

Type-Specific Quality Gates

Experiment Gates

  • Methods gate: Has experiment_type, model_system, and methods_summary (not null)
  • Results gate: Has at least 1 measurement with p-value or effect size
  • Source gate: Has paper_pmid linking to a registered paper
  • Confidence gate: extraction_confidence >= 0.5

KG Edge Gates

  • Evidence gate: evidence_strength > 0 and evidence_sources not empty
  • Entity gate: source_id and target_id exist in wiki_pages or known entity list
  • Relation gate: relation is from controlled vocabulary

Model Gates

  • Specification gate: Has model_type, equations/architecture, and parameters
  • Validation gate: Has at least 1 validation metric (accuracy, R-squared, etc.)
  • Data gate: Training data provenance documented

Dataset Gates

  • Schema gate: Column names and types documented
  • Provenance gate: Source URL or generation method documented
  • Size gate: row_count > 0

Paper Gates (External)

  • Citation gate: Has valid PMID or DOI
  • Abstract gate: Has non-empty abstract

General Gates (All Types)

  • Title gate: title length >= 10 characters, not placeholder
  • Metadata gate: metadata is valid JSON, not null
  • Schema compliance gate: metadata validates against registered schema (if available)

Acceptance Criteria

artifact_quality_gates.py module with per-type gate functions
run_gates(artifact_id) → returns list of (gate_name, pass/fail, details)
☑ Gates integrated into register_artifact() — run automatically on creation
☑ Gate mode: enforce (reject on fail), warn (log but allow), skip
☑ Default: warn during rollout, switch to enforce per type as gates stabilize
☑ Gate results logged to artifact_gate_results table (new, optimised for per-type queries)
☑ Failed gates block lifecycle transition from draft to listed (can_promote_to_listed)
☑ API: GET /api/artifacts/{id}/gates — gate results for an artifact
☑ API: GET /api/quality-gates/summary — pass/fail rates by type
☑ Dashboard shows gate health per artifact type (Senate quality-gates page)

Dependencies

  • None (can start immediately, works with existing artifact_registry)

Dependents

  • exch-qm-01-MEXT — Market listing depends on gate passage
  • exch-qm-03-LIFE — Lifecycle draft→listed transition requires gate pass
  • sen-sg-03-VALD — Schema validation is one gate

Work Log

2026-04-25 — Slot 40 (resumed by Slot N)

  • Read AGENTS.md, spec, and existing codebase state
  • Found scidex/atlas/artifact_quality_gates.py already written by prior agent run (untracked)
  • Found migrations/123_artifact_gate_mode_column.py already written (untracked)
  • Found artifact_gate_results table already exists in DB with gate_mode column
  • Remaining work: integrate gates into register_artifact(), add API routes, add dashboard section
  • Added run_gates() call in scidex/atlas/artifact_registry.py:register_artifact() (warn mode, non-blocking)
  • Fixed migration savepoint handling for "column already exists" PostgreSQL edge case
  • Added GET /api/artifacts/{artifact_id}/gates route to api.py
  • Added GET /api/quality-gates/summary route to api.py
  • Added per-type gate health section to /senate/quality-gates dashboard
  • All files pass python3 -m py_compile; gates evaluated correctly on test data
  • Committed and pushed to task branch

2026-04-26 — Slot N (verification pass)

  • Confirmed all acceptance criteria satisfied on origin/main (commit 06a6948c7)
  • Verified via direct import and unit test: all gate functions work correctly
  • artifact_quality_gates.py (582 lines): 7 artifact types with per-type gates, run_gates, can_promote_to_listed, get_gate_results, get_gate_summary
  • artifact_gate_results table: schema confirmed with all required columns (gate_mode, artifact_type, etc.)
  • register_artifact() integration confirmed at line 475-476 in artifact_registry.py
  • API routes confirmed at lines 22707 (/api/artifacts/{id}/gates) and 5246 (/api/quality-gates/summary)
  • Dashboard confirmed at /senate/quality-gates with per-type gate health section
  • Task is done — no additional work needed

Payload JSON
{
  "requirements": {
    "coding": 8,
    "safety": 9
  }
}

Sibling Tasks in Quest (Artifact Quality Markets) ↗

Task Dependencies

↓ Referenced by (downstream)