[Senate] Domain scope enforcement — reject out-of-scope schema proposals

Goal

Prevent schema sprawl by enforcing domain scope on schema proposals. New fields and types
must be relevant to neuroscience, scientific discovery, or the SciDEX platform. An LLM
evaluator assesses proposals for scope compliance before they reach voting.

Acceptance Criteria

- In scope: experimental methodology, biological entities, disease mechanisms,
therapeutic targets, statistical results, data provenance, evidence quality,
scientific metadata, knowledge representation
- Out of scope: social media, marketing, unrelated domains, purely aesthetic metadata

Dependencies

• sen-sg-02-PROP — Proposals system must exist for scope check

Dependents

• None (enrichment task)

Work Log

2026-04-26 — sen-sg-05-SCOP implementation

What was done:

Created scidex/senate/scope_evaluation.py (sen-sg-05-SCOP):

- evaluate_scope(proposal_dict) — LLM-based scope evaluation using llm.complete() with structured JSON response parsing
- Fast precheck with keyword matching before LLM call (avoids LLM cost for obvious cases)
- IN_SCOPE_KEYWORDS (~60 terms): neuroscience, bioinformatics, statistical, etc.
- OUT_OF_SCOPE_KEYWORDS (~20 terms): social media, marketing, crypto, sports, entertainment
- evaluate_scope_from_proposal_id() convenience wrapper
- appeal_scope_rejection() in schema_governance.py for re-evaluation with appeal context
- SCOPE_CRITERIA_IN_SCOPE and SCOPE_CRITERIA_OUT_OF_SCOPE as reusable prompt fragments

Created migrations/125_scope_evaluation.py (originally 114, renamed to avoid conflict with 114_skill_registry_canonical.py):

- Adds to senate_proposals: scope_check_passed (BOOL), scope_evaluation_reason (TEXT),
scope_appealed (BOOL DEFAULT FALSE), scope_appeal_reason (TEXT), scope_appeal_evaluation (TEXT)

Modified scidex/senate/schema_governance.py:

- Removed SQLite-only get_db() override and BASE/DB constants (dead code after PG migration)
- create_schema_proposal() now calls evaluate_scope() before INSERT:
- In-scope proposals → status='voting'
- Out-of-scope → status='rejected', with scope fields populated
- scope_check_passed and scope_evaluation_reason stored on the proposal row
- get_proposal_status() now returns all scope fields
- Added appeal_scope_rejection(proposal_id, appeal_reason) — re-evaluates with appeal context,
reopens voting if appeal succeeds

Modified api_routes/senate.py:

- POST /api/senate/schema-proposal/{proposal_id}/appeal — trigger appeal with justification
- POST /api/senate/schema-proposal/scope-evaluate — dry-run scope eval (no DB write)

Verified behavior:

• In-scope proposals (e.g., "Add statistical_power field to experiments") → in_scope=True
• Out-of-scope proposals (e.g., "Add twitter_sentiment field to papers") → rejected via precheck ("Contains out-of-scope keyword: 'social media'")
• Appeal correctly re-evaluated with proposer's justification; remained rejected when justification invoked marketing/social media terms

2026-04-26 04:30 UTC — Re-verification after rebase

Confirmed on current main (2e7f0af2d):

• Scope evaluation code is NOT on main; only migration 125 rename commit is (2a52d9a16)
• Worktree contains full implementation: scidex/senate/scope_evaluation.py, scidex/senate/schema_governance.py modifications, api_routes/senate.py scope routes, migration 125
• PostgreSQL PGShimConnection with _qmark_to_format translates ? to %s at execute time — safe to use SQLite-style placeholders
• Scope columns verified to exist in DB; scope evaluation columns already present in senate_proposals
• End-to-end test confirmed: in-scope proposals enter voting, out-of-scope rejected at creation time, appeal endpoint works

Tasks using this spec (1)

[Senate] Domain scope enforcement — reject out-of-scope sche

File: sen-sg-05-SCOP_domain_scope_enforcement_spec.md

Modified: 2026-04-28 03:24

Size: 4.4 KB