SciDEX — Task: [Senate] Implement daily cost budgets with auto-pa

## REOPENED TASK — CRITICAL CONTEXT This task was previously marked 'done' but the audit could not verify the work actually landed on main. The original work may have been: - Lost to an orphan branch / failed push - Only a spec-file edit (no code changes) - Already addressed by other agents in the meantime - Made obsolete by subsequent work **Before doing anything else:** 1. **Re-evaluate the task in light of CURRENT main state.** Read the spec and the relevant files on origin/main NOW. The original task may have been written against a state of the code that no longer exists. 2. **Verify the task still advances SciDEX's aims.** If the system has evolved past the need for this work (different architecture, different priorities), close the task with reason "obsolete: " instead of doing it. 3. **Check if it's already done.** Run `git log --grep=''` and read the related commits. If real work landed, complete the task with `--no-sha-check --summary 'Already done in '`. 4. **Make sure your changes don't regress recent functionality.** Many agents have been working on this codebase. Before committing, run `git log --since='24 hours ago' -- ` to see what changed in your area, and verify you don't undo any of it. 5. **Stay scoped.** Only do what this specific task asks for. Do not refactor, do not "fix" unrelated issues, do not add features that weren't requested. Scope creep at this point is regression risk. If you cannot do this task safely (because it would regress, conflict with current direction, or the requirements no longer apply), escalate via `orchestra escalate` with a clear explanation instead of committing.

Completion Notes

Auto-completed by supervisor after successful deploy to main

Git Commits (2)

[Senate] Implement daily cost budgets with auto-pause2026-04-20

Spec File

[Senate] Implement daily cost budgets with auto-pause

Quest: Resource Governance Priority: P4 Status: done

Goal

Set a daily budget for LLM API costs (default: $50/day). When the budget is reached, pause new analyses and enrichment until the next day. Track cumulative daily spend in real-time and display on Senate dashboard.

Acceptance Criteria

☑ DAILY_COST_BUDGET config (default: $50)

☑ Real-time cost tracking using resource_usage table

☑ Auto-pause when daily budget reached (no new analyses started)

☑ Warning at 80% budget consumed

☑ Budget resets at midnight UTC

☑ Senate dashboard shows budget utilization gauge

☑ Override: SCIDEX_BUDGET_OVERRIDE=1 to bypass for critical analyses

Approach

Create daily_budget.py with BudgetChecker class

Query resource_usage for today's cumulative cost

Add check in orchestrator before starting analysis

Add check in enrichment scripts before making Bedrock calls

Add budget gauge to Senate dashboard

Add warning logging at 80% threshold

Dependencies

scidex.core.database (PostgreSQL), resource_usage table with llm_tokens rows

Dependents

orchestrator (agent.py), Senate dashboard (api.py)

Work Log

2026-04-20T22:45:00Z — Implementation completed

Created scidex/senate/daily_budget.py (230 lines) with BudgetChecker class:

- DAILY_COST_BUDGET env var (default: $50/day)
- SCIDEX_BUDGET_OVERRIDE=1 bypasses budget for critical analyses
- Queries resource_usage table for today's cumulative LLM token cost
- Uses model-specific pricing table to compute cost from token amounts
- 80% warning threshold logged (once per day per checker instance)
- can_start_analysis() returns False when budget exceeded
- Budget resets at midnight UTC (by nature of querying created_at >= CURRENT_DATE)

Modified agent.py:

- Added budget check in process_queue() before calling start_queued_analysis
- Added budget check in run_single() before calling run_debate; queues analysis if budget exceeded
- Both checks wrap in try/except — failure to check budget doesn't block analysis

Modified api.py (Senate dashboard):

- Added daily_cost_budget_section to Senate dashboard page (after task effectiveness section)
- Shows: budget amount, today's spend, utilization %, can-start status
- Visual progress bar gauge with color coding (green/yellow/red)
- Warning messages for 80%+ and budget-exceeded states
- Override active indicator when SCIDEX_BUDGET_OVERRIDE=1

Payload JSON

{
  "requirements": {
    "coding": 7,
    "reasoning": 6
  }
}

Sibling Tasks in Quest (Resource Governance) ↗

✓[Senate] Runaway-agent circuit breaker - rate-limit artifact creationP94

✓[Senate] Egress allowlist firewall for sandboxed analysesP92

✓[Senate] Link 50 isolated artifacts into the governance graphP80

✓[Senate] Implement hard memory and CPU limits for analysis processesP5

✓[Senate] Add concurrent analysis limits and queue managementP5

✓[Senate] Build real-time resource monitoring dashboardP4

✓[Senate] Add process watchdog that kills runaway analyses and orphaned processesP4

✓[Senate] Design executor plugin architecture for external compute scalingP3

[Senate] Implement daily cost budgets with auto-pause done coding:7 reasoning:6