[Senate] Implement daily cost budgets with auto-pause
Quest: Resource Governance
Priority: P4
Status: done
Goal
Set a daily budget for LLM API costs (default: $50/day). When the budget is reached, pause new analyses and enrichment until the next day. Track cumulative daily spend in real-time and display on Senate dashboard.
Acceptance Criteria
☑ DAILY_COST_BUDGET config (default: $50)
☑ Real-time cost tracking using resource_usage table
☑ Auto-pause when daily budget reached (no new analyses started)
☑ Warning at 80% budget consumed
☑ Budget resets at midnight UTC
☑ Senate dashboard shows budget utilization gauge
☑ Override: SCIDEX_BUDGET_OVERRIDE=1 to bypass for critical analyses
Approach
Create daily_budget.py with BudgetChecker class
Query resource_usage for today's cumulative cost
Add check in orchestrator before starting analysis
Add check in enrichment scripts before making Bedrock calls
Add budget gauge to Senate dashboard
Add warning logging at 80% thresholdDependencies
scidex.core.database (PostgreSQL), resource_usage table with llm_tokens rows
Dependents
- orchestrator (
agent.py), Senate dashboard (api.py)
Work Log
2026-04-20T22:45:00Z — Implementation completed
- Created
scidex/senate/daily_budget.py (230 lines) with BudgetChecker class:
-
DAILY_COST_BUDGET env var (default: $50/day)
-
SCIDEX_BUDGET_OVERRIDE=1 bypasses budget for critical analyses
- Queries
resource_usage table for today's cumulative LLM token cost
- Uses model-specific pricing table to compute cost from token amounts
- 80% warning threshold logged (once per day per checker instance)
-
can_start_analysis() returns False when budget exceeded
- Budget resets at midnight UTC (by nature of querying
created_at >= CURRENT_DATE)
- Added budget check in
process_queue() before calling
start_queued_analysis - Added budget check in
run_single() before calling
run_debate; queues analysis if budget exceeded
- Both checks wrap in try/except — failure to check budget doesn't block analysis
- Modified
api.py (Senate dashboard):
- Added
daily_cost_budget_section to Senate dashboard page (after task effectiveness section)
- Shows: budget amount, today's spend, utilization %, can-start status
- Visual progress bar gauge with color coding (green/yellow/red)
- Warning messages for 80%+ and budget-exceeded states
- Override active indicator when
SCIDEX_BUDGET_OVERRIDE=1