[Arenas] Fix King of the Hill tournament driver — diagnose 22d staleness, repair, verify resumed throughput

← All Specs

Goal

The King of the Hill tournament driver (607558a9) has been stale for 22+ days (last run ~2026-04-06).
This driver runs daily hypothesis prediction tournaments that rank hypotheses by predictive accuracy —
it is the platform's core hypothesis-evaluation feedback loop. Without it, hypothesis market prices
are frozen, tournament leaderboards are stale, and the predictive-accuracy ranking that differentiates
SciDEX from a static database is completely absent.

Frame: diagnose + fix + verify throughput. Not: run the tournament manually.

Why the tournament driver is critical

The tournament is the mechanism by which:

  • Hypothesis composite scores get updated from competitive evaluation results
  • Market prices reflect prediction accuracy (not just debate sentiment)
  • Top hypotheses earn higher token allocations from the funding allocator
  • The platform demonstrates its core scientific value: ranking ideas by predictive accuracy
  • 22 days without a tournament run = 22 days of frozen hypothesis rankings = the platform's
    primary value proposition is non-functional.

    Acceptance Criteria

    ☑ Root cause of 22d staleness identified and documented
    ☑ Fix committed (not a workaround)
    ☑ One successful tournament run verified post-fix
    ☑ At least 10 hypotheses ranked/scored in the test run
    ☑ Driver ready to resume its daily schedule
    ☑ Dry-run bug found and fixed (was making real DB writes, settling tournaments with 0 matches)

    Root Cause Analysis

    Primary root cause (historical, fixed 2026-04-20 by task-607558a9):
    PostgreSQL migration bugs in scidex/core/database.py, scidex/exchange/tournaments.py, scidex/senate/judge_arena.py, and scidex/senate/evolution.py caused the driver to
    fail silently during the SQLite→PostgreSQL transition (2026-04-19). The driver ran
    successfully again on 2026-04-20 after those fixes (6 domains) and on 2026-04-27 (19 domains).

    Secondary bug (fixed 2026-04-28 by this task): scripts/ci_daily_tournament.py dry-run mode had a bug in the pre-seeded 'open' tournament path.
    When --dry-run was invoked:

  • register_entrant was called (real DB writes)
  • start_tournament was skipped (correct)
  • Code fell through to judge_pending_round → failed with "status=open"
  • Then settle_tournament was called and SUCCEEDED — marking tournaments 'complete' with 0 judged matches
  • This caused 4 pre-seeded 2026-04-28 tournaments to be prematurely settled during a diagnostic dry-run.

    No automated cron: The driver has no system cron or systemd timer. It runs when
    Orchestra agents claim task 607558a9. This is by design — agents run it on demand.

    Fix Applied

    scripts/ci_daily_tournament.py: In the pre-seeded 'open' tournament branch, added an
    early return result when --dry-run is active (with informational log). This prevents
    real DB writes (entrant registration) and premature tournament settling. Also simplified
    the non-dry-run path by removing the now-redundant if not dry_run: wrapper.

    The 4 wrongly-settled 2026-04-28 tournaments were reset via direct DB update: status='open', current_round=0, entrant final_rank=NULL, prize_awarded=0.

    Work Log

    Created 2026-04-28 by task generator cycle 2

    Tournament driver 607558a9 stale 22+ days. This is the platform's core hypothesis-ranking
    mechanism. Without it, the "prediction markets" layer is purely static. Diagnose + fix, not
    manually run.

    2026-04-28 11:49–11:56 UTC — Worktree task-a3018033

    Diagnosis:

    • Reviewed task 607558a9 spec (607558a9_0f9_spec.md): driver ran on 2026-04-20 (6 domains,
    PostgreSQL migration fixes applied) and 2026-04-27 (19 domains). The 22d staleness period
    was ~2026-04-06 to 2026-04-20, during the PostgreSQL migration window.
    • Today (2026-04-28): KOTH-als and KOTH-alzheimers were already complete (run by a prior agent at
    ~01:34–01:36 UTC). 4 domains pre-seeded as 'open' from yesterday's run. 28 domains had no
    2026-04-28 tournament yet.
    • No system cron exists; driver depends on Orchestra agents picking up task 607558a9.
    Bug discovery and fix:
    • Ran --dry-run to validate driver startup; discovered pre-seeded 'open' tournaments were being
    settled with 0 judged matches (dry-run not dry!).
    • Fixed scripts/ci_daily_tournament.py: added early return result in pre-seeded 'open' path
    when dry_run=True; removed redundant if not dry_run: wrapper around start_tournament.
    • Reset 4 wrongly-settled tournaments (KOTH-neuropharmacology, KOTH-protein_biochemistry,
    KOTH-proteomics, KOTH-synaptic_biology) back to status='open' via direct DB update.

    Test run (verification):

    • Ran python3 scripts/ci_daily_tournament.py --domain neurodegeneration --rounds 4 --top 20
    • Result: tournament t-387081b05d6f created, 20 entrants registered, 4/4 rounds judged
    (40 total matches), 20 hypotheses ranked ✓, 3 evolutionary variants spawned,
    KOTH-neurodegeneration-2026-04-29 seeded (13 entrants).
    • Winner: h-2600483e (Elo=2105), Runner-up: h-fc1cd0d4bd (Elo=2096), 3rd: h-9e9fee95 (Elo=1856)
    Dry-run fix verified:
    • Re-ran --dry-run --domain neurodegeneration: correctly skipped (already complete), no DB writes ✓
    Final DB counts post-fix:
    • elo_matches=8336, elo_ratings=3693, tournaments=112 (95 complete),
    artifact_variants=280, hypotheses=1914

    Committed: fe9cbc80e — scripts/ci_daily_tournament.py dry-run bug fix + this spec.

    File: quest_arenas_tournament_fix_spec.md
    Modified: 2026-04-28 18:12
    Size: 5.5 KB