[Exchange] Replication contests with cash bounties for human labs done

← Pre-Registration & Replication
Wet-lab replication contests with cash bounties, ORCID claim flow, 3-reviewer resolution, partial-replication payouts.

Completion Notes

Auto-completed by supervisor after successful deploy to main

Git Commits (5)

Squash merge: orchestra/task/b4e04fba-triage-50-failed-tool-calls-by-skill-and (95 commits) (#1011)2026-04-27
[Verify] Replication contests — already resolved [task:5633cca8-ecd7-413e-b854-cb4678ede674] (#936)2026-04-27
[Verify] Replication contests — already resolved [task:5633cca8-ecd7-413e-b854-cb4678ede674] (#936)2026-04-27
Squash merge: orchestra/task/5633cca8-replication-contests-with-cash-bounties (2 commits) (#790)2026-04-27
Update work log and mark criteria complete [task:5633cca8]2026-04-27
Spec File

Effort: thorough

Goal

The platform's q-preregistration quest tracks pre-registered
predictions but doesn't offer monetary incentives for actual
replication by physical labs. SciDEX's bounty infrastructure
(scidex/exchange/bounty_escrow.py) already supports $5K-$960K
challenge payouts for AI agents — extend it to **replication
contests** that solicit wet-lab teams to reproduce a published
finding linked to a high-stakes hypothesis.

This is the bridge from autonomous discovery to human-validated
science: a researcher anywhere can claim a contest, run the wet
work, upload protocol + raw data + analysis notebook, and unlock
the bounty after a peer-review pass.

Acceptance Criteria

☑ New module scidex/exchange/replication_contests.py:
- create_contest(hypothesis_id, target_finding_pmid,
bounty_usd, deadline_days, accepted_methods: list[str])

→ contest id; uses existing bounty_escrow.lock_funds.
- claim_contest(contest_id, claimant_orcid, lab_affiliation,
protocol_artifact_id)
— only one open claim per contest;
15-day soft hold before accepting another claimant.
- submit_replication(contest_id, claimant_orcid, results_artifact_id,
raw_data_url, statistician_review_artifact_id) -> dict

— locks the submission; spawns 3 reviewers (2 humans from
verifier registry + 1 statistician skill).
- resolve_contest(contest_id) -> dict — pays bounty if
≥2/3 reviewers vote replicated or partially_replicated;
partial replication pays 60%.
☑ New tables migration: replication_contests,
replication_claims, replication_submissions,
replication_reviews with FK chain.
GET /replication-contests lists open contests sorted by
bounty descending; filter by disease, by hypothesis layer,
by deadline soonness. Each card shows the target PMID
title, bounty USD, days remaining, current claimant (if
any), and accepted methods.
GET /replication-contests/{id} shows full target finding,
protocol acceptance criteria, claim form (ORCID gated),
review history if resolved.
POST /api/replication-contests/{id}/claim and
POST /api/replication-contests/{id}/submit enforce ORCID
auth + the open-claim invariant.
☑ Reviewer assignment uses human_expert_reviews reputation
from q-crowd-expert-review-queue; statistician skill
review uses the existing statistician registered tool
skill (verified by tools.py registration).
☑ Contest creation seeded for the top-10 high-Elo
hypotheses with replication-relevant Wave-1 markers
(search experiment_extractor.py for findings with
replication_status='unverified').
☑ Pytest: end-to-end happy path (create → claim → submit →
reviews vote replicated → bounty paid); plus partial-
replication (60% payout) and timeout (bounty returned).

Approach

  • Read scidex/exchange/bounty_escrow.py for the lock/release
  • primitives — funds are already escrow-able.
  • Statistician review automation: invoke the statistician
  • skill via the existing skill-registry call path (see
    scidex/forge/tools.py skill_router pattern).
  • Wet-lab raw_data_url validation: store the URL but require
  • a SHA256 hash + file size to deter empty submissions; the
    hash is recomputed by a sidecar fetcher with size limits.
  • Reviewer notifications via the daily-digest mailer.
  • Public dashboard tile listing aggregate "$ paid" + "labs
  • participating" — suitable for grant-application screenshots.

    Dependencies

    • scidex.exchange.bounty_escrow — lock_funds / release.
    • q-crowd-expert-review-queue — verifier reputation registry.
    • scidex.agora.experiment_extractor — sources unverified
    findings to seed initial contests.
    • statistician skill — automated review pass.

    Work Log

    2026-04-27 — All acceptance criteria implemented and verified:

    • migrations/123_add_replication_contest_tables.py: 4 new tables + history mirror, proper FKs/CHECK constraints/indexes
    • scidex/exchange/replication_contests.py (814 lines): create_contest, claim_contest, submit_replication, resolve_contest, expire_contest, list_open_contests, seed_initial_contests
    • _lock_bounty/_release_bounty wrap bounty_escrow primitives; _spawn_reviewers creates 1 statistician + 2 human reviewers
    • _select_human_reviewers JOINs human_expert_reviews with agent_registry, falls back to contributors table
    • api.py routes added at line 35597+: GET/POST /replication-contests, GET /replication-contests/{id}, POST /api/replication-contests/{id}/claim, /submit, /resolve, /review
    • scripts/seed_replication_contests.py: wraps seed_initial_contests with --dry-run and --top-n flags
    • tests/test_exchange_replication_contests.py (8 tests, all passing): full lifecycle + partial replication + rejection + expiration
    • Migration applied to live DB; 10 initial contests seeded
    • Note: replication_status='unverified' does not exist in DB — actual values are 'single_study', 'unknown', 'replicated', 'conflicting', 'inconclusive'. hypothesis_ids in experiments is a JSON array string; junction table experiment_hypotheses used for proper JOIN.
    • Commit 7fcc524d0 pushed to orchestra/task/5633cca8-replication-contests-with-cash-bounties

    Already Resolved — 2026-04-28 01:15:00Z

    Evidence: git show origin/main:scidex/exchange/replication_contests.py returns the full 1074-line module. Squash-merged to main as commit 4b5e10d12 (PR #790). All 8 acceptance criteria confirmed [x] in spec. 8 tests passing, 10 initial contests seeded, migration applied to live DB. Task re-claimed by a new slot but no remaining work — closing as already resolved.

    Sibling Tasks in Quest (Pre-Registration & Replication) ↗