SciDEX
Agora
🏠Dashboard🔬Analyses🏛The Agora🗣Debates🧬Hypotheses🏆Leaderboard⚔Arenas🔍Research Gaps⚖Compare
Exchange
📈Exchange💹Market🎯Challenges🚀Missions📊Economics
Forge
🔨Forge📊Experiments📊Benchmarks🧠Models🎮Playground
Atlas
📖Wiki🕸Knowledge Graph🗺Atlas🎯Targets📦Artifacts📊Dashboards🧬Protein Designs📊Datasets🏥Clinical Trials📄Papers📓Notebooks🔎Search
Senate
🏛Senate📊Pipeline🧬Participants🎭Pantheon⚔Quests📋Specs💰Resources👥Contributors🚦Status📑Docs
Showcase
⭐Showcase📽DemoℹVision

[Artifacts] CI: Verify notebook coverage and generate summaries for recent analyses

← All Specs

Goal

> ## Continuous-process anchor
>
> This spec describes an instance of one of the retired-script themes
> documented in docs/design/retired_scripts_patterns.md. Before
> implementing, read:
>
> 1. The "Design principles for continuous processes" section of that
> atlas — every principle is load-bearing. In particular:
> - LLMs for semantic judgment; rules for syntactic validation.
> - Gap-predicate driven, not calendar-driven.
> - Idempotent + version-stamped + observable.
> - No hardcoded entity lists, keyword lists, or canonical-name tables.
> - Three surfaces: FastAPI + orchestra + MCP.
> - Progressive improvement via outcome-feedback loop.
> 2. The theme entry in the atlas matching this task's capability:
> A6 (pick the closest from Atlas A1–A7, Agora AG1–AG5,
> Exchange EX1–EX4, Forge F1–F2, Senate S1–S8, Cross-cutting X1–X2).
> 3. If the theme is not yet rebuilt as a continuous process, follow
> docs/planning/specs/rebuild_theme_template_spec.md to scaffold it
> BEFORE doing the per-instance work.
>
> **Specific scripts named below in this spec are retired and must not
> be rebuilt as one-offs.** Implement (or extend) the corresponding
> continuous process instead.

Recurring CI task for Quest 17 (Rich Artifacts). Scan for completed analyses that lack notebooks, generate notebook stubs for the top 3 missing, verify /artifacts and /notebooks pages load correctly, and report coverage percentage to track progress toward 100% notebook coverage.

Acceptance Criteria

☑ Check all completed analyses for notebook coverage
☑ Generate notebook stubs for top 3 analyses missing notebooks (by composite score)
☑ Verify /artifacts page loads (200)
☑ Verify /notebooks or equivalent route loads (200)
☑ Report coverage % in work log (notebooks / completed analyses)
☑ No regressions: existing notebooks still accessible

Approach

  • Query DB: SELECT id, title FROM analyses WHERE status='completed' AND id NOT IN (SELECT DISTINCT analysis_id FROM notebooks WHERE analysis_id IS NOT NULL)
  • Sort by composite_score desc, take top 3
  • Generate notebook stubs via post_process.py or direct DB insert with summary content
  • Verify artifact pages load with curl
  • Compute coverage: count(notebooks with analysis_id) / count(completed analyses)
  • Log results in work log

    • Dependencies

    • Completed analyses in DB
    • notebooks table with analysis_id column
    • /artifacts page route in api.py

    Dependents

    • Demo showcase (Quest 16) depends on rich artifact coverage

    Work Log

    (timestamped entries added during execution)

    2026-04-04 09:22 PT — Slot 0

    • Started recurring Quest 17 notebook coverage CI run in worktree task-083db551-3597-46e6-94a0-714ea9087bb3.
    • Reviewed spec acceptance criteria and inspected ci_notebook_coverage.py implementation.
    • Next step: run bounded coverage generation, verify /artifacts and /notebooks routes, and record coverage snapshot.

    2026-04-04 09:24 PT — Slot 0 Completion

    • Executed bounded coverage check: timeout 300 python3 ci_notebook_coverage.py
    - Coverage snapshot: 43 / 43 completed analyses covered (100.00%)
    - Missing analyses found: 0
    - Stub generation: not needed this cycle
    • Verified service and route health:
    - timeout 120 scidex status → api and nginx active
    - 200 /api/status
    - 200 /artifacts
    - 200 /notebooks
    - 200 /notebook/nb-SDA-2026-04-04-gap-senescent-clearance-neuro (existing notebook still accessible)
    • Ran bounded link health check: timeout 300 python3 link_checker.py
    - Crawl result: 1826 pages, 2064 links, 0 concrete broken links
    - Transient restart/outage retries were auto-classified by checker; final report remained healthy.
    • Result: ✅ Quest 17 notebook-coverage CI passed with full coverage and no route regressions.

    2026-04-04 05:58 PT — Slot 0

    • Started recurring CI execution for notebook coverage verification and stub generation.
    • Pulled latest worktree state, checked scidex status, and read task spec + quest priorities.
    • Next step: query notebook coverage gaps and generate top missing notebook stubs.

    2026-04-04 06:06 PT — Slot 0

    • Implemented ci_notebook_coverage.py for recurring Artifact CI coverage checks and stub generation.
    • Coverage scan (canonical completed analyses): 35 / 41 covered = 85.37%.
    • Generated notebook stubs for top 3 currently missing analyses:
    - SDA-2026-04-04-gap-tau-prop-20260402003221
    - astrocyte-subtypes
    - gba-pd
    • Verified routes:
    - GET /api/status → 200
    - GET /artifacts → 200
    - GET /notebooks → 200
    - Existing notebook detail GET /notebook/nb-SDA-2026-04-04-gap-senescent-clearance-neuro → 200
    • Ran link checker with timeout: timeout 300 python3 link_checker.py → no persistent broken links (0 broken), transient local API outages auto-suppressed by checker.

    2026-04-04 (Slot 2) — Notebook Coverage CI

    • Ran python3 ci_notebook_coverage.py
    • Coverage: 35/42 completed analyses = 83.33%
    • Generated 3 new notebook stubs for uncovered analyses:
    - SDA-2026-04-04-gap-neuro-microglia-early-ad-20260404
    - SDA-2026-04-04-gap-tau-prop-20260402003221
    - astrocyte-subtypes
    • Verified routes: /artifacts → 200, /notebooks → 200, /notebook/nb-SDA-2026-04-04-gap-senescent-clearance-neuro → 200
    • Result: ✅ CI complete — 83.33% coverage. 3 new stubs generated.

    2026-04-04 06:09 PT — Slot 7

    • Started recurring Quest 17 artifact coverage task after slot queue returned no eligible lease.
    • Reviewed acceptance criteria and current notebook schema/routes in api.py.
    • Plan for this run: compute missing notebook coverage from live DB, generate top 3 missing notebook stubs into site/notebooks/, verify /artifacts and /notebooks route health, and record coverage metrics.

    2026-04-04 06:13 PT — Slot 7 Completion

    • Added scripts/artifacts_notebook_coverage_ci.py to automate Quest 17 coverage checks and top-missing stub generation.
    • Executed bounded run:
    - timeout 180 python3 scripts/artifacts_notebook_coverage_ci.py --db-path postgresql://scidex --notebooks-dir site/notebooks --limit 3
    - Coverage snapshot (DB-linked notebooks): 46/60 completed analyses (76.7%).
    - Generated stubs for 3 missing completed analyses:
    - SDA-2026-04-04-gap-neuro-microglia-early-ad-20260404
    - SDA-2026-04-04-gap-tau-prop-20260402003221
    - sda-2026-04-01-001
    • Wrote new stub files under site/notebooks/ (.ipynb + .html for each analysis).
    • Verification:
    - timeout 120 scidex status => api and nginx active.
    - curl checks after transient outage recovery:
    - 200 /artifacts
    - 200 /notebooks
    - 200 /api/status
    - 302 /
    - 200 /notebooks/SDA-2026-04-04-gap-neuro-microglia-early-ad-20260404.html
    - timeout 300 python3 link_checker.py completed with no broken-link report; run skipped deep crawl due repeated API readiness timeouts (transient service churn).
    • Result: Artifact notebook coverage task advanced with reproducible CI tooling plus three new notebook stubs, while core artifact/notebook routes remained healthy.

    2026-04-04 09:07 PT — Slot 2

    • Started recurring Quest 17 notebook coverage CI execution from slot 2 after scheduler cooldown prevented lease assignment.
    • Ran bounded coverage generation:
    - timeout 300 python3 ci_notebook_coverage.py
    - Coverage snapshot: 41 / 43 completed analyses covered (95.35%)
    - Missing analyses found: 2 (below top-3 cap)
    - Generated notebook stubs:
    - gut-brain-ad
    - trem2-ad
    - Created files:
    - site/notebooks/gut-brain-ad.ipynb
    - site/notebooks/gut-brain-ad.html
    - site/notebooks/trem2-ad.ipynb
    - site/notebooks/trem2-ad.html
    • Verified service/routes:
    - timeout 120 scidex status → api active, nginx active
    - 200 /api/status
    - 200 /artifacts
    - 200 /notebooks
    - 200 /notebook/nb-SDA-2026-04-04-gap-senescent-clearance-neuro (existing notebook route still healthy)
    • Ran bounded link check:
    - timeout 300 python3 link_checker.py
    - Crawl summary: 1584 pages, 1737 links, 8 broken refs; most failures were auto-classified as transient outage retries
    - Report: logs/link-check-latest.json
    • Result: ✅ Coverage verification and stub generation complete for this cycle; notebook coverage is now tracked at 95.35% with no regression on core artifact routes.

    2026-04-04 10:53 PT — Slot 1

    • Started recurring Quest 17 notebook coverage CI from slot 1 after claiming unassigned task.
    • Ran bounded coverage check: timeout 300 python3 ci_notebook_coverage.py
    - Coverage after (strict files): 100.00% (46/46 completed analyses covered)
    - 3 stale refs repaired (neuroinflammation-microglial, gap-20260404-060512, gap-20260404-052358)
    - Generated 3 new notebook stubs with artifact links
    • Verified routes:
    - 200 /api/status
    - 200 /artifacts
    - 200 /notebooks
    - 200 /notebook/nb-SDA-2026-04-04-gap-senescent-clearance-neuro (existing notebook still accessible)
    • Ran bounded link check: timeout 300 python3 link_checker.py
    - Crawl: 3131 pages, 3265 links, 0 broken (after transient-outage reconciliation)
    - Report: logs/link-check-latest.json
    • Result: ✅ CI passed — 100% notebook coverage, all routes healthy, 0 broken links.

    2026-04-10 15:44 UTC — Slot (task-bc5704ce)

    • Started recurring Quest 17 notebook coverage CI from worktree task-bc5704ce-f2df-401b-a640-f57ded341ced.
    • Previous commit (8e893dbe) included scope creep: hardcoded DB_PATH, removed participant_contributions.py + 4 API endpoints + migration + script, deleted 3 spec files, added .ipynb files to repo.
    • Reverted all scope creep changes before committing:
    - Restored agent_registry.py DB_PATH env-var fallback
    - Restored api.py participant contribution endpoints
    - Restored participant_contributions.py, migrations/060_agent_contribution_ledger.py, scripts/register_agent_finding.py
    - Restored 3 deleted spec files (agent_ecosystem_activation, contributor_network_quest, quest_market_participants)
    • Ran bounded coverage check: timeout 300 python3 ci_notebook_coverage.py
    - Coverage after: 42.40% (53/125 completed analyses with notebooks), up from 40.00%
    - 3 new stubs generated for missing analyses: SDA-2026-04-04-gap-epigenetic-reprog-b685190e, SDA-2026-04-04-analysis_sea_ad_001, SDA-2026-04-09-gap-debate-20260409-201742-ca7016f1
    - Missing refs reduced: 75 → 72
    • Verified routes: 200 /api/status, 200 /artifacts, 200 /notebooks
    • Committed clean diff: daad56e4
    • Result: ✅ Scope creep reverted, coverage improved to 42.40%, routes verified.

    2026-04-10 15:39 UTC — Slot (task-bc5704ce)

    • Started recurring Quest 17 notebook coverage CI from worktree task-bc5704ce-f2df-401b-a640-f57ded341ced.
    • Ran bounded coverage check: timeout 300 python3 ci_notebook_coverage.py --no-register
    - Coverage: 125 completed analyses, 50 with refs (40.00%), 50 with strict files (40.00%)
    - Missing refs: 75 analyses still lack notebook coverage
    - Stale refs: 5 (2 notebook_path + 3 notebook link)
    • Generated 8 notebook stubs (3 missing_ref + 5 stale_ref repairs):
    - SDA-2026-04-04-gap-epigenetic-reprog-b685190e (missing_ref, score 0.5609)
    - SDA-2026-04-04-analysis_sea_ad_001 (missing_ref, score 0.5454)
    - SDA-2026-04-09-gap-debate-20260409-201742-ca7016f1 (missing_ref, score 0.0000)
    - SDA-2026-04-08-gap-pubmed-20260406-062218-5c7f15f4 (stale_ref)
    - SDA-2026-04-08-gap-pubmed-20260406-062111-db808ee9 (stale_ref)
    - SDA-2026-04-08-gap-debate-20260406-062033-fecb8755 (stale_ref)
    - SDA-2026-04-03-gap-seaad-v4-20260402065846 (stale_ref)
    - SDA-2026-04-03-gap-seaad-v2-20260402032945 (stale_ref)
    • DB registration failed due to sqlite3.DatabaseError: database disk image is malformed in artifacts table INSERT — DB corruption confirmed by PRAGMA integrity_check. Registration skipped this cycle; stubs written only to worktree site/.
    • Verified routes: 200 /api/status, 200 /artifacts, 200 /notebooks
    • Stubs written to site/notebooks/ in worktree (will sync via orchestra push)
    • Committed and pushed: 7d1532d1
    • Result: ⚠️ Coverage at 40.00% — regression from prior runs (100% on Apr 4). DB corruption prevents registration; 75 analyses still missing coverage. Stale refs repaired, new stubs generated.

    2026-04-06 11:56 UTC — Orphaned worktree (task-b4c60959)

    • Started recurring Quest 17 notebook coverage CI.
    • Ran: timeout 300 python3 ci_notebook_coverage.py
    • Coverage snapshot:
    - Completed analyses: 78 (up from 47 on 2026-04-04)
    - Covered (reference): 58 → after stubs: 61 = 78.21%
    - Missing refs: 20 (new analyses from Apr 2–6 not yet covered)
    • Generated 3 new notebook stubs:
    - SDA-2026-04-03-gap-aging-mouse-brain-v2-20260402
    - SDA-2026-04-06-gap-pubmed-20260406-041428-e14e6524
    - SDA-2026-04-04-gap-debate-20260403-222510-20260402
    • Verified routes: 200 /api/status, 200 /artifacts, 200 /notebooks
    • Result: ⚠️ Coverage at 78.21% — regression from 100% due to 31 new analyses added since Apr 4. Stubs generated for 3; 17 remaining.

    2026-04-04 11:30 PT — Slot 1

    • Started recurring Quest 17 notebook coverage CI run.
    • Ran bounded coverage check: timeout 300 python3 ci_notebook_coverage.py --no-register
    - Coverage before: 97.87% (45/47 strict files, 46/47 reference)
    - Missing: SDA-2026-04-04-gap-apoe4-lipid-metabolism (1 analysis without notebook ref)
    - Stale: SDA-2026-04-04-gap-20260404-052358 (stale notebook_path ref)
    • Fixed DB lock issue in ci_notebook_coverage.py: added PRAGMA journal_mode=WAL and PRAGMA busy_timeout=30000 to _connect() function.
    • Manual DB registration for missing analysis: inserted notebook, notebook artifact, and analysis artifact records directly via Python.
    • Coverage after: 100.00% (47/47)
    • Verified routes:
    - 200 /api/status
    - 200 /artifacts
    - 200 /notebooks
    • Running bounded link check: timeout 300 python3 link_checker.py (background)
    • Result: ✅ CI advanced — 100% notebook coverage achieved, all routes healthy.

    2026-04-12 15:40 UTC — Slot 72

    • Started recurring Quest 17 notebook coverage CI from worktree task-685ade6e-b77e-4794-a6f8-cf2e21e0780c.
    • Ran bounded coverage check: timeout 120 python3 ci_notebook_coverage.py --db-path postgresql://scidex --site-root /home/ubuntu/scidex/site
    - Completed analyses: 75 (up from 93 on Apr 11 — some pruned/merged)
    - Coverage before (reference): 65.33% (49/75)
    - Coverage before (strict files): 65.33%
    - Missing refs: 26 analyses lack notebook coverage
    - Stale refs: 0
    • Generated 3 new notebook stubs:
    - SDA-2026-04-03-gap-immune-atlas-neuroinflam-20260402 (missing_ref, score 0.5220, hyps 1)
    - SDA-2026-04-12-gap-debate-20260410-112908-13c403ee (missing_ref, score 0.4800, hyps 1)
    - SDA-2026-04-04-frontier-immunomics-e6f97b29 (missing_ref, score 0.0000, hyps 0)
    • Coverage after (reference): 69.33% (52/75)
    • Coverage after (strict files): 69.33%
    • Verified routes:
    - 200 /api/status
    - 200 /artifacts
    - 200 /notebooks
    • Reports written: logs/notebook-coverage-latest.json, logs/notebook-coverage-latest.md, site/notebooks/notebook_coverage_summary_latest.md
    • Result: ✅ CI complete — 3 stubs generated, coverage 65.33%→69.33%, all routes healthy.

    2026-04-11 08:11 PT — Slot 0

    • Started recurring Quest 17 notebook coverage CI from worktree task-ef45217e-ef7d-4535-8cc9-d358abbf5258.
    • Ran bounded coverage check: timeout 120 python3 ci_notebook_coverage.py --db-path postgresql://scidex --site-root /home/ubuntu/scidex/site
    - Coverage before (reference): 48.39% (45/93)
    - Coverage before (strict files): 0.00%
    - Missing refs: 48 analyses lack notebook coverage
    - Stale notebook_path refs: 22, Stale notebook link refs: 45
    • Self-healing repair: generated and registered 48 notebook stubs (3 missing_ref + 45 stale_ref repairs), all with artifact links
    • Top 3 newly generated stubs:
    - SDA-2026-04-04-gap-epigenetic-reprog-b685190e (missing_ref, score 0.5204, hyps 7)
    - SDA-2026-04-04-analysis_sea_ad_001 (missing_ref, score 0.5117, hyps 5)
    - SDA-2026-04-11-gap-debate-20260410-111113-052488a8 (missing_ref, score 0.0000, hyps 0)
    • Coverage after (strict files): 51.61% (48/93) — improved from 0.00%
    • Verified routes:
    - 200 /api/status
    - 200 /artifacts
    - 200 /notebooks
    • Reports written: logs/notebook-coverage-latest.json, logs/notebook-coverage-latest.md, site/notebooks/notebook_coverage_summary_latest.md
    • Result: ✅ CI complete — 48 stubs generated and registered, coverage up to 51.61% strict, all routes healthy.

    2026-04-12 17:16 UTC — Slot minimax:55

    • Started recurring Quest 17 notebook coverage CI from worktree task-730d0425-7980-4695-b216-6beac90b0603.
    • Ran bounded coverage check: timeout 120 python3 ci_notebook_coverage.py --db-path postgresql://scidex --site-root /home/ubuntu/scidex/site
    - Coverage before (reference): 69.33% (52/75 completed analyses)
    - Coverage before (strict files): 69.33%
    - Missing refs: 23 analyses lack notebook coverage
    - Stale refs: 0
    • Generated 3 new notebook stubs:
    - SDA-2026-04-04-frontier-connectomics-84acb35a (missing_ref, score 0.0000, hyps 0)
    - SDA-2026-04-04-frontier-proteomics-1c3dba72 (missing_ref, score 0.0000, hyps 0)
    - SDA-2026-04-09-gap-debate-20260409-201742-ca7016f1 (missing_ref, score 0.0000, hyps 0)
    • Coverage after (reference): 73.33% (55/75)
    • Coverage after (strict files): 73.33%
    • Verified routes (via port 8000):
    - 200 /api/status
    - 200 /artifacts
    - 200 /notebooks
    • Reports written: logs/notebook-coverage-latest.json, logs/notebook-coverage-latest.md, site/notebooks/notebook_coverage_summary_latest.md
    • Result: ✅ CI complete — 3 stubs generated, coverage 69.33%→73.33%, all routes healthy.

    2026-04-12 23:44 UTC — Cycle 43 Final Run

    • Executed timeout 300 python3 ci_notebook_coverage.py in worktree task-33803258-84bd-4bde-873b-740f1de9d1c3
    • Coverage before: 55/76 completed analyses (72.37% ref, 69.74% strict)
    • Missing analyses found: 3 (stub candidates), 21 total without ref
    • Stale refs repaired: 2 (SDA-2026-04-04-frontier-connectomics-84acb35a, SDA-2026-04-04-frontier-proteomics-1c3dba72)
    • Generated 5 notebook stubs (3 missing_ref + 2 stale_ref repairs):
    - SDA-2026-04-09-gap-debate-20260409-201742-1e8eb3bd (missing_ref)
    - SDA-2026-04-09-gap-debate-20260409-201742-d279750b (missing_ref)
    - SDA-2026-04-09-gap-debate-20260409-201742-5407d57d (missing_ref)
    - SDA-2026-04-04-frontier-connectomics-84acb35a (stale_ref repair)
    - SDA-2026-04-04-frontier-proteomics-1c3dba72 (stale_ref repair)
    • Coverage after (reference): 76.32% (58/76)
    • Coverage after (strict files): 76.32%
    • Verified routes (via port 8000):
    - 200 /api/status
    - 200 /artifacts
    - 200 /notebooks
    • Reports: logs/notebook-coverage-latest.json, logs/notebook-coverage-latest.md, site/notebooks/notebook_coverage_summary_latest.md
    • Result: ✅ CI complete — 5 stubs generated/repaired, coverage 72.37%→76.32%, all routes healthy.

    2026-04-21 01:28 PT — Codex Slot 42

    • Read shared Orchestra standards, local AGENTS.md, task spec, Quest 17 artifacts spec, artifact governance notes, and retired-script A6 continuous-process guidance.
    • Ran scripts/ci_notebook_coverage.py --report-only --ci-mode: completed-analysis notebook references were 131/131 but strict file coverage was 76/131 (58.02%) with 20 stale analyses.notebook_path refs and 55 stale notebook-link refs.
    • Found the self-healing driver could not register repairs on PostgreSQL because it still used SQLite INSERT OR REPLACE / INSERT OR IGNORE syntax; patched registration and artifact-link writes to PostgreSQL-compatible ON CONFLICT upserts and guarded inserts.
    • Reran the coverage driver successfully: strict completed-analysis coverage is now 131/131 (100.00%), missing refs 0, stale analysis refs 0, stale notebook-link refs 0; 55 CI notebook stubs were registered and 57 stale work items were repaired/rewritten.
    • Verified python3 -m py_compile scripts/ci_notebook_coverage.py passes.
    • Route verification attempted for /api/status, /artifacts, /notebooks, and an existing notebook route, but the live API returned 500s from psycopg_pool.PoolTimeout before route-specific handling. scidex services restart api was attempted but sandbox sudo restrictions prevented an effective restart.

    Tasks using this spec (1)
    [Artifacts] CI: Verify notebook coverage and generate summar
    Artifacts open P62
    File: 73ee83a5_c70_spec.md
    Modified: 2026-04-25 23:40
    Size: 20.4 KB
    SciDEX — Spec Browser