[Forge] CI: Test all scientific tools for availability open coding:5

← Forge
Run a health check on all registered Forge tools (PubMed, Semantic Scholar, Allen Brain, etc). Report failures.

Completion Notes

Auto-release: recurring task had no work this cycle

Git Commits (15)

[Forge] Restore Forge tool availability self-test [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-21
[Forge] CI: Add Forge tool health check script — tests all 87 tools, reports failures [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-12
[Forge] Forge tool CI: 11/17 OK, AlphaFold regression detected [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-11
[Forge] Forge tool CI: 11/17 OK, AlphaFold regression detected [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-11
[Forge] Forge tool CI: 11/17 OK, AlphaFold regression detected [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-11
[Forge] CI: 6/8 tools OK — Reactome empty, OpenTargets failing [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-04
[Forge] Daily tool integration verification - 830 executions [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-04
[Forge] Add STITCH chemical interactions tool with DGIdb fallback2026-04-04
[Forge] Run daily scientific tool availability CI sweep [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-04
[Forge] Record daily scientific tool availability CI run [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-04
[Forge] CI: Test all scientific tools for availability2026-04-04
[Forge] CI: Tool health check - 39/44 tools passed, 5 warnings, 0 failures [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-04
[Forge] Harden tool health checks and fix OpenAlex null handling [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-04
[Forge] Add daily health check for all scientific tools [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-04
[Forge] Tool health check: 13/18 OK, 5 EMPTY (external API issues) [task:3df010f4-0323-44a0-9699-6d65381c98a5]2026-04-02
Spec File

Goal

Run a recurring health check across registered Forge scientific tools and their backing APIs to ensure they are reachable and returning usable results. Capture failures and degradations with enough detail to distinguish external API issues from SciDEX regressions. Keep an auditable task log for operational continuity.

Acceptance Criteria

☑ Core Forge scientific tools are exercised with live requests
☑ Availability outcomes are documented per tool (OK/EMPTY/FAIL + brief reason)
☑ Results and summary are logged in the Work Log
☑ Service health verification recorded

Approach

  • Review current tool registry and expected callable tool functions.
  • Run tool smoke checks using existing SciDEX tooling with a bounded timeout.
  • Classify outcomes (OK/EMPTY/FAIL), identify likely root cause category.
  • Verify site/API health for regression safety.
  • Update Work Log and complete recurring task.
  • Dependencies

    • f13a8747-0876-451c-8c75-635af18029c2 — Forge tool library expansion

    Dependents

    • None (recurring CI monitoring task)

    Work Log

    2026-04-11 14:55 PT — Slot 4

    • Started recurring task run for Forge tool availability CI.
    • Executed tool availability sweep: timeout 300 python3 tools.py.
    • Tool status summary (17 exercised):
    - OK (11): PubMed Search, Gene Info, Clinical Trials, Enrichr, STRING Interactions, STRING Enrichment, GWAS, UniProt, Reactome, ClinVar, ChEMBL
    - EMPTY/DEGRADED (4): Semantic Scholar (empty result set), GTEx (no data + signature mismatch gtex_tissue_expression() got an unexpected keyword argument 'dataset'), DisGeNET Gene->Disease (API key required / non-JSON), DisGeNET Disease->Gene (API key required / non-JSON)
    - FAIL (1): AlphaFold (KeyError: 'uniprot_accession' — regression: previously returned OK, now crashes. API response format may have changed.)
    • Instrumentation logging: no such table: tool_calls persists (DB migration missing tool_calls table; tool execution itself proceeds normally).
    • Registry check: SELECT COUNT(*) FROM skills => 83 registered tools (up from 37 at last run).
    • Service verification:
    - curl http://localhost/ => 302
    - curl http://localhost:8000/api/status => 200 with valid JSON (227 analyses, 335 hypotheses, 688392 edges)
    - Page checks: / 302, /exchange 200, /gaps 200, /graph 200, /analyses/ 200, /atlas.html 200, /how.html 301
    • Result: 11/17 tools returned usable data, 4 degraded (upstream/auth), 1 FAIL (AlphaFold regression).

    2026-04-04 05:23 PT — Slot 4

    • Started recurring task run for Forge tool availability CI.
    • Confirmed lease for slot 4 and reviewed existing self-test coverage in tools.py.
    • Proceeding with bounded smoke execution (timeout 300) and per-tool outcome classification.

    2026-04-04 00:00 PT — Slot 8

    • Started recurring task: run Forge tool availability CI sweep.
    • Verified task lease via Orchestra and confirmed this worktree context.
    • Found task spec_path missing on disk; created this spec file to restore compliance.
    • Reviewed tool check entry points in tools.py and historical spec notes before executing live checks.

    2026-04-04 00:16 PT — Slot 8

    • Executed tool availability sweep with bounded timeout: timeout 300 python3 tools.py.
    • Tool status summary (17 exercised):
    - OK: PubMed Search, Gene Info, Clinical Trials, Enrichr, STRING Interactions, STRING Enrichment, GWAS, UniProt, Reactome, ClinVar, AlphaFold, PubChem (12)
    - EMPTY/DEGRADED: Semantic Scholar (empty result set), GTEx (no data returned; also signature mismatch in self-test call), ChEMBL (HTTP 400 / no data), DisGeNET Gene->Disease (non-JSON response), DisGeNET Disease->Gene (non-JSON response) (5)
    - FAIL: none hard-failed the run (0)
    • External/infra issue observed during run: instrumentation logging repeatedly emitted no such table: tool_calls (tool execution still proceeded).
    • Registry check: SELECT COUNT(*) FROM skills on main DB => 37 registered tools.
    • Service verification:
    - curl http://localhost/ => 301
    - curl http://localhost:8000/api/status => 200 with valid JSON
    - Page checks: / 302, /exchange 200, /gaps 200, /graph 200, /analyses/ 200, /atlas.html 200, /how.html 301
    - scidex services list: API/nginx/neo4j active; agent/improve/bridge/linkcheck/forge-improve inactive (expected current state)
    • Result: Completed recurring Forge tool availability check; 12/17 tools returned usable data, 5 degraded due to upstream/auth/endpoint behavior, 0 hard failures.

    Payload JSON
    {
      "completion_shas": [
        "78dcb1cad9bfb476336a453069f00ef745158c7e"
      ],
      "completion_shas_checked_at": "2026-04-12T17:21:29.384970+00:00",
      "completion_shas_missing": [
        "b64188057ad3bc68a227895aad19f79e8474e79b",
        "2a8b39731e3806e9fd73443008c960317c4b12f8",
        "02b4a7235283dc9aef7ca793f56d9b1cb9df6f27",
        "783b30e7a3bdc00a37cec1a8bfecdaac287a910d",
        "5ca75972972d5875460041782f1fdf47e7f0060f",
        "5c87d8e8314939a122401af388a58f1b06129bbe",
        "4a51d9a11347cba1098d5fc334576bdd78f9eec6",
        "bfc940d91115e7350b19733454d0ae79159ea606",
        "2bddcf6c913dbd6fa6415fb38df153d606224b44",
        "3389022271de1ae061182c3d24fc9a042bf404c9",
        "3cd69ea3d6874d83a59c525be8e2a585d19a1deb",
        "063b823acca64f77f55f132e3cf89b16851dfb77",
        "193bc578bd64c793a1c6ab7e41e880317473168f"
      ],
      "requirements": {
        "coding": 5
      }
    }

    Sibling Tasks in Quest (Forge) ↗