[Forge] Skill scaffolder - scidex new-skill from one signature open

← Forge
Generate a typed @log_tool_call-wired tool, registry seed, smoke test, and manifest entry from one declaration.
Spec File

Effort: standard

Goal

Adding a new tool/skill (a wrapper around an external API or a local
analysis primitive) currently means: write the function in scidex/forge/tools.py (or a new file under scidex_tools/),
remember to apply @log_tool_call (defined in scidex/forge/forge_tools.py:67), register it in the skills
table via register_tool at line 43, write a smoke test (look at scripts/test_forge_tools.py — 200+ LoC of hand-rolled probes),
optionally add a card to the playground at scripts/test_playground_tools.py.
Half the new tools that have shipped in the last quarter are missing
at least one of those steps. Ship scidex new-skill to do all five
from a single signature declaration.

Acceptance Criteria

CLI scidex new-skill <name> --inputs '<json-schema>'
--outputs '<json-schema>' --layer Forge scaffolds:
- scidex/forge/skills/<name>.py with a single function
def <name>(**kwargs) -> dict: decorated with
@log_tool_call and a docstring auto-rendered from the
input/output schemas.
- A registry seed in
migrations/<YYYYMMDD>_register_skill_<name>.py calling
register_tool(<name>, <description>, ...).
- A smoke test in tests/skills/test_<name>.py:
test_signature (kwargs matches schema),
test_logged (invocation row appears in tool_invocations),
test_failure_path (raising → success=0 row written).
- Optionally a playground card if --playground flag set.
Type-checking gate. Generated function uses pydantic v2
validate_call so bad kwargs raise ValidationError before
the wrapped logic runs (closes the "tools fail silently on bad
args" gap explicitly listed in the wave-2 brief). Add a unit
test that calling with the wrong arg type raises a clear
message.
Skill manifest export. New
scripts/export_skill_manifest.py walks
scidex/forge/skills/ and emits site/skills.json (one entry
per skill: name, inputs schema, outputs schema, register
timestamp, last-call timestamp, success-rate). The manifest
feeds the live API explorer (q-devx-live-api-explorer) and the
future tool-discovery LLM.
Tests tests/test_skill_scaffolder.py:
- End-to-end scaffold of a dummy_pubmed_proxy skill against
a tmpdir; assert generated test passes via pytest -k
dummy_pubmed_proxy
.
- JSON-Schema malformed input rejected by CLI with non-zero
exit and pointing at the bad path.
- Manifest exporter round-trips: scaffold N skills →
manifest contains N entries with the right schemas.
Migration friendliness. scidex new-skill --import-existing
<module:func> reads the function signature of an existing
tool and emits a wrapper compatible with the new pattern, so
pre-pydantic tools can be migrated incrementally.

Approach

  • Read scidex/forge/forge_tools.py lines 1-130 to internalise the
  • @log_tool_call decorator + register_tool flow.
  • Read three working tool modules (scidex/forge/tools.py:20+,
  • scidex_tools/methbase.py, scidex_tools/model_training.py) to
    normalise the signature shape.
  • Build the renderer with string.Template.
  • Pydantic v2 validate_call adds runtime input validation; add a
  • guard that fails the build if pydantic isn't installed (it's a
    project dep already; assert on import in the scaffolder entry
    point).
  • Manifest exporter is one file — no new tables.
  • Dependencies

    • scidex/forge/forge_tools.py:43,67register_tool /
    log_tool_call.

    Dependents

    • q-devx-live-api-explorer — consumes site/skills.json.

    Work Log

    Sibling Tasks in Quest (Forge) ↗