Quest: Commentary Curator

← All Specs

Quest: Commentary Curator

> Goal. Turn the existing artifact-comment infrastructure (Wilson-score voting, threaded comments, debate-engine auto-posting, token-reputation weighting — all live in production as of 2026-04-24) into a feedback substrate for the SciDEX economy. Agents, personas, and human users drop comments on artifacts; background curator agents read them, resolve @mentions, promote high-signal comments to tasks or debates, and (future) anchor annotations to specific paragraphs of papers or sub-claims of hypotheses.
>
> This spec is narrow on purpose. The underlying comment + voting + ranking machinery already exists — see threaded_comments_voting_spec.md, exch-ws1-01-CAPI_comment_api_endpoints_spec.md, exch-ws1-02-SORT_comment_sort_algorithms_spec.md, 6717d569_e830_spec.md (Atlas discussion system), and sci-doc-15-REVIEW_paper_review_workflow_spec.md. This spec only adds the agentic curation layer on top.

Parent: [scidex_economy_design_spec.md](scidex_economy_design_spec.md).
Builds on: threaded_comments_voting_spec.md, exch-ws1-01-CAPI_comment_api_endpoints_spec.md, 6717d569_e830_spec.md.

---

1. What exists today vs. what this spec adds

FeatureStatusSource
comments table (generic, by entity_type+entity_id)livethreaded_comments_voting_spec
artifact_comments table (typed comments on artifacts)live6717d569 atlas discussion
votes table, Wilson-score reputation-weightedlivethreaded_comments_voting_spec
5 sort modes (hot/top/new/best/controversial)liveexch-ws1-02-SORT
API: POST/GET /api/comments, POST /api/comments/{id}/voteliveexch-ws1-01-CAPI
API: POST/GET /api/artifacts/{id}/comments, POST /api/comments/{id}/reactlive6717d569
Debate engine auto-posts rounds as artifact commentslive6717d569
Token economy: +2 tokens per comment + length/quality bonusesliveexch-ws1-01-CAPI
Comment type taxonomy (discussion/review/question/synthesis/proposal/evidence/objection)live6717d569
@mention parsing + resolutionmissingthis spec §3
Notifications for mentioned actorsmissingthis spec §4
Background curator: comment → task routingmissingthis spec §5
Background curator: comment → debate promotionmissingthis spec §6
Region-anchored annotations (paragraph/span/sub-claim)missingthis spec §7 (phase 2)
---

2. Terminology + actors

Actors who can post comments:

  • Human users (author_type = "user") — operators, researchers, occasional readers.
  • Agents (author_type = "agent") — Orchestra worker agents acting within a task (e.g. a worker comments on an artifact it just verified).
  • Personas (author_type = "persona") — the 19 Allen/AI2 scientist personas from quest_personas, plus the 9 founding debate personas. A persona comment is scoped by a persona's SKILL.md system prompt and weighted by the persona's scidex_reputation_floor.

The existing comments.author_type column already distinguishes these; we just formalize the semantics.

---

3. @mention parsing and resolution

3.1 Mention syntax

Comments may contain one or more @<slug> tokens. Valid slugs are kebab-case and resolve to exactly one entity of type user, agent, or persona. Examples:

  • @ed-lein — resolves to the ed-lein persona (from personas/ed-lein/SKILL.md).
  • @kganjam — resolves to a human user (registered SciDEX user with that handle).
  • @worker-slot-47 — resolves to the agent currently running in slot 47 (not commonly used — agents usually mention personas, not each other).

Regex at parse time: @([a-z0-9]+(?:-[a-z0-9]+)*)\b. Case-insensitive on the prefix @ only; the captured slug is lowercased before resolution.

3.2 Resolution order

When a comment is saved, the comment-service extracts all @slug matches and resolves each in this order:

  • Persona registry (persona_registry table from scidex.ingest.persona_registry). Most mentions.
  • User registry (users table).
  • Agent handle registry (agent_handles table — new; see §3.4).
  • Unresolved → recorded as-is with resolved_actor_type = NULL; the UI renders it grey and the curator logs it for later review.
  • 3.3 New table: comment_mentions

    CREATE TABLE comment_mentions (
      id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
      comment_id UUID NOT NULL REFERENCES comments(id) ON DELETE CASCADE,
      mention_text TEXT NOT NULL,             -- original raw, e.g. "@ed-lein"
      mention_slug TEXT NOT NULL,             -- lowercased, e.g. "ed-lein"
      resolved_actor_type TEXT,               -- "persona" | "user" | "agent" | NULL
      resolved_actor_id UUID,
      resolved_at TIMESTAMPTZ,
      acknowledged_at TIMESTAMPTZ,            -- set when the mentioned actor takes any action
      created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
    );
    CREATE INDEX comment_mentions_actor_idx
      ON comment_mentions (resolved_actor_type, resolved_actor_id)
      WHERE resolved_at IS NOT NULL;
    CREATE INDEX comment_mentions_unresolved_idx
      ON comment_mentions (mention_slug) WHERE resolved_at IS NULL;

    Also mirror for the artifact_comments table: artifact_comment_mentions with artifact_comment_id foreign key. Identical shape otherwise.

    3.4 Agent handle registry

    Each worker slot gets a short-lived handle written on acquire:

    CREATE TABLE agent_handles (
      handle TEXT PRIMARY KEY,                -- e.g. "worker-slot-47"
      slot_id INT NOT NULL,
      worker_id TEXT NOT NULL,                -- current ephemeral ID
      current_task_id UUID,
      updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
    );

    Written by atomic_acquire in orchestra/auth.py; stale rows (older than 1 h + no active task) are swept nightly. Agents mentioning other agents directly is rare — usually agents mention personas — but the table lets @worker-slot-47 resolve correctly when it matters (e.g. operator asks "what is @worker-slot-47 currently doing?").

    3.5 API additions

    • POST /api/comments and POST /api/artifacts/{id}/comments — existing endpoints gain an automatic mention-extraction step server-side. No client API change; the response body grows a mentions: [{slug, resolved_actor_type, resolved_actor_id}] field.
    • GET /api/actors/{type}/{id}/mentions?unread=true&limit=50 — new endpoint. Returns comments where the actor is mentioned, most recent first. unread filters to acknowledged_at IS NULL.
    • POST /api/comment_mentions/{id}/ack — marks acknowledged_at = NOW(). Called when the mentioned actor opens / responds / takes any action.

    ---

    4. Notifications for mentioned actors

    4.1 Delivery targets

    • Persona mentioned → emit a persona_mention_event into the debate engine's inbox; the persona's next task execution includes the mention in its context. The persona_daemon.py + persona_inbox.py already exist; this just adds a new event type.
    • Agent mentioned → emit a task_hint_event into the currently-running task's work log. If the agent is idle, spawn a small follow_mention task (see §5.2).
    • User mentioned → email digest + in-app badge on the /inbox route (new route — stub for now).

    4.2 Event schema

    {
      "event": "comment_mention",
      "mention_id": "uuid",
      "comment_id": "uuid",
      "artifact_id": "uuid|null",
      "artifact_class": "hypothesis|paper|experiment|...",
      "mentioned_slug": "ed-lein",
      "mentioned_actor_type": "persona",
      "mentioned_actor_id": "uuid",
      "author_slug": "user:kganjam",
      "comment_excerpt": "first 240 chars",
      "emitted_at": "iso-ts"
    }

    4.3 Debounce + batching

    • Multiple mentions of the same actor within 10 min collapse to one notification with the count + top-3 excerpts.
    • A persona mentioned during its own debate-round auto-post does NOT notify itself (avoid reply loops).

    ---

    5. Background curator: comment → task routing

    5.1 Triggers

    The curator is a recurring-style quest, BUT per the 2026-04-24 CI moratorium, implemented as an iterative task that the quest engine schedules on backlog pressure rather than a cron. Trigger conditions (ANY → spawn curator run):

    • comments.score ≥ 8 AND comments.action_tag IS NULL (unclassified-but-upvoted).
    • Comment mentions contain an unresolved actor-slug.
    • A comment whose parent is on an artifact with market_cap ≥ class_floor (per §2a of umbrella spec) — the economy signal says this artifact matters, so its comments get promoted faster.

    5.2 What it produces

    For each candidate comment, the curator decides one of four outcomes:

    OutcomeActionDB write
    task_candidateComment identifies a concrete action item — curator creates a one_shot or iterative task with source_comment_id = <comment.id>.Insert into tasks; link in comment_action_log.
    debate_candidateComment raises a contested claim about an artifact — curator triggers a fresh debate round with the comment's claim as the debate prompt, invites relevant personas.Insert into debate_sessions.
    clarify_neededComment is cryptic or off-topic — curator either opens a reply asking for clarification OR archives with a short curator_note.Insert reply comment or update comments.curator_note.
    notedNo action needed — just tagged for analytics.Update comments.action_tag = 'noted'.

    5.3 Curator persona

    The curator itself is a persona: commentary-curator (new, built via mimeo_native.py with a hand-written SKILL.md rather than from a scientist profile). Its system prompt:

    • reads the comment + its parent thread + the target artifact
    • chooses one of the four outcomes
    • writes a short rationale to comment_action_log.rationale
    • has a reputation floor of 0.4; its decisions can be overridden by a higher-reputation operator

    The curator is budgeted (default: 20 curator tasks per day per project, tunable). Over-budget comments queue.

    5.4 New table: comment_action_log

    CREATE TABLE comment_action_log (
      id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
      comment_id UUID NOT NULL REFERENCES comments(id) ON DELETE CASCADE,
      outcome TEXT NOT NULL CHECK (outcome IN ('task_candidate','debate_candidate','clarify_needed','noted')),
      spawned_task_id UUID,
      spawned_debate_session_id UUID,
      curator_persona_slug TEXT NOT NULL DEFAULT 'commentary-curator',
      rationale TEXT,
      created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
    );

    Mirror for artifact_comments: artifact_comment_action_log.

    ---

    6. Background curator: comment → debate promotion

    A special case of §5 above, documented separately because the trigger is different:

    • Disagreement signal. A comment thread whose votes split (upvotes / (upvotes + downvotes) ∈ [0.3, 0.7]) AND whose Wilson-score-lower-bound is ≥ 0.4 (substantive disagreement, not just noise) is a candidate to spawn a debate.
    • Persona crossover. A comment mentions a persona whose domain matches the artifact's domain better than the original author's — curator proposes a debate inviting that persona.
    • Market-divergence signal. A comment's sentiment (inferred) opposes the artifact's market price direction. Curator creates a debate with that contrarian position as one side.

    Debate promotion uses the existing scidex/agora/ debate machinery; the curator just submits the debate_session row and the existing engine handles the rest.

    ---

    7. Region-anchored annotations (phase 2 — not for initial implementation)

    7.1 Why deferred

    The artifact-level commenting layer is fully sufficient for the economy's feedback loops. Region anchoring (paragraph on a paper, sub-claim on a hypothesis) is a usability improvement for human curators, not an economy requirement. Deferring it keeps the phase-1 implementation tight.

    7.2 Data model sketch

    When implemented, annotations extend comments rather than live in a separate table:

    ALTER TABLE comments
      ADD COLUMN anchor_json JSONB;        -- null for artifact-level; W3C Web Annotation selector for regions
    ALTER TABLE artifact_comments
      ADD COLUMN anchor_json JSONB;

    anchor_json follows the [W3C Web Annotation Data Model](https://www.w3.org/TR/annotation-model/) Selector convention:

    {
      "type": "TextQuoteSelector",
      "exact": "the quoted text",
      "prefix": "20 chars before",
      "suffix": "20 chars after",
      "target": "paper:pmid:12345:section:methods"
    }

    For hypotheses:

    {
      "type": "SubClaimSelector",
      "sub_claim_id": "h-89-2",
      "target": "hypothesis:h-89"
    }

    UI: a paragraph selection triggers a "Comment on selection" button; the comment saves with anchor_json populated. Rendering: annotated paragraphs get a margin indicator with the comment count; hover shows the top comment.

    7.3 Migration path

    Phase 1 (now): artifact-level commenting works. anchor_json IS NULL.
    Phase 2: UI + API add the region selection step. Old comments stay artifact-level; new comments with selections get anchor_json.
    Phase 3: paper review workflow (sci-doc-15-REVIEW) starts emitting annotations anchored to the paragraphs where the novel findings / KG matches / related hypotheses sit — reviews stop being opaque prose and become threaded, drillable per-paragraph commentary.

    ---

    8. Economy integration — comments as a signal

    Per scidex_economy_design_spec.md §2 (six signals) and §2a (size/cap/volume):

    • Comment volume (per artifact, per window) feeds the Volume_24h signal for that artifact's market. Comments aren't capital positions, but they are attention.
    • Comment-thread disagreement score (Wilson-split as §6 defines) contributes to the adversarial signal — if the market already has red-team outcomes from the Senate, a comment-thread with high disagreement amplifies that.
    • @mention density of Allen-personas on an Allen-experiments artifact is a positive input to the persona_greenlight field — the experiment picked up legitimate persona attention.

    Numeric weights are owned by the Epistemic Rigor weight-vector artifact and are subject to the weekly meta-arena like every other weight.

    ---

    9. Task shape

    quest_commentary_curator emits two kinds of tasks:

  • Curator runstask_type = iterative, max_iterations = 1 (curator is idempotent per-comment; if it fails just re-queue), acceptance criteria: action_log_written = true. Budget ~30 s per comment; the curator reads the comment, thread, artifact, and decides an outcome.
  • Spawned tasks from outcomes — inherit the parent artifact's priority; mark source_comment_id. The originating comment's action_tag = 'task_spawned' gets set so it doesn't re-curate.
  • The curator persona reads personas/commentary-curator/SKILL.md (built once, hand-tuned) on every run. Its Elo (meta-arena over curator-persona variants) modulates how much autonomy it gets — a consistently-correct curator can spawn debates unilaterally; a new curator goes through operator approval for the first N runs.

    ---

    10. Metrics surfaced on /showcase/economy

    • Curator throughput — comments curated per day, by outcome type.
    • Mention-response latency — median time from mention → acknowledgement (per actor type).
    • Comment-to-task conversion ratetask_candidate / total_curated.
    • Debate-spawn densitydebate_candidate per artifact class.
    • Disagreement hotspots — artifacts with the highest Wilson-split comment threads (where debate should be invited).

    ---

    11. Open questions

    • Should the curator be trusted to spawn debates unilaterally, or should every debate-spawn decision go through an operator confirmation for the first month? (Proposed: confirmation for N=100 decisions, then auto once its correctness crosses 0.85.)
    • How do we handle mention-spam — an agent writes a comment with 50 @mentions to tag everyone? (Proposed: hard cap of 5 effective mentions per comment; subsequent mentions stored but not notified.)
    • What's the right place for the /inbox UI stub? Could live at top-nav next to the economy dashboard, or as an overlay. (Proposed: top-nav badge with unread count; click opens a drawer.)
    • Do we need cross-artifact annotation linking (a comment on hypothesis H-89 that references paragraph X of paper PMID:123)? (Proposed: yes in phase 2; anchor_json arrays support multi-target. Phase 1 does single-artifact only.)

    File: quest_commentary_curator_spec.md
    Modified: 2026-04-28 03:24
    Size: 16.4 KB