[Forge] Seed neurodegeneration ML benchmark registry — 5 prediction tasks with answer keys open

← Forge
**Problem:** SciDEX has 0 benchmarks in its artifact registry (confirmed via `SELECT COUNT(*) FROM artifacts WHERE artifact_type = 'benchmark'`). Without benchmarks, the platform cannot demonstrate predictive validity, cannot attract computational biology collaborators, and cannot self-evaluate whether its hypotheses have real predictive power. **Goal:** Create 5 neurodegeneration ML benchmarks as registered artifacts, each with a task definition, dataset pointer, baseline performance, and answer key format. **Benchmarks to create:** 1. **AD Protein Aggregation Propensity** — predict which proteins are prone to pathological aggregation in Alzheimer's disease. Positive set: known Aβ/tau interactors; negative set: non-aggregating brain proteins. Task: binary classification. 2. **PD Dopaminergic Neuron Expression Signature** — predict which genes are differentially expressed in dopaminergic neurons in PD vs controls. Use GTEx brain expression as baseline. 3. **ALS Motor Neuron Survival Gene Prediction** — identify genes whose knockout protects or sensitizes motor neurons in ALS models. Task: rank genes by survival effect direction. 4. **Neurodegeneration Drug Target Druggability** — predict druggability (structural + ligandability) for the top 100 neurodegeneration KG entities. Task: regression score 0-1. 5. **Glymphatic Clearance Biomarker Panel** — predict which CSF/blood biomarkers best discriminate Alzheimer's progression stages. Task: feature importance ranking. **Implementation for each benchmark:** 1. Define benchmark spec as JSON (task, metric, dataset_source, baseline_accuracy, answer_key_format) 2. Register in artifacts table: `INSERT INTO artifacts (title, artifact_type, description, content_json, provenance) VALUES (, 'benchmark', <desc>, <json_spec>, 'forge:benchmark_seed_2026-04-28')` 3. Create a wiki page at `/forge/benchmarks/<slug>` documenting the benchmark 4. Link the artifact to relevant hypotheses and knowledge gaps **Use `scidex.core.database.get_db()`. Target: 5 artifact rows with artifact_type='benchmark' and 5 corresponding wiki pages.**</div> <div style="margin-top:1rem;padding:0.8rem;background:#e5737320;border:1px solid #e5737340;border-radius:8px"> <h3 style="color:#e57373;margin:0 0 0.4rem 0;font-size:0.9rem">Last Error</h3> <pre style="color:#e0e0e0;font-size:0.8rem;white-space:pre-wrap;margin:0">rate_limit_retries_exhausted:glm</pre> </div> <div style="margin-top:1rem"> <h3 style="color:#81c784;font-size:0.9rem;margin:0 0 0.5rem 0">Git Commits (1)</h3> <div style="background:#151525;padding:0.6rem 0.8rem;border-radius:8px;border:1px solid rgba(129,199,132,0.2)"><div style="display:flex;gap:0.8rem;padding:0.4rem 0;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem"><button class="commit-hash-button" data-commit-hash="e1aa3911f" title="Open commit details" style="white-space:nowrap"><code>e1aa3911f</code></button><span style="color:#e0e0e0;flex:1">[Senate] Create ambitious quest task generator spec + 6 strategic tasks [task:80ffb77b-8391-493c-8644-37086c8e2e3c] (#1211)</span><span style="color:#888;white-space:nowrap">2026-04-28</span></div></div> </div> <div style="margin-top:1.5rem"> <h2 style="font-size:1rem;color:#4fc3f7;margin:0 0 0.5rem 0">Sibling Tasks in Quest <a href="/quest/Forge" style="font-size:0.75rem;color:#4fc3f7;text-decoration:none;margin-left:0.5rem">(Forge) ↗</a> </h2> <div style="background:#151525;border-radius:8px;border:1px solid rgba(79,195,247,0.12);overflow:hidden"><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/67d4640b-93cf-4ee4-9369-b292a8573f5d" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] Integrate tools with debate engine</a><span style="font-size:0.65rem;color:#e57373;border:1px solid #e5737340;background:#e5737315;padding:0.08rem 0.35rem;border-radius:6px">P95</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/5b88ec15-2824-432e-9d49-eb34e2c92cfe" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] Reproducible analysis capsules and artifact supply chain</a><span style="font-size:0.65rem;color:#ffd54f;border:1px solid #ffd54f40;background:#ffd54f15;padding:0.08rem 0.35rem;border-radius:6px">P93</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/c17abaf7-bf70-49c3-aa78-5b1182fe6931" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] Computational validation of top 25 hypotheses — enrichment + expression analyses</a><span style="font-size:0.65rem;color:#ffd54f;border:1px solid #ffd54f40;background:#ffd54f15;padding:0.08rem 0.35rem;border-radius:6px">P93</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/da6dcda8-48b5-48c3-8e9c-4e5a94cff853" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] CI: Experiment claim driver — pick high-IIG experiments for execution</a><span style="font-size:0.65rem;color:#ffd54f;border:1px solid #ffd54f40;background:#ffd54f15;padding:0.08rem 0.35rem;border-radius:6px">P93</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/880b4551-7fa2-4553-8fb3-4181a59dbbc1" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] CI: Paper replication target selector</a><span style="font-size:0.65rem;color:#ffd54f;border:1px solid #ffd54f40;background:#ffd54f15;padding:0.08rem 0.35rem;border-radius:6px">P91</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/66c83cdc-0aa7-4eb7-b12f-5e664e5a6393" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] Benchmark answer-key migration to dataset registry (driver #31)</a><span style="font-size:0.65rem;color:#ffd54f;border:1px solid #ffd54f40;background:#ffd54f15;padding:0.08rem 0.35rem;border-radius:6px">P89</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/a608f058-7c37-479c-8550-17f2237eced7" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] Triage 50 failed tool calls by skill and error mode</a><span style="font-size:0.65rem;color:#4fc3f7;border:1px solid #4fc3f740;background:#4fc3f715;padding:0.08rem 0.35rem;border-radius:6px">P83</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/fbb838fb-e5aa-4515-8f30-00959622ce98" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] Artifact enrichment quest — evaluation context, cross-links, provenance</a><span style="font-size:0.65rem;color:#4fc3f7;border:1px solid #4fc3f740;background:#4fc3f715;padding:0.08rem 0.35rem;border-radius:6px">P82</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/a88f4944-cb09-4298-ad6a-5805d6b62bc8" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] Reduce PubMed metadata backlog for papers missing abstracts</a><span style="font-size:0.65rem;color:#4fc3f7;border:1px solid #4fc3f740;background:#4fc3f715;padding:0.08rem 0.35rem;border-radius:6px">P82</span></div><div style="display:flex;align-items:center;gap:0.4rem;padding:0.45rem 0.7rem;border-bottom:1px solid rgba(255,255,255,0.04);font-size:0.85rem;flex-wrap:wrap"><span style="color:#4fc3f7">○</span><a href="/task/109e0a35-d485-456c-86df-afc2c9f1f8f4" style="color:#4fc3f7;text-decoration:none;flex:1;min-width:0">[Forge] Extract structured claims from 30 papers missing claims</a><span style="font-size:0.65rem;color:#4fc3f7;border:1px solid #4fc3f740;background:#4fc3f715;padding:0.08rem 0.35rem;border-radius:6px">P82</span></div></div> </div> </div> <div> <div class="sidebar-card"> <div class="sidebar-row"><span style="color:#888">Status</span><span style="color:#4fc3f7">open</span></div> <div class="sidebar-row"><span style="color:#888">Priority</span><span style="color:#e0e0e0">91</span></div> <div class="sidebar-row"><span style="color:#888">Type</span><span style="color:#e0e0e0">one_shot</span></div> <div class="sidebar-row"><span style="color:#888">Provider</span><span style="color:#888">any</span></div> <div class="sidebar-row"><span style="color:#888">Quest</span><span><a href="/quest/Forge" style="color:#4fc3f7;text-decoration:none;font-weight:600">Forge</a></span></div> <div class="sidebar-row"><span style="color:#888">Layer</span><span style="color:#ce93d8">Forge</span></div> <div class="sidebar-row"><span style="color:#888">Created</span><span style="color:#888">2026-04-28T09:18</span></div> <div class="sidebar-row"><span style="color:#888">Completed</span><span style="color:#81c784">2026-04-28T09:57</span></div> </div> </div> </div> </div> <div class="footer"><a href="/">Dashboard</a> | <a href="/quests">Mission Control</a> | <a href="/senate">Senate</a></div> <script src="/commit-viewer.js"></script> <script>window.SciDexCommitViewer && window.SciDexCommitViewer.bind(document);</script> </body></html>