Quest: Code Health

Quest: Code Health

Vision

SciDEX grows fast — 367 Python files, 124K LOC, 30K-line god file, 48 duplicate
function definitions. This quest applies conservative, continuous pressure to improve
code quality without removing important functionality or destabilizing the system.

Principles:

Conservative first — never remove code that might be in use; archive, don't delete

Verify before acting — check git blame, imports, cron jobs, systemd before touching

Test after every change — all pages return 200, services healthy, API responds

Incremental — small, safe steps; never refactor everything at once

Never contradict SciDEX goals — code health serves the mission, not the other way around

Current State (Baseline Metrics)

Metric	Current	Target
Root-level Python files	305	<80
api.py line count	30,283	<5,000 (split into modules)
Duplicate get_db() definitions	48	1 (shared module)
Duplicate search_pubmed()	15	1
Duplicate extract_edges_from_abstract()	12	1
Version-suffixed dead code files	27+	0 (archived)
Migration number collisions	2	0
Linting config	None	ruff in pyproject.toml
Test files	7 (580 lines)	20+ (2K+ lines)
Files with raw sqlite3.connect()	287	<50 (core scripts only)

8 Tasks (Dependency Order)

Recurring Sweep ──> (informs priorities for all below)

Fix Migrations (independent, quick win)

Add Linting (independent, infrastructure)

Archive Dead Code ──> Organize File Sprawl

Consolidate DB Patterns ──┐
                          ├──> Break Apart api.py
Consolidate Utility Fns ──┘

Safety Protocol

Every code health change MUST:

Check references before removing/moving anything:

# Is this file imported anywhere?
   grep -r "import filename" --include="*.py" .
   grep -r "from filename" --include="*.py" .
   # Is it in a systemd service?
   grep -r "filename" /etc/systemd/system/scidex-*.service
   # Is it in a cron job?
   crontab -l | grep filename

Test after changes:

for page in / /exchange /analyses/ /graph /gaps /senate /forge; do
     code=$(curl -s -o /dev/null -w '%{http_code}' "http://localhost:8000${page}")
     echo "${code} ${page}"
   done
   curl -s http://localhost:8000/api/status | python3 -m json.tool

Commit atomically — one logical change per commit, easy to revert

Keep archive/ — moved files go to archive/ directory, not deleted

Key Files

• quality_gates.py — existing pre-merge quality gates (extend, don't replace)
• quality_review.py — existing syntax validation for core files
• api.py — primary refactoring target (30K lines)
• tools.py — second largest file (3,337 lines)

File: quest_code_health.md

Modified: 2026-04-25 22:00

Size: 3.0 KB