SciDEX — Task: [Senate] Image quality evaluation pipeline via vis

Use GLM-4.6V-Flash (free) or sonnet-4.6 as vision evaluator. Score images on 4 dimensions (1-5): scientific_accuracy, clarity, informativeness, aesthetic_quality. Auto-regenerate below threshold (avg < 2.5). Log quality trends to data/metrics/image_quality.json. Update artifact quality_score from evaluation. ## REOPENED TASK — CRITICAL CONTEXT This task was previously marked 'done' but the audit could not verify the work actually landed on main. The original work may have been: - Lost to an orphan branch / failed push - Only a spec-file edit (no code changes) - Already addressed by other agents in the meantime - Made obsolete by subsequent work **Before doing anything else:** 1. **Re-evaluate the task in light of CURRENT main state.** Read the spec and the relevant files on origin/main NOW. The original task may have been written against a state of the code that no longer exists. 2. **Verify the task still advances SciDEX's aims.** If the system has evolved past the need for this work (different architecture, different priorities), close the task with reason "obsolete: " instead of doing it. 3. **Check if it's already done.** Run `git log --grep=''` and read the related commits. If real work landed, complete the task with `--no-sha-check --summary 'Already done in '`. 4. **Make sure your changes don't regress recent functionality.** Many agents have been working on this codebase. Before committing, run `git log --since='24 hours ago' -- ` to see what changed in your area, and verify you don't undo any of it. 5. **Stay scoped.** Only do what this specific task asks for. Do not refactor, do not "fix" unrelated issues, do not add features that weren't requested. Scope creep at this point is regression risk. If you cannot do this task safely (because it would regress, conflict with current direction, or the requirements no longer apply), escalate via `orchestra escalate` with a clear explanation instead of committing.

Last Error

watchdog repair f653b69f landed on main at 66c0760b; retry original task with PostgreSQL-safe image quality evaluator

Git Commits (2)

[Senate] Fix image quality evaluator PostgreSQL pipeline [task:692f13b8-876d-4139-8a8a-42f98cef3f46]2026-04-20

[Senate] Verify image quality pipeline on main — update work log [task:692f13b8-876d-4139-8a8a-42f98cef3f46]2026-04-17

Spec File

[Senate] Image quality evaluation pipeline via vision models

ID: 692f13b8-876 Priority: 72 Type: one_shot Status: open

Goal

Acceptance Criteria

☐ Concrete deliverables created

☐ Work log updated with timestamped entry

Work Log

2026-04-17 03:55 PT — Slot minimax:67

Status: Task already implemented on main — verification confirms NO new work needed
Evidence: scidex/forge/image_quality_evaluator.py (601 lines) exists on origin/main, authored in commits:

- 75f3726af — [Senate] Package refactor: scidex/ package with 6 layer subpackages + re-export shims
- 90f1ffff7 — [Senate] Prioritization run 57
- bd3b63bd8 — [Forge] Supervisor quota awareness (last update to this file)

Module verification: All functions available, 4 dimensions (scientific_accuracy, clarity, informativeness, aesthetic_quality) scoring 1-5, threshold at 2.5, auto-regeneration logic, metrics logging to data/metrics/image_quality.json, quality_score update in artifacts table
Result: Task already delivered — no changes needed

2026-04-17 04:17 PT — Slot minimax:62

Status: Verified on origin/main after rebase to 8ad527919
Verification: scidex/forge/image_quality_evaluator.py confirmed present with all functions:

- evaluate_image_quality() — GLM/Sonnet vision scoring on 4 dimensions
- should_regenerate() — avg < 2.5 triggers regeneration
- regenerate_image_with_improvements() — auto-regen with improvements
- update_artifact_quality_score() — updates artifacts table quality_score
- log_quality_metrics() — logs to data/metrics/image_quality.json
- evaluate_and_improve_artifact() — full pipeline
- batch_evaluate_artifacts() — CLI batch evaluation

Spec file only updated (no code changes needed — already on main)
Result: Task already complete — spec work log updated

2026-04-20 20:52 PDT — Slot codex:41 (Watchdog repair f653b69f)

Root cause: The image quality evaluator existed, but the PostgreSQL retirement exposed SQLite-era code paths: sqlite3.Row row factory assignment, PRAGMA journal_mode, SQLite json_set(), and direct json.loads() of PostgreSQL jsonb dict values. The previous task could be verified by grep but would fail when exercised against the live PostgreSQL artifacts table.
Fix: Updated scidex/forge/image_quality_evaluator.py to use PostgreSQL-safe metadata handling: parse jsonb/text fields through helpers, remove row factory and PRAGMA calls, merge quality evaluation metadata in Python, and write metadata with an explicit ?::jsonb cast.
Tests: Added tests/test_image_quality_evaluator.py; ran pytest -q tests/test_image_quality_evaluator.py (2 passed), python3 -m py_compile scidex/forge/image_quality_evaluator.py, and a live PostgreSQL smoke inserting/updating/deleting a temporary figure artifact (quality_score=0.75, metadata quality evaluation persisted).
Result: Original task should now retry successfully instead of failing on PostgreSQL/SQLite incompatibilities.

2026-04-21 04:36 UTC — Slot codex:43 (Watchdog repair e4b26d91)

Root cause: The evaluator code path was already PostgreSQL-compatible, but the task's verification surface still had a stale legacy helper at scripts/test_image_quality_evaluator.py. Running it directly failed because the repo root was not on sys.path, and collecting it together with tests/test_image_quality_evaluator.py failed with a pytest import-file mismatch because both files imported as test_image_quality_evaluator.
Fix: Renamed the legacy helper to scripts/check_image_quality_evaluator.py and added explicit repo-root sys.path setup so the legacy image_quality_evaluator shim imports reliably. This removes the duplicate pytest module name while preserving the manual verification helper.
Tests: pytest -q tests/test_image_quality_evaluator.py scripts/check_image_quality_evaluator.py (7 passed, warnings only because the legacy helper returns booleans); python3 scripts/check_image_quality_evaluator.py (5/5 passed); python3 -m py_compile scidex/forge/image_quality_evaluator.py image_quality_evaluator.py; live PostgreSQL smoke inserted/evaluated/deleted temporary figure artifact tmp-image-quality-watchdog-e4b26d91 and verified quality_score=0.75, metadata quality_evaluation, and metrics output.
Result: Original task can retry without the verification collision/import failure; evaluator functionality is verified against the live PostgreSQL path.

Payload JSON

{
  "requirements": {
    "coding": 7,
    "analysis": 6
  },
  "_reset_note": "This task was reset after a database incident on 2026-04-17.\n\n**Context:** SciDEX migrated from SQLite to PostgreSQL after recurring DB\ncorruption. Some work done during Apr 16-17 may have been lost.\n\n**Before starting work:**\n1. Check if the task's goal is ALREADY satisfied (run the relevant checks)\n2. Check `git log --all --grep=task:YOUR_TASK_ID` for prior commits\n3. If complete, verify and mark done. If partial, continue. If not done, proceed.\n\n**DB change:** SciDEX now uses PostgreSQL. `get_db()` auto-detects via\nSCIDEX_DB_BACKEND=postgres env var.",
  "_reset_at": "2026-04-18T06:29:22.046013+00:00",
  "_reset_from_status": "done",
  "_watchdog_repair_task_id": "e4b26d91-9c32-4c2e-8f43-d43223074123",
  "_watchdog_repair_created_at": "2026-04-21T04:08:07.320948+00:00",
  "_watchdog_repair_commit": "f4819db0429c6ecae8e5fac53deb4431f790e6ae"
}