Auto-Push for CVE Fixes: Guardrails, a Local LLM, and the 96/100 Pilot

Manually remediating 26 image tags means 26 rounds of scan, triage, rebuild, rescan, push. So we built cve-agent: a staged loop that does all five, asks a local LLM to audit the triage, and pushes to Quay itself — but only through gates that are code, not prompts. Then we pointed it at a crash-prone Mac LLM host and judged the whole thing. It scored 96/100, and its first live push caught a real bug in itself.

Scope: quay.io/hellodk/base-image · Trivy 0.70.0, 2026-10-11 DB · LLM at http://192.0.2.64:8080 (llama.cpp, Apple GPU) · agent defaults to push dry-run, auto-push is one env flip.

1. The Feature

Auto-push is not “the LLM pushes images”. It is a pipeline with eight stages — DISCOVER → SCAN → TRIAGE → PLAN → FIX → VERIFY → PUSH → REPORT — where the model acts in exactly one of them, and seven guardrails decide everything that matters:

Guardrail	What it enforces (in code)
G1 No overwrite	New tags bump (base2→base3, max +5) and are registry-checked before any plan is made
G2 Allowlisted mutations	Only apk/dnf upgrade, pull-refresh rebuild, reverse-dep-verified removal, byte-identical JAR transplant
G3 Push gate	Zero HIGH — or every remaining HIGH has no published fix and the total decreased; any new CRITICAL or failed functional probe blocks the push (§3)
G4 One push, labeled	At most one push per image per run; provenance OCI labels mandatory (managed-by, run-id, base-ref, base-digest, strategy)
G5 Secrets	Environment only, redacted from every log and report
G6 Validated LLM output	Schema-checked; invalid output falls back to conservative needs-human
G7 Dry-run default	Push plans are computed and logged; nothing leaves the box unless PUSH_MODE=auto

Pipeline: LLM Audits Once, Gates Decide Everywhere

2. Building It

1. Scaffolded the loop — config-driven targets, Trivy/docker/Quay tool layer, OpenAI-compatible client with token accounting, and a judge scoring accuracy/efficiency/methodology.
2. v1 failed honestly. LLM-led triage on the small local model classified everything “accept” and proposed verify-only for a 587-finding image. The guardrails contained it (no bad push), but containment isn’t performance — so the architecture flipped: code classifies from feed facts, the model only audits (confirm, narrow to needs-human, or write notes). Widening is structurally impossible.
3. Piloted on three images — partial-fix (ubi9-base: 587→572, both openssl HIGHs fixed, boots), unfixable (ubi-micro-base: refresh-compare caught a real upstream change, 56→38, and escalated with evidence instead of acting on stale data), clean (temurin-base2: verified zero, touched nothing).
4. Judging caught a real bug: digests were read from the image Config block, which has no digest field — every run pinned an empty string. Fixed, proven, and it cost 4 methodology points on the way: 96/100 (50 accuracy, 20 efficiency, 26 methodology).
5. Armed the push (PUSH_MODE=auto) on the smallest live target first — and the first attempt immediately caught a second real bug (§5).

3. The Push Gate

G3 is a pure function of two scans plus a functional probe — no model output anywhere in it:

# allow push iff:
new_critical == 0 and functional_probe_ok and (
    remaining_high == 0
    or (all_remaining_high_have_no_published_fix
          and total_findings_decreased))

That middle branch is the whole philosophy: a Red Hat image whose only remaining HIGHs are pcre2 with no erratum in any feed is improved and shippable; an image with one fixable HIGH left is not, no matter how small its total. Tag bumps (base2→base3, registry-checked, max +5) mean a published digest is never mutated — the blog tables that pin digests stay true forever.

4. Local LLM Wiring

The model host is a Mac Mini on the LAN serving OpenAI-compatible chat (llama.cpp, Apple GPU) with five models listed. Probing each one live gave an unambiguous stability matrix:

Model	Result
Qwen3-1.7B-4bit	Stable — ~12/12 calls clean, ~45 tok/s. The only one the agent may use.
Qwen3.5-4B-4bit	Flaky — one valid 318-token triage, then crash-looped the server on its first real run.
Qwen3.5-9B-4bit, Qwen3.5-4B-8bit	Unusable here — inference crash-loops the host, repeatedly verified.
gemma-4-e2b-it-4bit	Answers tiny probes; unproven for work.

Wiring it into opencode

The global opencode.jsonc already carried that host twice (vllm-metal → :8080, mlx-gemma → same box). We added exactly one entry — the stable tiny model — to vllm-metal:

"mlx-community/Qwen3-1.7B-4bit": {
  "name": "vllm-metal (Qwen3-1.7B-4bit, stable tiny, same host)",
  "limit": { "context": 32768, "output": 4096 }
}

The 9B/8B entries were deliberately not added: putting crash-prone models in the picker is inviting breakage. Restart opencode to pick it up.

Wiring it into the agent

Four env vars, no secrets in repo: LLM_BASE_URL (default the Mac), LLM_MODEL (default the 1.7B), PUSH_MODE, QUAY_* creds. Three small-model tactics make 1.7B viable: condensed inputs (package-group rows, ~40 lines instead of ~600 CVE rows), an if/then audit prompt with a worked example, and a JSON-repair retry plus backoff’d retries degrading to needs-human — which survived three real server outages mid-session.

5. Proof Push

First live target, smallest blast radius: python-3.14-base (1 MEDIUM, zlib CVE-2026-85091). The agent scanned, triaged os-upgrade-apk, bumped past the existing base2 to base3, rebuilt, rescanned 1 → 0, probed Python 3.14.8, passed G3, and pushed quay.io/hellodk/base-image:python-3.14-base3 — independently re-pulled and rescanned clean, all five provenance labels present, running as app.

6. Reproduce

# unit gates (no network) + dry-run pilot
python3 test_guardrails.py
python3 run.py --config config.yaml
# arm the push once dry-run is green
PUSH_MODE=auto python3 run.py --config config.yaml --only <tag>
# score it
python3 judge.py /tmp/opencode/cve-agent/<run>/report.json

Repo: dockerfiles/cve-agent/ (committed). Registry: quay.io/hellodk/base-image. Known next work, deliberately untouched: the ubi-micro-fixed3 rebuild the agent escalated, mislabeled -base tags that carry JARs, and dead-tag deprecation.