Skip to content

v3.2.0 release runbook — the ripple inventory

Compiled 2026-08-02 by a read-only cross-repo sweep, before the v3.2.0 scaled region eval ran. This is a point-in-time inventory: file:line citations were correct on that date and drift as files change — verify each against the code before acting on it. The dependency chain at the bottom is the operative part; everything above it is the evidence.

Scope: every surface that moves when this repo ships v3.2.0 (the scaled region eval + version bump). Items are marked mechanical (a checklist edit) or human (needs judgment or owner sign-off). Two decisions are deliberately NOT made here — see "Design forks" at the end.

Repo 1 — defense-news-classifier (first, in this order)

A. Mechanical, CI-enforced (order matters)

  1. pyproject.toml:3 — version = "3.1.0" → 3.2.0. Single source; scripts/gen_metrics_artifact.py (_project_version()) reads it.
  2. Regenerate evals/metrics.json via uv run python scripts/gen_metrics_artifact.py. Until then the "Metrics artifact is current" step in tests.yml is red. The gold block is computed from evals/gold_predictions_v3.csv (n=54) — a scaled region eval changes no gold value, so the only byte that moves is version. That byte-identical-except-version diff is the standing proof the published numbers still describe what ships. Blocking precondition: main() runs provenance.check() before writing. If this release touched classify.SYSTEM_PROMPT or the model constants, regeneration hard-fails until a paid gold re-run or a waiver. Human judgment if it fires.
  3. tests/test_metrics_artifact.py:83 — pinned "3.1.0" literal → "3.2.0", in the same commit as items 1–2 or CI is red either way. Its docstring ("measured at v3.0.0… v3.1.0 added eval machinery only") needs a human-written line for what v3.2.0 measured.
  4. src/api.py:40 — version="3.1.0" on the FastAPI app. No test pins this (grep-confirmed). Mechanical but silent — easy to miss.
  5. scripts/gen_readme_metrics.py --check — verified: a pure version bump is a no-op for the README metrics block (version appears only in the print message). This gate does not go red.
  6. contracts/classify-response.schema.json — untouched by a version bump (schema is derived from model fields + label enums, carries no version). kb-agent's and notes-api's contract checks both stay green. Nothing on the SYS-004 seam moves for a MINOR.

B. Release paperwork (mechanical shape, human content)

  1. CHANGELOG.md — [Unreleased] already holds shipped work that rides the v3.2.0 tag. Add the ## [3.2.0] heading; update the link refs at the bottom ([Unreleased]: …compare/v3.1.0...HEAD → v3.2.0...HEAD, plus a new [3.2.0] compare line).
  2. CLAUDE.md — the "Not yet shipped" bullet (~124-128) is the scaled region eval; it must move out. MINOR is correct per the versioning table (additive, contract untouched).
  3. A new ADR for the scaled region eval takes the next free number and must carry ## Downstream surfaces (scripts/lint_decisions.py grandfathers only 001–014).
  4. Tag + GitHub release — per SYS-020 this repo is the reference implementation. Tags never work from containers (proxy 403) — tag pushes are owner-only (HANDOFF). Hand San the exact commands.

C. Prose that goes stale (human judgment)

  1. README.md:38,40-41 — "Current state — v3.1.0" heading + paragraph.
  2. README.md:74 — already stale today: cites the scaled region eval as "(v3.1.0)"; the renumber missed this line.
  3. README.md:93 — already stale today: "no region labels until v3.1.0".
  4. README.md:161-182 — the v2.1.0 "Tighter error bars: the scaled eval" section; the new region CI table plausibly belongs beside it. Placement and framing are human calls.
  5. Version-free "until the scaled region eval ships" phrasings become factually stale the moment it ships: src/ml_loop.py:40, src/baseline_ml.py:24,239, docs/specs/l4-multi-agent.md:110, docs/specs/ml-baseline-bakeoff.md:65. These are disclosed-limitation statements in live code/specs, not frozen records — update them.
  6. HANDOFF.md — post-release rewrite: the "v3.1.0 is shipped" line, the roadmap-renumber note, Job 1 (which is the thing just shipped), the owner-only pending list. The version-bump caution (~136-140) is correct and stays.
  7. DO NOT TOUCH — frozen records: evals/gold_eval_v3.txt, evals/baseline_eval.txt, evals/l4_eval.txt, and every ADR that says "v3.1.0" (ADRs are correct on their date; this repo amends decision records rather than rewriting them).

Repo 2 — architecture (only AFTER the classifier's metrics.json is on main)

  1. The reddening item: program/README.md:54 — <!-- version:classifier -->**v3.1.0** → **v3.2.0**. The guard (scripts/check_program_metrics.py) asserts it against the live-fetched evals/metrics.json on the classifier's main. Constraints:
    • Fix it before the classifier ships and the failure inverts (prose 3.2.0 vs artifact 3.1.0). Order matters.
    • raw.githubusercontent serves Cache-Control: max-age=300; allow ~5 minutes after the classifier merge before expecting green.
    • Never wrap the value in backticks — the guard strips code spans and a backticked marker matches nothing.
  2. program/README.md:60 — already stale today: the roadmap bullet says "v3.1.0, scale the region eval… unscheduled" (backticked, invisible to the guard). Must become shipped + right version. The <!-- metric:judge_region_agreement -->100.0% marker on the same line stays valid. This bullet also feeds the generated portal roadmap.
  3. program/README.md:77 (risk R2) — six gold markers, unchanged by v3.2.0. But R2's narrative (the region ceiling / one named cluster) is exactly what a scaled number updates. Human judgment.
  4. product/one-pager.md and case-study/README.md — marked gold metrics only; unchanged. (case-study/README.md:90 says "at v3.0.0" — backticked, ungated, cosmetically stale; optional.)
  5. Portal (portal/**, site/**) is generated by scripts/build_portal.py on push to architecture main — it re-clones the classifier and self-heals when item 18 merges. Never hand-edit.
  6. Nothing else in architecture pins the classifier version, schema, or a metric (swept; SYS pages and mkdocs references are name-only).
  7. When does it actually go red? checks.yml runs on push, PR, dispatch, and a Monday 13:00 UTC cron. If nothing pushes to architecture, the red surfaces at the next Monday cron — it can sit red-but-unobserved for up to a week. Do item 18 promptly after the classifier ships; don't let the cron find it first.

Repo 3 — portfolio (verified: no breakage)

  1. New keys in metrics.json are backward-compatible with scripts/check-published-metrics.cjs (unknown-key failure fires when a page names a key the artifact lacks, never the reverse). A version bump alone is invisible to it (no version-marker concept). All 16 existing markers stay green because no gold value moves.
  2. Stale prose (unguarded, human judgment, post-run): the classifier project page's "scaled run is earned, not assumed" gate paragraph (src/pages/projects/defense-news-classifier.astro:305-312) becomes "the run happened"; ADR-006's "25 metrics, v3.1.0" aside is dated prose in a decision record — leave or amend deliberately.
  3. If the site publishes n=300 region numbers: precedent is the bake-off's frozen figures — bare prose, no data-metric marker (a marker for a key the artifact lacks fails the build). See Design fork 1.

Repo 4 — the long tail

  1. Highest-value stale item: kb-agent/kb/projects/defense-news-classifier.md:5 — the KB stub an agent will speak: "Measured results at v3.0.0… Region has no scaled number yet — that is v3.1.0 and unscheduled." Version label and roadmap slot already wrong; the claim becomes false at v3.2.0. Nothing guards this file.
  2. kb-agent freshness manifest: the classifier README edits land inside the first 8000 chars, so ingest.py --check flips the stub stale (manual check — not wired into CI). Remedies: hand-edit-then---accept (preferred: keeps hand-curation), --accept alone (blesses without fixing content), or --force (regenerates, loses hand edits). After any stub edit, re-run scripts/index.py or ChromaDB serves the old text.
  3. learning-notes/03-reading-the-numbers.md — six gold markers, gated by its own checker, unchanged → stays green. The checker walks top-level *.md only and has no version concept.
  4. learning-notes/glossary.md:97,119 — figures still correct but labeled "v3.0.0", two releases behind; source of truth for the tooltip feature. Human judgment.
  5. learning-notes/15-semantic-versioning.md:34,44 — teaches semver on this repo with "Planned v3.0.0" as the example; a v3.2.0 MINOR is a natural refresh. Human judgment.
  6. notes-api: nothing to do (its only coupling is the unchanged contract).
  7. Clean (verified negative): .env* templates, all classifier workflow YAML, docker tags, agent-ops (then named claude-ops), career (private, out of scope), lockfile version coincidences elsewhere.

The dependency chain (operative summary)

pyproject.toml:3
  → gen_metrics_artifact.py (rewrites evals/metrics.json version)
    → tests/test_metrics_artifact.py pinned literal   [SAME commit, or CI red]
  → src/api.py:40                                     [independent, unguarded]
  → gen_readme_metrics.py --check                     [verified no-op]
  → CHANGELOG + link refs → ADR (with Downstream surfaces) → prose sweep
  → tag + GH release                                  [OWNER-ONLY]
      ↓ classifier main serves 3.2.0; wait ~5 min for the raw CDN
architecture/program/README.md:54 + :60 (one PR)      [do promptly; Monday cron
      ↓ portal self-heals on that push                 otherwise finds the red]
kb-agent: hand-edit stub → ingest.py --accept → index.py
portfolio / learning-notes: green throughout; prose edits human-judged

Out-of-order costs: editing architecture first inverts the red; bumping pyproject without regenerating the artifact AND updating the pinned literal in the same commit reds this repo's own tests — which HANDOFF already records as "the guard working, not a bug."

Design forks (owner decisions, deliberately not made here)

  1. Where do the n=300 region numbers get published? They cannot be data-metric/metric: marked anywhere (no artifact key exists), and architecture's UNMARKED_ALLOWED ratchet is shrink-only (program/README.md is allowed exactly 2 unmarked figures — the v2.1.0 scale numbers). Options: (a) bake-off precedent — frozen dated figures in this repo's own README/eval artifacts, bare prose elsewhere, architecture prose points at the classifier rather than quoting; (b) extend metrics.json with scale keys — a deliberate SYS-019 amendment (it currently publishes the live gold block only, and the bake-off deliberately kept frozen figures out); (c) raise the ratchet — against its own shrink-only rule. Recommendation on record: (a).
  2. Does the scale run get a CI floor in thresholds.toml? Default no — the scale eval is a dated measurement, not a live gate, and the v2.1.0 scale numbers have no thresholds either. Adding one is an owner decision per HANDOFF's escalation rule.