cast/drills/README.md

# Drills

Per-release evidence: **one file per version**, named `<version>.md`, where
`<version>` matches `package.json`'s `version` exactly — `0.2.0` is recorded in
`0.2.0.md`, `0.2.0-rc1` in `0.2.0-rc1.md`.

A release PR's version must have its file here, holding at least one
non-whitespace character, before CI will let it merge
(`.github/scripts/drill-recorded.sh`, wired into ci.yml).

## One file per version, and why the parser went away

Records used to share a single `drill/RUNS.md`, which meant the gate had to
*parse* it: a heading grammar, an em-dash field match, an optional ` — DATE`
tail, a whole-version comparison, a non-blank-body rule. That machinery existed
only because records shared a file — and it shipped two defects in review: a
`sed '/./,$!d'` extraction where `.` matches a space (so a heading followed by
one tab satisfied the gate), and heading-grammar drift from box's stricter form.

One file per version makes nearly all of that unrepresentable. `0.2.0.md` and
`0.2.0-rc1.md` are different files, so the whole-version rule is the
filesystem's rather than a comparison anyone can get wrong, and there is no
heading to drift. One rule survives: a file of only whitespace is not a record.

The directory is plain `drills/`, **not** `.drills/`. Dot-prefixed directories
are invisible to globs without `dotglob` — the blind spot behind #118, #121 and
box#116.

## What a record should contain

- **What ran** — which legs, against which manifest.
- **On what host** — the instances, their versions, who operated it.
- **The pinned candidate refs** — box, rig and cast SHAs under test.
- **The numbers** — counts, elapsed time, whatever the legs emit.
- **What failed** — and "nothing failed" is itself a finding worth a line.

**A failed drill is still a valid record.** The gate wants evidence, not
success. A maintainer may ship on a failed or partial drill — what they may not
do is ship on silence, so a **waiver** is also a legitimate record: say who
waived it, why, and what is untested. Requiring a record makes skipping a
deliberate, reviewable commit instead of the default outcome of forgetting.

## This directory is the record, not the instrument

cast has **no drill harness script of its own**. Its legs run by hand against
the documented procedure: two live Coolify instances and the full A→B
promotion —

    team → apply → diff (idempotent) → smoke → inventory → emit-draft →
    fleet → destroy → read-only guard

A harness would make the run reproducible; it would not make it recorded. Those
are separate problems, and this directory is the second one.

## Per-repo, by construction

cast records **cast's own** legs. It does not read box's or rig's drill records
to decide whether cast may ship: a cross-repo lookup silently degrades to
"pass" the moment it fails to resolve — the unreadable-rollup class of bug,
where a guard that cannot read its input reports the happy answer.

**The three repos' drills are independent.** Run them in any order, on any
schedule, in separate sittings. They are not phases of one script.

What makes that safe is that every drill **pins the same fixed set of candidate
refs** (`RIG_REPO` / `RIG_REF` are mint-time variables, default
`heavy-duty/rig@main`), so each drill exercises exactly the combination that
will ship rather than whatever `main` happens to be that afternoon. The run
drills **candidate refs, not released artifacts**.

That pinning — **not sequencing** — is what dissolves the box↔rig recursion.
box and rig *are* mutually recursive: rig builds the host that runs box, and
box's seed calls rig back to converge the guest. But candidate refs are static
identifiers that exist as soon as the release branches do, long before any
drill runs, so a cycle at runtime becomes independent tests against one fixed
pair. No repo has to be released before another can be drilled, and there is
**no fixed order in which the three releases must be published**.

Each repo also drills a **different thing**: box asserts the isolation contract
(the VM trust boundary), rig asserts convergence (a machine reaches its role,
idempotently), cast asserts promotion (A→B reproduces, and the diff is
idempotent). Three different exercises sharing a substrate — which is exactly
why the records are per-repo.

cast's legs are the **least coupled** of the three. Two Coolify instances can
be stood up by hand; the July drill did exactly that for instance B, via a
parameterised compose file. Nothing about cast's drill requires box or rig to
have been drilled first, or at all, on that day.

Within a single drill you obviously bring the substrate up before probing it —
a host before a guest before Coolify. That is how you run *a* drill, not an
ordering rule *between repos*.

If a defect shows up only in the combination: patch, re-drill, re-record. The
three releases converge on a set that holds together; they are not required to
be right in one pass.

**Drilling the candidate is drilling the release.** A release PR's diff is the
version file and `CHANGELOG.md` — nothing executable differs between the tree
that was drilled and the tree that ships, so the evidence carries across the
ceremony commit.

Each record cites the shared **run ID** naming the pinned set, plus the other
two repos' SHAs — which is what lets separate records be reassembled into one
picture, while each repo's evidence still lives in its own tree.

## Worked example

Illustrative only. The version below is a **placeholder that can never collide
with a real release** — a realistic-looking version here would be a real record
for it, and the gate would wave that release through on documentation. The
mechanism changed with the move to one file per version; the caution did not.

`drills/9.9.9.md`:

```markdown
# Release drill — 9.9.9 — YYYY-MM-DD

Run ID: `drill-YYYYMMDD-NN` (names the pinned candidate set).
Stack under test: box `<sha>`, rig `<sha>`, cast `<sha>` — candidate refs,
pinned at mint time via RIG_REPO/RIG_REF.
Instances: A `coolify-a.example` (v4.x), B `coolify-b.example` (v4.x),
B stood up by hand from the parameterised compose file.
Manifest: `examples/two-env.yaml`, 3 applications, 2 environments.
Operator: @maintainer. Elapsed: 41m.

| Leg | Result | Notes |
|---|---|---|
| team | pass | 2 teams, 4 members reconciled |
| apply (A→B) | pass | 3 apps created, 11 env vars set |
| diff (idempotent) | pass | second apply: 0 changes |
| smoke | pass | 3/3 endpoints 200 |
| inventory | pass | 3 apps, 2 envs, matches manifest |
| emit-draft | pass | draft matches inventory round-trip |
| fleet | pass | both instances listed, versions read |
| destroy | pass | 3 apps removed, absence asserted |
| read-only guard | pass | write refused against B with the guard on |

Failures: none. One rough edge: `smoke` needed a 20s retry window on B —
filed as #NNN, not a release blocker.
```

## Records

*None yet.* cast has recorded no drill runs.