2578a570ca
CONTRIBUTING has always asked for the full real-hardware drill on a release. Nothing asserted it, so it was performed exactly as often as a reviewer remembered to ask — which is never, across every release in the family, until a reviewer bot finally blocked on it. The gate moves out of memory and into the tree. drill/RUNS.md is cast's own run log, starting empty: no fabricated history, and an honest note that cast has no drill harness script yet — its legs are run by the documented procedure. The file is the record, not the instrument. .github/scripts/drill-recorded.sh reads package.json and asserts that a bare version has a non-empty '## Release drill — X.Y.Z' section. A -dev tree has no ship claim and passes trivially. The version is matched WHOLE via awk field equality, release-notes.sh's fix for the same trap: 0.2.0 is not satisfied by 0.2.0-rc1, or the reverse. It requires a RECORD, not a PASS. A maintainer waiver is legal and is itself a section in drill/RUNS.md, so skipping the drill stays possible and stays a deliberate, reviewable commit rather than an oversight. The drill itself is ONE orchestrated run over the whole stack: rig bootstraps a bare host and installs box, box new mints a seed, the seed calls rig back to converge, and cast's legs run on the result. rig sits below box and above it, so the repos are mutually recursive rather than linearly ordered and their releases are not published in a fixed sequence. The run pins candidate refs (RIG_REPO/RIG_REF at mint time), so no repo must ship before another can be drilled, and drilling the candidate is drilling the release — a release diff is the version file and CHANGELOG.md, nothing executable. Each repo records its own legs from that run, citing the shared run ID and the other repos' SHAs. cast never reads box's or rig's drill log to decide whether cast may ship: a cross-repo lookup degrades to "pass" the moment it fails to resolve — the unreadable-rollup class. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
4.1 KiB
Drill runs
The log of cast's real-hardware drill legs — one section per run, appended.
A release PR's version must have a section here before CI will let it merge
(.github/scripts/drill-recorded.sh, wired into ci.yml).
This file is the record, not the instrument. cast has no drill harness script of its own yet. Its legs are run by hand, against the documented procedure: two live Coolify instances and the full A→B promotion —
team → apply → diff (idempotent) → smoke → inventory → emit-draft →
fleet → destroy → read-only guard
A harness would make the run reproducible; it would not make it recorded. Those are separate problems, and this file is the second one. When cast grows a harness, its output gets pasted into a section here in the same format — the gate does not change.
Per-repo, by construction
cast records cast's own legs. It does not read box's or rig's drill log to decide whether cast may ship: a cross-repo lookup silently degrades to "pass" the moment it fails to resolve — the unreadable-rollup class of bug, where a guard that cannot read its input reports the happy answer.
The legs above are the top of one orchestrated run over the whole stack
(CONTRIBUTING.md, Releasing): rig bootstrap --host yes on bare Debian
installs box and runs setup-host; box new mints a seed; the seed converges
by calling rig bootstrap <tenant>-box; cast's legs run on the result. rig
appears twice — below box and above it — so the three repos are mutually
recursive, not linearly ordered, and their releases are not published in a
fixed order.
The run drills candidate refs rather than released artifacts (RIG_REPO /
RIG_REF are mint-time variables, default heavy-duty/rig@main), which is
what makes that possible: no repo has to ship before another can be drilled.
A record therefore cites the shared run ID and the other two repos' commit
SHAs — three records, one run, reassemblable — while each repo's evidence
still lives in its own tree.
Format
The gate looks for a heading of exactly this shape, and requires the section under it to be non-empty:
## Release drill — X.Y.Z — YYYY-MM-DD
The version is matched whole: a 0.2.0 release is not satisfied by a
0.2.0-rc1 section, or the other way round.
A record says three things: what ran, the numbers, and what failed. "Nothing failed" is a finding and is worth a line; an empty section is not a record and the gate refuses it.
Example of the shape
Illustrative only — no run has happened yet, and the placeholder version keeps this block from ever satisfying the gate for a real release.
## Release drill — X.Y.Z — YYYY-MM-DD
Run ID: `drill-YYYYMMDD-NN` (the stack run these legs are the top of).
Stack under test: box `<sha>`, rig `<sha>`, cast `<sha>` — candidate
refs, pinned at mint time via RIG_REPO/RIG_REF.
Instances: A `coolify-a.example` (v4.x), B `coolify-b.example` (v4.x).
Manifest: `examples/two-env.yaml`, 3 applications, 2 environments.
Operator: @maintainer. Elapsed: 41m.
| Leg | Result | Notes |
|---|---|---|
| team | pass | 2 teams, 4 members reconciled |
| apply (A→B) | pass | 3 apps created, 11 env vars set |
| diff (idempotent) | pass | second apply: 0 changes |
| smoke | pass | 3/3 endpoints 200 |
| inventory | pass | 3 apps, 2 envs, matches manifest |
| emit-draft | pass | draft matches inventory round-trip |
| fleet | pass | both instances listed, versions read |
| destroy | pass | 3 apps removed, absence asserted |
| read-only guard | pass | write refused against B with the guard on |
Failures: none. One rough edge: `smoke` needed a 20s retry window on B —
filed as #NNN, not a release blocker.
Waivers
A maintainer may ship without a passing drill, but not without a record. The waiver is a section under the same heading saying who waived it, why, and what is untested — a deliberate, reviewable commit. The gate requires a record, not a pass, precisely so that skipping is visible.
Records
None yet. cast has recorded no drill runs. New runs are appended below, newest first.