af81f8f4cf
The security-trade paragraph lands verbatim in the README (its twin is in rig-templates' README, per the issue's docs task), with the 2026-07-24 ruling recorded in place.
123 lines
6.2 KiB
Markdown
123 lines
6.2 KiB
Markdown
# Drill records
|
|
|
|
Per-release evidence for rig's real-hardware drill. **One file per version**,
|
|
named for the version exactly as `VERSION` carries it:
|
|
|
|
drills/<version>.md
|
|
|
|
So `0.3.0` is recorded in `drills/0.3.0.md`, and `0.3.0-rc1` in
|
|
`drills/0.3.0-rc1.md`. They are different files, which is the whole point: the
|
|
filesystem does the whole-version comparison that an earlier single-file
|
|
version of this had to do with an awk extractor and a heading grammar. A
|
|
record for the candidate cannot be mistaken for evidence for the final.
|
|
|
|
The directory is `drills/`, not `.drills/` — a dot-directory is invisible to
|
|
any glob without `dotglob`, which is how #70 here and box#116 / box#118 all
|
|
happened.
|
|
|
|
**This directory is the record, not the instrument.** The instrument is
|
|
[`drill/drill.sh`](../drill/README.md) (#105): it runs the legs, asserts the
|
|
pinned refs actually landed, decides idempotence by a mechanical state diff,
|
|
and emits the record file this directory holds. rig does not reach into
|
|
another repo's harness to decide whether rig may ship: a cross-repo lookup
|
|
that fails silently degrades to "pass", which is the UNREADABLE-vs-NONE shape
|
|
#90 fixed. The gate reads a file in this repo, and nothing else.
|
|
|
|
## What the gate requires
|
|
|
|
The `drill-recorded` guard (heavy-duty/ceremony's action, pinned in
|
|
`ci.yml`) runs on every PR. On a `-dev` tree it
|
|
asserts nothing — a development tree has no release to evidence. On a bare
|
|
`VERSION` — a release ceremony tree — it requires `drills/<version>.md` to
|
|
exist and to hold at least one non-whitespace character. An empty file, or one
|
|
of only spaces and tabs, is not a record (box#149 and cast#138 both shipped an
|
|
extractor where a single tab satisfied the gate).
|
|
|
|
The guard requires a **record**, not a passing result. A maintainer waiver is
|
|
a legitimate outcome of a release — but it is written in that version's file,
|
|
so that skipping the drill is a deliberate, reviewable commit rather than a
|
|
silence. **A failed drill is still a valid record**: the gate wants evidence,
|
|
not success.
|
|
|
|
## The drill
|
|
|
|
rig's legs (#105; `drill/drill.sh` runs them):
|
|
|
|
- `rig bootstrap <role>` converges the machine to its role — then runs
|
|
**again**, and the captured state must diff **empty** (idempotence,
|
|
decided mechanically). On a host=yes role this is also what installs the
|
|
pinned box and asserts its host stack stands.
|
|
- `bash test/db-integration.sh` against a real Postgres on the machine
|
|
- the GitHub runner lifecycle — register, take a job, deregister — against a
|
|
fork
|
|
- a coolify install, pinned, `AUTOUPDATE=false`
|
|
|
|
box and rig are **mutually recursive**: `rig bootstrap --host yes` installs box
|
|
and runs box's `setup-host`, while box's guests converge back through rig's
|
|
installer. Within a single drill you naturally bring the substrate up before
|
|
probing it — a host before a guest — but that is how you run a drill, not an
|
|
ordering rule between repos.
|
|
|
|
**The three repos' drills are independent.** Run them in any order, on any
|
|
schedule, in separate sittings. What makes that safe is that every drill
|
|
**pins the same fixed set of candidate refs**: rig's drill runs `--host yes`
|
|
with `BOX_REF=release/<box-version>`, so it exercises the box that will
|
|
actually ship; box's drill mints with `RIG_REF=release/<rig-version>`, so it
|
|
exercises the rig that will actually ship. Both measure the same pair. The
|
|
record also cites the **rig-templates SHA** the converge read (#110) — the
|
|
candidate tree's `RIG_TEMPLATES_PIN` unless the drill was pointed elsewhere
|
|
via `RIG_TEMPLATES_REF` — so the mechanism+registry pair a release freezes
|
|
is the pair the drill proved.
|
|
|
|
That — not sequencing — is what dissolves the box↔rig recursion. The refs are
|
|
static identifiers that exist as soon as the release branches do, long before
|
|
any drill runs, so a cycle at runtime becomes two independent tests against
|
|
one fixed pair. It also means **candidate refs, not released artifacts**:
|
|
`RIG_REPO`/`RIG_REF` are mint-time environment variables, so no repo has to be
|
|
released before another can be drilled. Drilling the candidate *is* drilling
|
|
the release, because a release PR's diff is `VERSION` + `CHANGELOG.md` and
|
|
nothing executable differs.
|
|
|
|
Each repo drills in a **different way** and asserts a different thing: rig
|
|
asserts **convergence** (a machine reaches its role, idempotently), box asserts
|
|
the **isolation contract** (the VM trust boundary), cast asserts **promotion**
|
|
(A→B reproduces, the diff is idempotent). Three different exercises sharing a
|
|
substrate, not three phases of one script — which is exactly why the records
|
|
are per-repo.
|
|
|
|
Drills that share a substrate share **one run ID**; each repo records its own
|
|
legs in its own file, citing that run ID and the other repos' commit SHAs, so
|
|
the records can be joined after the fact. If a defect shows up only in the
|
|
combination: patch, re-drill, re-record. The three releases converge on a set
|
|
that holds together; they are not required to be right in one pass. Releases
|
|
do **not** have to be published in a fixed order.
|
|
|
|
## What a record should contain
|
|
|
|
What ran, on what host, the pinned candidate refs, the numbers, and what
|
|
failed. Below is the *shape*, in a file named `drills/9.9.9.md` — a version
|
|
that can never collide with a real release. **No drill has been recorded here
|
|
yet**; this log starts empty rather than reconstructing runs from memory, since
|
|
an invented number is worse than no number.
|
|
|
|
```markdown
|
|
# Release drill — 9.9.9 — 2026-01-01
|
|
|
|
Run ID: drill-2026-01-01-a. Host: bare Debian 13 cloud image, 4 vCPU / 8 GB.
|
|
Candidate refs: box@1a2b3c4 (BOX_REF=release/0.4.0), rig@5d6e7f8, cast@9a0b1c2.
|
|
|
|
| Leg | Result |
|
|
| --- | --- |
|
|
| convergence — bootstrap staging-server reaches its role | PASS (312s) |
|
|
| re-converge (idempotence) | clean, no changes |
|
|
| --host yes: pinned box installed, host stack up | PASS — box doctor clean |
|
|
| `test/db-integration.sh` | PASS — 14 passed, 0 failed |
|
|
| runner lifecycle against a fork | PASS — registered, took a job, deregistered clean |
|
|
| coolify install (4.1.2) | PASS (6 min) |
|
|
|
|
Failed: `rig users apply` left one revoked key in `authorized_keys`
|
|
(filed #NNN). Everything else clean.
|
|
```
|
|
|
|
State what failed. A record with no failures listed reads as "nothing broke",
|
|
so if a leg was not run, say that instead of omitting it.
|