refactor: one drill record per version, in drills/ #151

Merged
dan-claude-bot merged 2 commits from refactor/drills-per-version into main 2026-07-21 17:41:42 +00:00
6 changed files with 364 additions and 172 deletions

View file

@ -1,35 +1,53 @@
#!/usr/bin/env bash #!/usr/bin/env bash
set -euo pipefail set -euo pipefail
# drill-recorded.sh [<runs-file>] [<version-file>] — assert that a RELEASE # drill-recorded.sh [<drills-dir>] [<version-file>] — assert that a RELEASE
# tree carries a drill record: that the full real-hardware drill this repo # tree carries a drill record: that the full real-hardware drill this repo
# says a release rests on was actually run for this version, and written # says a release rests on was actually run for this version, and written
# down in drill/RUNS.md. # down as drills/<version>.md.
# #
# CONTRIBUTING.md has said since #96 that "this PR is where the release # CONTRIBUTING.md has said since #96 that "this PR is where the release
# ritual hangs: the full drill on real hardware, recorded in drill/RUNS.md". # ritual hangs: the full drill on real hardware, recorded". No release ever
# No release has ever done it. #95, #114 and #148 all shipped as a VERSION # did it. #95, #114 and #148 all shipped as a VERSION bump plus a
# bump plus a CHANGELOG.md stamp and nothing else, and the file they were # CHANGELOG.md stamp and nothing else. That is three releases through the
# supposed to append to has no '## Release drill' section anywhere in it. # same gap, because the gate was a sentence in a document and the only thing
# That is three releases through the same gap, because the gate was a # standing on it was a reviewer remembering to ask. A reviewer finally did —
# sentence in a document and the only thing standing on it was a reviewer # which is the point: the ONE time it was caught is the time somebody
# remembering to ask. A reviewer finally did — which is the point: the ONE # happened to look, and that is not a gate, it is luck with good manners.
# time it was caught is the time somebody happened to look, and that is not a
# gate, it is luck with good manners.
# #
# So the rule moves into CI, where it fires on every release PR whether or # So the rule moves into CI, where it fires on every release PR whether or
# not anyone is paying attention. The rule, keyed on VERSION for the same # not anyone is paying attention. It is keyed on VERSION for the same reason
# reason changelog-armed.sh is — the two states are genuinely different: # changelog-armed.sh is — the two states are genuinely different:
# #
# VERSION ends in -dev -> PASS. A development tree ships nothing, so # VERSION ends in -dev -> PASS. A development tree ships nothing, so
# there is nothing for it to have proven. Almost # there is nothing for it to have proven. Almost
# every PR in this repo is this case, and a guard # every PR in this repo is this case, and a guard
# that nagged all of them would be turned off. # that nagged all of them would be turned off.
# VERSION is bare -> the ceremony tree, the one about to ship. # VERSION is bare -> the ceremony tree, the one about to ship.
# drill/RUNS.md MUST carry a section headed # <drills-dir>/<version>.md MUST exist and carry
# '## Release drill — <version>' (optionally with # at least one non-whitespace character.
# ' — <date>' after it), and that section must #
# have prose in it. # ONE FILE PER VERSION, and that is the whole design. Records used to be
# sections sharing drill/RUNS.md, and every hard edge this script used to
# have existed only because of that sharing: em-dash field matching, an
# optional ' — DATE' tail, whole-version comparison so '0.9.0-rc1' could not
# satisfy '0.9.0', avoiding '\x' escapes because CI runs mawk not gawk, and a
# non-blank body rule to tell an empty section from a filled one. Two
# separate defects were found in review because of that complexity — a
# `sed '/./,$!d'` whitespace bypass, and heading-grammar drift from the
# sibling repos. Splitting the file makes almost all of it UNREPRESENTABLE:
# '0.9.0.md' and '0.9.0-rc1.md' are simply different files, so whole-version
# matching is free rather than a trap, and there is no grammar left to drift.
#
# The directory is plain 'drills', NOT '.drills'. A dot-directory is
# invisible to a glob without dotglob, which is exactly what caused #116 and
# #118 in this repo; evidence a sweep cannot see is evidence that goes
# missing quietly.
#
# drills/ is RELEASE EVIDENCE, one file per shipped version. It is NOT
# drill/RUNS.md, which stays exactly as it is: the harness's own run log,
# traps table and lore, a different artifact with a different purpose. This
# guard reads drills/ and never looks at drill/RUNS.md.
# #
# What this guard asserts is a RECORD, deliberately — not a passing drill. # What this guard asserts is a RECORD, deliberately — not a passing drill.
# CI cannot run the drill: it wants real hardware, a real Incus, and the # CI cannot run the drill: it wants real hardware, a real Incus, and the
@ -37,16 +55,19 @@ set -euo pipefail
# rehearsal job it runs instead). What CI can do is refuse to let a release # rehearsal job it runs instead). What CI can do is refuse to let a release
# claim a ritual it left no evidence of. That also leaves the maintainer # claim a ritual it left no evidence of. That also leaves the maintainer
# waiver intact and honest: a release that must ship without a full drill # waiver intact and honest: a release that must ship without a full drill
# records WHY under its own heading, which is a deliberate, reviewable commit # records WHY in its own file, which is a deliberate, reviewable commit in
# in the diff — rather than the silent skip that got us here. # the diff — rather than the silent skip that got us here.
# #
# A file of its own (not inlined in ci.yml) so test/release.sh can drive it # A file of its own (not inlined in ci.yml) so test/release.sh can drive it
# against constructed trees for both states — the same discipline as # against constructed trees for both states — the same discipline as
# changelog-armed.sh and release-notes.sh. # changelog-armed.sh and release-notes.sh.
runs="${1:-drill/RUNS.md}" drills="${1:-drills}"
version_file="${2:-VERSION}" version_file="${2:-VERSION}"
# A missing or empty version file is an ERROR, never a silent pass. A guard
# that cannot read the version cannot know whether this tree is its business,
# and "could not tell" must not resolve to "allowed".
[ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; } [ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; }
ver="$(tr -d '[:space:]' < "$version_file")" ver="$(tr -d '[:space:]' < "$version_file")"
@ -62,67 +83,43 @@ case "$ver" in
;; ;;
esac esac
[ -f "$runs" ] || { echo "drill-recorded: no such file: $runs" >&2; exit 1; } record="$drills/$ver.md"
# The section for this version: everything between its own heading and the # The one rule that survives the rewrite, and it survives because it was
# next '## '. The version is compared WHOLE — as a field, never as a # never really about heading parsing: a file of only spaces, tabs and
# substring or a regex — so '0.9.0' can never be satisfied by a # newlines is NOT a record. The first cut of the old guard extracted with
# '0.9.0-rc1' section (or vice versa), and there are no dots to escape. # `sed '/./,$!d'`, where `.` matches a space — so a record whose body was one
# release-notes.sh solves the identical trap the identical way; the two # tab satisfied a guard that promised "at least one non-blank line". An
# scripts must not disagree about what "the section for X" means. # evidence-free release for the price of an invisible character, on the one
# # check whose entire job is to demand evidence. Existence alone is a weaker
# The heading shape is '## Release drill — <ver>' with an OPTIONAL # claim than `touch` can defeat, so existence alone is not the test.
# ' — <date>' tail, i.e. fields: '##' 'Release' 'drill' '—' '<ver>' ['—' ...]. if [ ! -f "$record" ] || ! grep -q '[^[:space:]]' "$record"; then
# Pinning the leading fields as well as the version keeps some other '## '
# heading that merely mentions the number from counting as a record.
#
# The em dash is passed IN as a variable rather than written into the awk
# program, because '\x' escapes in an awk string are a gawk extension and CI
# runs on ubuntu-latest, where awk is mawk.
#
# `grab && NF` is the non-blank rule, and it is load-bearing rather than
# tidiness. NF is 0 on a line that is empty OR contains only whitespace, so
# `record` ends up non-empty exactly when a line with actual content exists.
# The first cut of this piped through `sed '/./,$!d'` instead, which drops
# leading blank lines but keeps a line of spaces — `.` matches a space. A
# heading followed by nothing but a tab therefore satisfied the guard and
# shipped an evidence-free release, while the failure text below promised "at
# least one non-blank line". The implementation was looser than its own
# contract, which on a gate is the whole ballgame: the bypass costs one
# invisible character. Caught by all three reviewers on #149, independently.
record="$(awk -v ver="$ver" -v dash="—" '
/^## / {
grab = ($2 == "Release" && $3 == "drill" && $4 == dash && $5 == ver \
&& (NF == 5 || $6 == dash))
next
}
grab && NF { print }
' "$runs")"
if [ -z "$record" ]; then
cat >&2 <<EOF cat >&2 <<EOF
drill-recorded: VERSION is '$ver' — a release — and $runs has no drill record drill-recorded: VERSION is '$ver' — a release — and there is no drill record
for it. The heading this looks for is: for it. The file this looks for is:
## Release drill — $ver — DATE $record
...with at least one non-blank line under it. Either the section is absent ...with something written in it. Either the file is absent entirely, or it
entirely, or it is present and empty; both mean the same thing, which is is present and blank; both mean the same thing, which is that this release
that this release is asserting a ritual it has left no evidence of. is asserting a ritual it has left no evidence of.
The unblock is to RUN THE DRILL (drill/drill.sh, on real hardware) and The unblock is to RUN THE DRILL (drill/drill.sh, on real hardware) and
record it in $runs under that heading — what it measured, what it found, record it at that path — what it measured, what it found, what it cost.
what it cost. CI cannot run the drill for you; it can only refuse a release See $drills/README.md for what a record should contain. CI cannot run the
that never ran one. drill for you; it can only refuse a release that never ran one.
(drill/RUNS.md is a different artifact — the harness's own run log and
traps. Appending there does not satisfy this guard, and is not meant to.)
If this release must ship without a full drill, that is a maintainer's call If this release must ship without a full drill, that is a maintainer's call
to make and it is still recorded: write the section under the same heading to make and it is still recorded: create the same file and say plainly that
and say plainly that the drill was WAIVED and why. The guard requires a the drill was WAIVED and why. The guard requires a record, not a passing
record, not a passing result — so a skip is a visible, reviewable line in result — so a skip is a visible, reviewable file in the diff rather than
the diff rather than the silent gap that let #95, #114 and #148 all ship the silent gap that let #95, #114 and #148 all ship unproven. See
unproven. See CONTRIBUTING.md, "Releases". CONTRIBUTING.md, "Releases".
EOF EOF
exit 1 exit 1
fi fi
echo "drill-recorded: VERSION '$ver' has a drill record in $runs" echo "drill-recorded: VERSION '$ver' has a drill record at $record"

View file

@ -17,7 +17,7 @@ which records not just what changed but what each drill run proved.
- Every fresh mint marks a `pristine` snapshot, before rig converges anything - Every fresh mint marks a `pristine` snapshot, before rig converges anything
(#104, heavy-duty/rig#62) (#104, heavy-duty/rig#62)
- A mint that converges a tenant role marks a `bootstrapped` snapshot (#130) - A mint that converges a tenant role marks a `bootstrapped` snapshot (#130)
- CI refuses a release PR with no drill record in `drill/RUNS.md` - CI refuses a release PR with no drill record at `drills/<version>.md`
### Changed ### Changed

View file

@ -163,39 +163,70 @@ A release is a PR, and merging it ships it
real hardware and the better part of an hour, so it runs on the release real hardware and the better part of an hour, so it runs on the release
PR's branch and nowhere else. PR's branch and nowhere else.
Record it in [drill/RUNS.md](drill/RUNS.md) under its own heading: Record it in its own file, named exactly for the version:
```markdown ```
## Release drill — X.Y.Z — DATE drills/X.Y.Z.md
``` ```
...with the run under it — what it measured, what it found, what it cost. ...with the run in it — what it measured, what it found, what it cost.
See [drills/README.md](drills/README.md) for the shape.
[.github/scripts/drill-recorded.sh](.github/scripts/drill-recorded.sh) [.github/scripts/drill-recorded.sh](.github/scripts/drill-recorded.sh)
asserts exactly that on every tree with a bare `VERSION`, which is every asserts exactly that on every tree with a bare `VERSION`, which is every
`release` PR and the merge that publishes it; a `-dev` tree passes with `release` PR and the merge that publishes it; a `-dev` tree passes with
nothing to assert. It is **no longer a thing a reviewer has to remember**. nothing to assert. It is **no longer a thing a reviewer has to remember**.
**`drills/` is release evidence; [`drill/RUNS.md`](drill/RUNS.md) is the
harness's own log.** Two artifacts, two purposes. `drill/RUNS.md` keeps
being what it has always been — every run of the harness, the traps table,
the lore — and is not release-scoped. `drills/X.Y.Z.md` is the per-release
record the gate reads. Appending to `drill/RUNS.md` does not satisfy the
gate, and is not meant to.
One file per version is also why the guard is short. When records shared a
file it had to match em-dash heading fields, tolerate an optional date
tail, and compare versions *whole* so `0.9.0-rc1` could not satisfy
`0.9.0` — and two separate defects were found in review because of that
complexity. Now `0.9.0.md` and `0.9.0-rc1.md` are simply different files
and none of it is representable.
It became CI's job because remembering did not work. The sentence this It became CI's job because remembering did not work. The sentence this
paragraph replaces described a step **no release had ever performed** paragraph replaces described a step **no release had ever performed**
`drill/RUNS.md` carried no `## Release drill` section at all — and #95, there was no drill record anywhere in the repo — and #95, #114 and #148
#114 and #148 all shipped through the gap as a `VERSION` bump plus a all shipped through the gap as a `VERSION` bump plus a `CHANGELOG.md`
`CHANGELOG.md` stamp. The one time it was caught was the one time somebody stamp. The one time it was caught was the one time somebody happened to
happened to look, which is not a gate. look, which is not a gate.
**The drill is ONE orchestrated run over the whole stack**, not three **The three drills are INDEPENDENT — there is no fixed order.** box, rig
drills in a queue. box and rig are **mutually recursive**, so there is no and cast each drill separately, in any order, on any schedule, in separate
linear order to put them in: rig sits *below* box as the host-builder sittings if you like. What makes that safe is not sequencing, it is that
every drill **pins the same fixed set of candidate refs**: rig's drill
runs `--host yes` with `BOX_REF=release/<box-version>`, so it exercises the
box that will actually ship; box's drill mints with
`RIG_REF=release/<rig-version>`, so it exercises the rig that will actually
ship. Both measure the same pair.
That is what dissolves the box↔rig recursion. box and rig really are
**mutually recursive** — rig sits *below* box as the host-builder
(`rig bootstrap --host yes` installs box and runs `setup-host`) and (`rig bootstrap --host yes` installs box and runs `setup-host`) and
*above* it as the guest-converger (a `box new` seed's cloud-init curls *above* it as the guest-converger (a `box new` seed's cloud-init curls
rig's installer and runs `rig bootstrap <tenant>-box`). box's own source rig's installer and runs `rig bootstrap <tenant>-box`); box's own source
says as much — the seeds "invert the rig→box install edge (rig#28: rig says as much, the seeds "invert the rig→box install edge (rig#28: rig
installs box on hosts; now box guests install rig)". The run goes: installs box on hosts; now box guests install rig)". A cycle at *runtime*
becomes two independent tests against one fixed pair, because the refs are
**static identifiers that exist as soon as the release branches do**, long
before any drill runs. Nothing has to be released, or drilled, first.
1. `rig bootstrap --host yes` on a bare Debian host — installs box, runs Within a single drill you of course bring the substrate up before probing
`setup-host` it — a host before a guest. That is how you run a drill; it is not an
2. `box new` mints a creds-free seed ordering rule between repos.
3. the seed converges itself via `rig bootstrap <tenant>-box`
4. cast on top Each repo drills in a **different way** and asserts a different thing: box
asserts the **isolation contract** (the 85-probe VM trust boundary), rig
asserts **convergence** (a machine reaches its role, idempotently), cast
asserts **promotion** (A→B reproduces, the diff is idempotent). Three
different exercises sharing a substrate — not three phases of one script,
which is exactly why the records are per-repo.
It drills **candidate refs, not released artifacts.** `RIG_REPO` and It drills **candidate refs, not released artifacts.** `RIG_REPO` and
`RIG_REF` are mint-time environment variables (defaults `RIG_REF` are mint-time environment variables (defaults
@ -206,11 +237,15 @@ A release is a PR, and merging it ships it
`CHANGELOG.md` and nothing else, so no executable byte differs between `CHANGELOG.md` and nothing else, so no executable byte differs between
the tree that was drilled and the tree that ships. the tree that was drilled and the tree that ships.
One run emits **one shared run ID**. Each repo records its own legs under The drills of a release set share **one run ID**. Each repo records its own
its own `## Release drill — X.Y.Z — DATE`, citing that run ID and the legs in its own `drills/X.Y.Z.md`, citing that run ID and the other two
other two repos' commit SHAs, so the three records reconcile into a single repos' commit SHAs, so the three records reconcile afterwards. The guard
run afterwards. The guard reads only this repo's file — it asserts box's reads only this repo's file — it asserts box's record exists, never the
record exists, never the other two. other two.
If a defect shows up only in the combination: **patch, re-drill,
re-record.** The three releases converge on a set that holds together;
they are not required to be right in one pass.
**A known gap, and box is where it belongs.** A *released* box still **A known gap, and box is where it belongs.** A *released* box still
defaults `RIG_REF` to `main`, so what a user mints a week after a drill is defaults `RIG_REF` to `main`, so what a user mints a week after a drill is
@ -224,10 +259,10 @@ A release is a PR, and merging it ships it
A maintainer **waiver** is possible, and it is still written down. The A maintainer **waiver** is possible, and it is still written down. The
guard requires a *record*, not a passing result, so a release that must guard requires a *record*, not a passing result, so a release that must
ship without a full drill puts the section under the same heading and says ship without a full drill creates `drills/X.Y.Z.md` anyway and says plainly
plainly that the drill was waived and why. Skipping then costs a that the drill was waived and why. Skipping then costs a deliberate,
deliberate, reviewable line in the diff — which is precisely what the reviewable file in the diff — which is precisely what the three silent
three silent skips above did not. skips above did not.
2. **The maintainer's merge IS the release.** 2. **The maintainer's merge IS the release.**
[release.yml](.github/workflows/release.yml) fires on the merged, [release.yml](.github/workflows/release.yml) fires on the merged,
`release`-labeled PR and asserts before creating anything: `VERSION` at `release`-labeled PR and asserts before creating anything: `VERSION` at

View file

@ -36,7 +36,7 @@ PR carries as many as apply.
| `blocker:conflict` | `#B60205` | GitHub says `CONFLICTING` — the agent owes a **rebase** | it merges cleanly | | `blocker:conflict` | `#B60205` | GitHub says `CONFLICTING` — the agent owes a **rebase** | it merges cleanly |
| `blocker:ci-red` | `#B60205` | a check failed — the agent owes a **fix**, which a rebase will not provide | checks are green | | `blocker:ci-red` | `#B60205` | a check failed — the agent owes a **fix**, which a rebase will not provide | checks are green |
| `blocker:unrequested` | `#E99695` | this head has no verdict from somebody — never reviewed, or staled by a push — and **nobody was asked** for one | reviews are requested | | `blocker:unrequested` | `#E99695` | this head has no verdict from somebody — never reviewed, or staled by a push — and **nobody was asked** for one | reviews are requested |
| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record in [drill/RUNS.md](drill/RUNS.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) under `## Release drill — X.Y.Z` | | `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record at [`drills/X.Y.Z.md`](drills/README.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) at `drills/X.Y.Z.md` |
`blocker:drill-pending` is the one blocker that says the diff is *fine*. The `blocker:drill-pending` is the one blocker that says the diff is *fine*. The
version is stamped, the changelog is right, CI's other guards are green — what version is stamped, the changelog is right, CI's other guards are green — what

82
drills/README.md Normal file
View file

@ -0,0 +1,82 @@
# drills/ — release evidence, one file per version
This directory holds the **evidence that a release was proven on real
hardware**. One file per shipped version, named exactly for the version:
```
drills/0.9.0.md
drills/0.9.0-rc1.md
```
The name must match the contents of `VERSION` exactly.
[.github/scripts/drill-recorded.sh](../.github/scripts/drill-recorded.sh)
refuses any tree with a bare `VERSION` that has no such file, or whose file is
blank. A `-dev` tree passes with nothing to assert.
Because each version owns a file, `0.9.0` and `0.9.0-rc1` can never be
confused for one another — they are simply different paths. That used to take
careful whole-version field matching inside one shared file; now it is free.
## This is not `drill/RUNS.md`
Two different artifacts, and the distinction is load-bearing:
| | what it is |
|---|---|
| [`drill/RUNS.md`](../drill/RUNS.md) | the **harness's own history** — every run of `drill/drill.sh`, the traps table, the lore about what broke and why. It is not release-scoped and it is not going anywhere. |
| `drills/<version>.md` | **release evidence** — the record that *this version* was drilled before it shipped. Release-scoped, one file, gated by CI. |
Appending to `drill/RUNS.md` does not satisfy the release gate, and is not
meant to. Keep using it for what it has always been for.
## What a record should contain
- **What ran** — which drill, how many probes, `drill/drill.sh` invocation.
- **On what host** — the machine, the OS, the Incus version. "Real hardware"
is the claim; name the hardware.
- **The pinned candidate refs** — the exact `BOX_REF` / `RIG_REF` /
`CAST_REF` under test, and the other repos' commit SHAs. A drill that does
not say what it drilled proves nothing later.
- **The shared run ID**, so this record reconciles with the sibling repos'.
- **The numbers** — passed, failed, how long it took.
- **What failed**, plainly.
**A failed drill is still a valid record.** The gate wants *evidence*, not
success. A record saying "83/85, criterion (m) regressed, here is the issue"
is a good record. So is a maintainer's written waiver explaining why this
release shipped without a full drill. What the gate refuses is silence — #95,
#114 and #148 all shipped unproven because a skip left no trace.
## Worked example
The version below is a **placeholder that can never be a real release**.
Copy the shape, not the number.
```markdown
# Release drill — 9.9.9
- **Run ID:** `drill-9.9.9-20260721-01` (shared with rig, cast)
- **Host:** bare Debian 13, Ryzen 7 5800X / 64 GB, Incus 6.0.2
- **Date:** 2026-07-21
- **Candidate refs:**
- box `release/9.9.9` @ `abc1234`
- rig `release/4.4.4` @ `def5678` (minted with `RIG_REF=release/4.4.4`)
- cast `release/2.2.2` @ `9abcdef`
## What ran
`bash drill/drill.sh --ref release/9.9.9` — the full end-to-end: install the
stack, mint every template cold, snapshot and restore, uninstall to zero
residue. Then `drill/multiuser.sh` for the two-user grant matrix.
## Result
**84/85 passed, 1 failed.** 41 minutes wall clock.
- Failed: `multiuser.sh` criterion (m) — the raw instance kept a stale route
after teardown. Filed as #999. Judged not release-blocking: it affects
teardown residue on a host that is about to be wiped, not the trust
boundary itself.
- The VM boundary probes (the 85-probe isolation contract) passed clean,
which is the assertion this repo's drill exists to make.
```

View file

@ -279,7 +279,7 @@ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \
grep -qF 'changelog-armed.sh' "$ROOT/CONTRIBUTING.md" grep -qF 'changelog-armed.sh' "$ROOT/CONTRIBUTING.md"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# drill-recorded.sh — a RELEASE tree has a drill record in drill/RUNS.md. # drill-recorded.sh — a RELEASE tree has a drill record at drills/<ver>.md.
# #
# The gap this closes is not a drift or a race; it is a rule that was only # The gap this closes is not a drift or a race; it is a rule that was only
# ever a sentence. CONTRIBUTING.md has said since #96 that the release PR is # ever a sentence. CONTRIBUTING.md has said since #96 that the release PR is
@ -292,108 +292,169 @@ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \
# that fired on every tree would be red on every ordinary PR in the repo and # that fired on every tree would be red on every ordinary PR in the repo and
# would be switched off inside a day. # would be switched off inside a day.
# #
# Every fixture carries its OWN VERSION and its OWN RUNS.md. Reaching for # Every fixture carries its OWN VERSION and its OWN drills/ dir. Reaching for
# $ROOT/VERSION instead is exactly the coupling #146 had to fix: the suite # $ROOT/VERSION instead is exactly the coupling #146 had to fix: the suite
# then passes or fails on whichever state the repo happens to be in, so it # then passes or fails on whichever state the repo happens to be in, so it
# goes red on the ceremony tree — the one tree where the release suite most # goes red on the ceremony tree — the one tree where the release suite most
# needs to be trustworthy. # needs to be trustworthy.
#
# Records are now ONE FILE PER VERSION, which is why this block is shorter
# than the one it replaces. The heading-parsing cases are gone because there
# is no heading to parse: em-dash fields, the optional ' — DATE' tail, and
# "an empty section must not borrow a neighbour's prose" were all artifacts
# of records sharing drill/RUNS.md. Deleting a test is only safe when the
# failure it described became unrepresentable, and that is the case here —
# except for the whitespace rule, which was never about headings and is
# pinned harder below.
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
DRILLED="$ROOT/.github/scripts/drill-recorded.sh" DRILLED="$ROOT/.github/scripts/drill-recorded.sh"
check "drill-recorded: runnable bash" 0 "" bash -n "$DRILLED" check "drill-recorded: runnable bash" 0 "" bash -n "$DRILLED"
check "drill-recorded: executable" 0 "" test -x "$DRILLED" check "drill-recorded: executable" 0 "" test -x "$DRILLED"
# dtree <dir> <version> <runs-body...> — a two-file tree, self-contained # dtree <dir> <version> [<record-version> <record-body...>] — a self-contained
# tree: its own VERSION, and its own drills/ dir created only when asked for.
# With 2 args there is NO drills/ dir at all, which is a case in its own right.
dtree() { dtree() {
local d="$WORK/$1" v="$2"; shift 2 local d="$WORK/$1" v="$2"; shift 2
mkdir -p "$d" mkdir -p "$d"
printf '%s\n' "$v" > "$d/VERSION" printf '%s\n' "$v" > "$d/VERSION"
{ echo "# Drill run log"; echo; printf '%s\n' "$@"; } > "$d/RUNS.md" if [ "$#" -gt 0 ]; then
local rec="$1"; shift
mkdir -p "$d/drills"
printf '%s\n' "$@" > "$d/drills/$rec.md"
fi
echo "$d" echo "$d"
} }
drilled() { bash "$DRILLED" "$1/RUNS.md" "$1/VERSION"; } drilled() { bash "$DRILLED" "$1/drills" "$1/VERSION"; }
# --- the -dev steady state: nothing to assert ------------------------------ # --- the -dev steady state: nothing to assert ------------------------------
# No record at all, and that must PASS. This is the case that makes the guard # No record at all, and that must PASS. This is the case that makes the guard
# survivable on an ordinary PR. # survivable on an ordinary PR.
T="$(dtree drill-dev 0.8.1-dev 'No drill sections in here at all.')" T="$(dtree drill-dev 0.8.1-dev)"
check "drill-recorded: a -dev tree with NO record at all passes" 0 "nothing to assert" drilled "$T" check "drill-recorded: a -dev tree with NO drills dir at all passes" 0 "nothing to assert" drilled "$T"
check "drill-recorded: ...and says why it declined to judge" 0 "development tree" drilled "$T" check "drill-recorded: ...and says why it declined to judge" 0 "development tree" drilled "$T"
# --- the ceremony tree: a record is required and must carry prose ---------- # --- the ceremony tree: a record is required and must not be blank ---------
T="$(dtree drill-ok 0.9.0 \ T="$(dtree drill-ok 0.9.0 0.9.0 '# Release drill — 0.9.0' '' '- 47/47 on real hardware')"
'## Release drill — 0.9.0 — 2026-07-21' '' '- 47/47 on real hardware' '' \ check "drill-recorded: a bare VERSION with a non-empty record for it passes" \
'## Run history' '' '- older prose')"
check "drill-recorded: a bare VERSION with a matching non-empty record passes" \
0 "has a drill record" drilled "$T" 0 "has a drill record" drilled "$T"
# The date tail is optional; the heading without it is still a record.
T="$(dtree drill-nodate 0.9.0 '## Release drill — 0.9.0' '' '- ran it, 47/47')"
check "drill-recorded: ...and the ' — DATE' tail is optional" 0 "has a drill record" drilled "$T"
T="$(dtree drill-none 0.9.0 '## Run history' '' '- prose about earlier runs')" # No drills/ dir AT ALL is the commonest way to fail this: the release branch
check "drill-recorded: a bare VERSION with NO record fails" 1 "no drill record" drilled "$T" # was cut and nobody ran anything. It must fail loudly, not error out on a
# missing directory in some way that reads like a broken guard.
T="$(dtree drill-nodir 0.9.0)"
check "drill-recorded: a bare VERSION with NO drills dir fails" 1 "no drill record" drilled "$T"
check "drill-recorded: ...and the failure names the version" 1 "VERSION is '0.9.0'" drilled "$T" check "drill-recorded: ...and the failure names the version" 1 "VERSION is '0.9.0'" drilled "$T"
# The dir exists — some other release left its record here — but this version
# has none. A directory listing is not evidence about a particular version.
T="$(dtree drill-otherver 0.9.0 0.8.0 '# Release drill — 0.8.0' '' '- the previous release')"
check "drill-recorded: a drills dir with no file for THIS version fails" \
1 "no drill record" drilled "$T"
check "drill-recorded: ...and names the unblock — run the drill and record it" \ check "drill-recorded: ...and names the unblock — run the drill and record it" \
1 "RUN THE DRILL" drilled "$T" 1 "RUN THE DRILL" drilled "$T"
check "drill-recorded: ...and names the path it wanted" 1 "drills/0.9.0.md" drilled "$T"
check "drill-recorded: ...and offers the recorded maintainer waiver as the other way out" \ check "drill-recorded: ...and offers the recorded maintainer waiver as the other way out" \
1 "WAIVED" drilled "$T" 1 "WAIVED" drilled "$T"
check "drill-recorded: ...and names the three releases that shipped through the gap" \
1 "#95, #114 and #148" drilled "$T"
# A heading with nothing under it is the same failure as no heading: the # A file that exists and says nothing is the same failure as no file: the
# ceremony wrote the shape and never wrote the evidence. # ceremony created the shape and never wrote the evidence. `touch` must not
T="$(dtree drill-empty 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' '' '' '## Run history' '' '- older prose')" # be able to satisfy a gate whose entire job is to demand evidence.
check "drill-recorded: a bare VERSION whose record is present but EMPTY fails" \ mkdir -p "$WORK/drill-emptyfile/drills"
1 "no drill record" drilled "$T" printf '0.9.0\n' > "$WORK/drill-emptyfile/VERSION"
: > "$WORK/drill-emptyfile/drills/0.9.0.md"
check "drill-recorded: a record file that is present but EMPTY fails" \
1 "no drill record" drilled "$WORK/drill-emptyfile"
# ...and WHITESPACE is not evidence either. The first cut extracted with # ...and WHITESPACE is not evidence either. This is the ONE rule carried over
# `sed '/./,$!d'`, where `.` matches a space — so a heading followed by one # from the heading-parsing guard, because it was never a heading problem. The
# tab passed the guard while the failure text promised "at least one non-blank # first cut extracted with `sed '/./,$!d'`, where `.` matches a space — so a
# line". An evidence-free release for the price of an invisible character, on # record whose body was one tab passed a guard that promised "at least one
# the one check whose entire job is to demand evidence. All three reviewers on # non-blank line". An evidence-free release for the price of an invisible
# #149 found it independently, which is the level of scrutiny it deserved and # character. All three reviewers on #149 found it independently, which is the
# not a level it should ever need again. # level of scrutiny it deserved and not a level it should ever need again.
T="$(dtree drill-blank 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' ' ' ' ' '' '## Run history' '' '- older prose')" # Splitting the file per version removed the parsing; it did not remove this.
check "drill-recorded: a record body of only spaces and tabs fails (#149)" \ mkdir -p "$WORK/drill-blank/drills"
1 "no drill record" drilled "$T" printf '0.9.0\n' > "$WORK/drill-blank/VERSION"
printf ' \n\t\n\n' > "$WORK/drill-blank/drills/0.9.0.md"
check "drill-recorded: a record of only spaces, tabs and newlines fails (#149)" \
1 "no drill record" drilled "$WORK/drill-blank"
# --- the version is matched WHOLE, both directions ------------------------- # --- the version is matched WHOLE, both directions -------------------------
# release-notes.sh's trap, in a new file: a substring match would let an rc # Under the old scheme this took deliberate field-matching against a heading,
# drill stand in for the release it was a candidate for, which is the one # and getting it wrong would have let a release-candidate drill stand in for
# thing a release-candidate drill by definition did not measure. # the release it was a candidate for — the one thing an rc drill by
T="$(dtree drill-rc-only 0.9.0 '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill')" # definition did not measure. One file per version makes it free: these are
check "drill-recorded: a 0.9.0-rc1 record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T" # simply different paths. Pinned anyway, in both directions, so that a future
T="$(dtree drill-rel-only 0.9.0-rc1 '## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')" # rewrite reaching for a prefix or glob match cannot quietly reintroduce it.
check "drill-recorded: ...and a 0.9.0 record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T" T="$(dtree drill-rc-only 0.9.0 0.9.0-rc1 '# Release drill — 0.9.0-rc1' '' '- the rc drill')"
# ...and with both present, each resolves to its own section and neither check "drill-recorded: a 0.9.0-rc1.md record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T"
# borrows the other's prose. T="$(dtree drill-rel-only 0.9.0-rc1 0.9.0 '# Release drill — 0.9.0' '' '- the release drill')"
T="$(dtree drill-both 0.9.0 \ check "drill-recorded: ...and a 0.9.0.md record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T"
'## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \
'## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')"
check "drill-recorded: with both present, the release resolves to its own" \
0 "has a drill record" drilled "$T"
# The sharp version of the same claim: the rc section is full, the release's
# own section is EMPTY. If emptiness were judged over anything but this
# version's own lines, this tree would pass on the rc's prose.
T="$(dtree drill-borrow 0.9.0 \
'## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \
'## Release drill — 0.9.0 — 2026-07-21' '')"
check "drill-recorded: ...and an empty release section cannot borrow the rc's prose" \
1 "no drill record" drilled "$T"
# --- degenerate trees refuse rather than pass by accident ------------------ # --- degenerate trees refuse rather than pass by accident ------------------
# A guard that cannot read the version cannot know whether this tree is its
# business, and "could not tell" must never resolve to "allowed".
check "drill-recorded: a missing VERSION refuses by path" 1 "no such file" \ check "drill-recorded: a missing VERSION refuses by path" 1 "no such file" \
bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/nope-version" bash "$DRILLED" "$WORK/drill-ok/drills" "$WORK/nope-version"
mkdir -p "$WORK/drill-empty-ver"; : > "$WORK/drill-empty-ver/VERSION" mkdir -p "$WORK/drill-empty-ver"; : > "$WORK/drill-empty-ver/VERSION"
check "drill-recorded: an empty VERSION refuses" 1 "is empty" \ check "drill-recorded: an empty VERSION refuses" 1 "is empty" \
bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/drill-empty-ver/VERSION" bash "$DRILLED" "$WORK/drill-ok/drills" "$WORK/drill-empty-ver/VERSION"
# The runs file is only needed once the tree is a release — a -dev tree must # The drills dir is only consulted once the tree is a release — a -dev tree
# not be refused for a file it has no business reading. # must not be refused for a directory it has no business reading.
check "drill-recorded: a release tree with a missing runs file refuses by path" \ check "drill-recorded: a -dev tree does not even look for the drills dir" 0 "nothing to assert" \
1 "no such file" bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-ok/VERSION" bash "$DRILLED" "$WORK/nope-drills" "$WORK/drill-dev/VERSION"
check "drill-recorded: ...but a -dev tree does not even look for it" 0 "nothing to assert" \
bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-dev/VERSION"
# --- and the tree under test ---------------------------------------------- # --- and the tree under test ----------------------------------------------
check "drill-recorded: THIS tree passes its own guard" 0 "" \ # The property is that the guard's VERDICT IS CORRECT FOR THIS TREE — not that
bash "$DRILLED" "$ROOT/drill/RUNS.md" "$ROOT/VERSION" # it always passes. Those come apart on a ceremony tree, and demanding success
# is wrong there in a way that took a red release PR to notice.
#
# A -dev tree is vacuous, so it must pass. A ceremony tree passes only once a
# human has run the drill and written the record — which is the entire point of
# the gate. Asserting exit 0 unconditionally therefore made test/release.sh
# UN-GREENABLE on every release branch before its drill, and reported it as a
# `release-flow tests` failure rather than as the gate doing its job: the same
# noise-and-misattribution shape as #146, where a fixture read the repo's real
# VERSION and only misbehaved on the ceremony tree.
#
# So: assert the verdict that this tree's own state entails. Green in all three
# states — dev, ceremony-without-record, ceremony-with-record — and still red if
# the guard ever disagrees with the tree in front of it.
THIS_VER="$(tr -d '[:space:]' < "$ROOT/VERSION")"
case "$THIS_VER" in
*-dev)
check "drill-recorded: THIS tree is -dev, and the guard is vacuous on it" 0 "nothing to assert" \
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION" ;;
*)
if [ -s "$ROOT/drills/$THIS_VER.md" ]; then
check "drill-recorded: THIS ceremony tree HAS its record, and the guard accepts it" 0 "" \
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION"
else
check "drill-recorded: THIS ceremony tree has NO record yet, and the guard refuses it" 1 "no drill record" \
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION"
fi ;;
esac
# drills/ is release evidence and drill/RUNS.md is the harness's own log. The
# rewrite is only coherent if both keep existing and the docs say which is
# which; a reader who appends to the wrong one gets a red gate and no clue.
check "drills/: the directory documents itself" 0 "" test -f "$ROOT/drills/README.md"
check "drills/: ...and is plain, not a dot-directory invisible to globs (#116)" 1 "" \
test -d "$ROOT/.drills"
check "drills/: ...and distinguishes itself from the harness log" 0 "" \
grep -qF 'drill/RUNS.md' "$ROOT/drills/README.md"
check "drills/: ...and its worked example uses a version that can never ship" 0 "" \
grep -qF '9.9.9' "$ROOT/drills/README.md"
check "drill/RUNS.md: the harness log is untouched and still present" 0 "" \
test -f "$ROOT/drill/RUNS.md"
# No real record may be invented to make the suite green. The repo is
# 0.8.1-dev; fabricating drills/0.8.1.md would defeat the whole gate.
check "drills/: no fabricated record for an unshipped version" 1 "" \
test -f "$ROOT/drills/0.8.1.md"
# The guard is only a guard if CI runs it, and only a rule if the document # The guard is only a guard if CI runs it, and only a rule if the document
# that used to carry the rule now points at it. # that used to carry the rule now points at it.
@ -401,21 +462,38 @@ check "ci.yml: runs the drill-recorded guard" 0 "" \
grep -qF 'drill-recorded.sh' "$ROOT/.github/workflows/ci.yml" grep -qF 'drill-recorded.sh' "$ROOT/.github/workflows/ci.yml"
check "CONTRIBUTING: names the guard that enforces the drill" 0 "" \ check "CONTRIBUTING: names the guard that enforces the drill" 0 "" \
grep -qF 'drill-recorded.sh' "$ROOT/CONTRIBUTING.md" grep -qF 'drill-recorded.sh' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: documents the drill record's heading format" 0 "" \ check "CONTRIBUTING: documents the drill record's path" 0 "" \
grep -qF '## Release drill — X.Y.Z — DATE' "$ROOT/CONTRIBUTING.md" grep -qF 'drills/X.Y.Z.md' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: the drill is ONE run over the whole stack, not a queue" 0 "" \ check "CONTRIBUTING: ...and distinguishes it from the harness's own log" 0 "" \
grep -qF "harness's own log" "$ROOT/CONTRIBUTING.md"
# The three drills are INDEPENDENT. An earlier draft of this section said the
# drill was "ONE orchestrated run over the whole stack", which over-constrains
# it into a fixed sequence the repos cannot satisfy. Pinning the corrected
# claim, and negatively pinning the fixed-order language, because "surely you
# drill rig before box" is exactly the simplification a future editor makes.
check "CONTRIBUTING: the three drills are independent, in any order" 0 "" \
grep -qF 'The three drills are INDEPENDENT' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and no fixed-order framing survives" 1 "" \
grep -qF 'ONE orchestrated run' "$ROOT/CONTRIBUTING.md" grep -qF 'ONE orchestrated run' "$ROOT/CONTRIBUTING.md"
# The load-bearing correction: box and rig are mutually recursive, so a # The load-bearing correction: box and rig are mutually recursive, so a
# "release A before you can drill B" rule is not merely bureaucratic, it is # "release A before you can drill B" rule is not merely bureaucratic, it is
# unsatisfiable in principle. Pinning the mechanism that dissolves it — # unsatisfiable in principle. What dissolves it is that every drill pins the
# mint-time candidate refs — because that is the claim a future editor is # same fixed candidate refs — static identifiers that exist as soon as the
# most likely to "simplify" back into a linear order. # release branches do — turning a runtime cycle into two independent tests.
check "CONTRIBUTING: ...drilling candidate refs, not released artifacts" 0 "" \ check "CONTRIBUTING: ...drilling candidate refs, not released artifacts" 0 "" \
grep -qF 'candidate refs, not released artifacts' "$ROOT/CONTRIBUTING.md" grep -qF 'candidate refs, not released artifacts' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and the same fixed refs are what dissolve the recursion" 0 "" \
grep -qF 'pins the same fixed set of candidate refs' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and each repo asserts a different thing" 0 "" \
grep -qF 'isolation contract' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and a combination defect means patch, re-drill, re-record" 0 "" \
grep -qF 're-drill' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and the RIG_REF gap is stated as outstanding (#81)" 0 "" \ check "CONTRIBUTING: ...and the RIG_REF gap is stated as outstanding (#81)" 0 "" \
grep -qF 'RIG_REF' "$ROOT/CONTRIBUTING.md" grep -qF 'RIG_REF' "$ROOT/CONTRIBUTING.md"
check "LABELS: documents blocker:drill-pending" 0 "" \ check "LABELS: documents blocker:drill-pending" 0 "" \
grep -qF 'blocker:drill-pending' "$ROOT/LABELS.md" grep -qF 'blocker:drill-pending' "$ROOT/LABELS.md"
check "LABELS: ...and points at the per-version record, not the harness log" 0 "" \
grep -qF 'drills/X.Y.Z.md' "$ROOT/LABELS.md"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# changelog-monotonic.sh (#122) — no SHIPPED release heading may be DELETED. # changelog-monotonic.sh (#122) — no SHIPPED release heading may be DELETED.