heavy-duty/box
7
0
Fork 1
Code Issues 9 Pull requests Projects Releases 4 Packages Wiki Activity Actions

Merge pull request #151 from dan-claude-bot/refactor/drills-per-version

Browse source 
refactor: one drill record per version, in drills/
This commit is contained in:
Daniel Marin 2026-07-21 18:41:42 +01:00 committed by GitHub
commit 0b5bedd6e0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 364 additions and 172 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

141
.github/scripts/drill-recorded.sh vendored
View file

@ -1,35 +1,53 @@
#!/usr/bin/env bash
set -euo pipefail
# drill-recorded.sh [<runs-file>] [<version-file>] — assert that a RELEASE
# drill-recorded.sh [<drills-dir>] [<version-file>] — assert that a RELEASE
# tree carries a drill record: that the full real-hardware drill this repo
# says a release rests on was actually run for this version, and written
# down in drill/RUNS.md.
# down as drills/<version>.md.
#
# CONTRIBUTING.md has said since #96 that "this PR is where the release
# ritual hangs: the full drill on real hardware, recorded in drill/RUNS.md".
# No release has ever done it. #95, #114 and #148 all shipped as a VERSION
# bump plus a CHANGELOG.md stamp and nothing else, and the file they were
# supposed to append to has no '## Release drill' section anywhere in it.
# That is three releases through the same gap, because the gate was a
# sentence in a document and the only thing standing on it was a reviewer
# remembering to ask. A reviewer finally did — which is the point: the ONE
# time it was caught is the time somebody happened to look, and that is not a
# gate, it is luck with good manners.
# ritual hangs: the full drill on real hardware, recorded". No release ever
# did it. #95, #114 and #148 all shipped as a VERSION bump plus a
# CHANGELOG.md stamp and nothing else. That is three releases through the
# same gap, because the gate was a sentence in a document and the only thing
# standing on it was a reviewer remembering to ask. A reviewer finally did —
# which is the point: the ONE time it was caught is the time somebody
# happened to look, and that is not a gate, it is luck with good manners.
#
# So the rule moves into CI, where it fires on every release PR whether or
# not anyone is paying attention. The rule, keyed on VERSION for the same
# reason changelog-armed.sh is — the two states are genuinely different:
# not anyone is paying attention. It is keyed on VERSION for the same reason
# changelog-armed.sh is — the two states are genuinely different:
#
# VERSION ends in -dev -> PASS. A development tree ships nothing, so
# there is nothing for it to have proven. Almost
# every PR in this repo is this case, and a guard
# that nagged all of them would be turned off.
# VERSION is bare -> the ceremony tree, the one about to ship.
# drill/RUNS.md MUST carry a section headed
# '## Release drill — <version>' (optionally with
# ' — <date>' after it), and that section must
# have prose in it.
# <drills-dir>/<version>.md MUST exist and carry
# at least one non-whitespace character.
#
# ONE FILE PER VERSION, and that is the whole design. Records used to be
# sections sharing drill/RUNS.md, and every hard edge this script used to
# have existed only because of that sharing: em-dash field matching, an
# optional ' — DATE' tail, whole-version comparison so '0.9.0-rc1' could not
# satisfy '0.9.0', avoiding '\x' escapes because CI runs mawk not gawk, and a
# non-blank body rule to tell an empty section from a filled one. Two
# separate defects were found in review because of that complexity — a
# `sed '/./,$!d'` whitespace bypass, and heading-grammar drift from the
# sibling repos. Splitting the file makes almost all of it UNREPRESENTABLE:
# '0.9.0.md' and '0.9.0-rc1.md' are simply different files, so whole-version
# matching is free rather than a trap, and there is no grammar left to drift.
#
# The directory is plain 'drills', NOT '.drills'. A dot-directory is
# invisible to a glob without dotglob, which is exactly what caused #116 and
# #118 in this repo; evidence a sweep cannot see is evidence that goes
# missing quietly.
#
# drills/ is RELEASE EVIDENCE, one file per shipped version. It is NOT
# drill/RUNS.md, which stays exactly as it is: the harness's own run log,
# traps table and lore, a different artifact with a different purpose. This
# guard reads drills/ and never looks at drill/RUNS.md.
#
# What this guard asserts is a RECORD, deliberately — not a passing drill.
# CI cannot run the drill: it wants real hardware, a real Incus, and the
@ -37,16 +55,19 @@ set -euo pipefail
# rehearsal job it runs instead). What CI can do is refuse to let a release
# claim a ritual it left no evidence of. That also leaves the maintainer
# waiver intact and honest: a release that must ship without a full drill
# records WHY under its own heading, which is a deliberate, reviewable commit
# in the diff — rather than the silent skip that got us here.
# records WHY in its own file, which is a deliberate, reviewable commit in
# the diff — rather than the silent skip that got us here.
#
# A file of its own (not inlined in ci.yml) so test/release.sh can drive it
# against constructed trees for both states — the same discipline as
# changelog-armed.sh and release-notes.sh.
runs="${1:-drill/RUNS.md}"
drills="${1:-drills}"
version_file="${2:-VERSION}"
# A missing or empty version file is an ERROR, never a silent pass. A guard
# that cannot read the version cannot know whether this tree is its business,
# and "could not tell" must not resolve to "allowed".
[ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; }
ver="$(tr -d '[:space:]' < "$version_file")"
@ -62,67 +83,43 @@ case "$ver" in
;;
esac
[ -f "$runs" ] || { echo "drill-recorded: no such file: $runs" >&2; exit 1; }
record="$drills/$ver.md"
# The section for this version: everything between its own heading and the
# next '## '. The version is compared WHOLE — as a field, never as a
# substring or a regex — so '0.9.0' can never be satisfied by a
# '0.9.0-rc1' section (or vice versa), and there are no dots to escape.
# release-notes.sh solves the identical trap the identical way; the two
# scripts must not disagree about what "the section for X" means.
#
# The heading shape is '## Release drill — <ver>' with an OPTIONAL
# ' — <date>' tail, i.e. fields: '##' 'Release' 'drill' '—' '<ver>' ['—' ...].
# Pinning the leading fields as well as the version keeps some other '## '
# heading that merely mentions the number from counting as a record.
#
# The em dash is passed IN as a variable rather than written into the awk
# program, because '\x' escapes in an awk string are a gawk extension and CI
# runs on ubuntu-latest, where awk is mawk.
#
# `grab && NF` is the non-blank rule, and it is load-bearing rather than
# tidiness. NF is 0 on a line that is empty OR contains only whitespace, so
# `record` ends up non-empty exactly when a line with actual content exists.
# The first cut of this piped through `sed '/./,$!d'` instead, which drops
# leading blank lines but keeps a line of spaces — `.` matches a space. A
# heading followed by nothing but a tab therefore satisfied the guard and
# shipped an evidence-free release, while the failure text below promised "at
# least one non-blank line". The implementation was looser than its own
# contract, which on a gate is the whole ballgame: the bypass costs one
# invisible character. Caught by all three reviewers on #149, independently.
record="$(awk -v ver="$ver" -v dash="—" '
/^## / {
grab = ($2 == "Release" && $3 == "drill" && $4 == dash && $5 == ver \
&& (NF == 5 || $6 == dash))
next
}
grab && NF { print }
' "$runs")"
if [ -z "$record" ]; then
# The one rule that survives the rewrite, and it survives because it was
# never really about heading parsing: a file of only spaces, tabs and
# newlines is NOT a record. The first cut of the old guard extracted with
# `sed '/./,$!d'`, where `.` matches a space — so a record whose body was one
# tab satisfied a guard that promised "at least one non-blank line". An
# evidence-free release for the price of an invisible character, on the one
# check whose entire job is to demand evidence. Existence alone is a weaker
# claim than `touch` can defeat, so existence alone is not the test.
if [ ! -f "$record" ] || ! grep -q '[^[:space:]]' "$record"; then
cat >&2 <<EOF
drill-recorded: VERSION is '$ver' — a release — and $runs has no drill record
for it. The heading this looks for is:
drill-recorded: VERSION is '$ver' — a release — and there is no drill record
for it. The file this looks for is:
## Release drill — $ver — DATE
$record
...with at least one non-blank line under it. Either the section is absent
entirely, or it is present and empty; both mean the same thing, which is
that this release is asserting a ritual it has left no evidence of.
...with something written in it. Either the file is absent entirely, or it
is present and blank; both mean the same thing, which is that this release
is asserting a ritual it has left no evidence of.
The unblock is to RUN THE DRILL (drill/drill.sh, on real hardware) and
record it in $runs under that heading — what it measured, what it found,
what it cost. CI cannot run the drill for you; it can only refuse a release
that never ran one.
record it at that path — what it measured, what it found, what it cost.
See $drills/README.md for what a record should contain. CI cannot run the
drill for you; it can only refuse a release that never ran one.
(drill/RUNS.md is a different artifact — the harness's own run log and
traps. Appending there does not satisfy this guard, and is not meant to.)
If this release must ship without a full drill, that is a maintainer's call
to make and it is still recorded: write the section under the same heading
and say plainly that the drill was WAIVED and why. The guard requires a
record, not a passing result — so a skip is a visible, reviewable line in
the diff rather than the silent gap that let #95, #114 and #148 all ship
unproven. See CONTRIBUTING.md, "Releases".
to make and it is still recorded: create the same file and say plainly that
the drill was WAIVED and why. The guard requires a record, not a passing
result — so a skip is a visible, reviewable file in the diff rather than
the silent gap that let #95, #114 and #148 all ship unproven. See
CONTRIBUTING.md, "Releases".
EOF
exit 1
fi
echo "drill-recorded: VERSION '$ver' has a drill record in $runs"
echo "drill-recorded: VERSION '$ver' has a drill record at $record"

2
CHANGELOG.md
View file

@ -17,7 +17,7 @@ which records not just what changed but what each drill run proved.
- Every fresh mint marks a `pristine` snapshot, before rig converges anything
(#104, heavy-duty/rig#62)
- A mint that converges a tenant role marks a `bootstrapped` snapshot (#130)
- CI refuses a release PR with no drill record in `drill/RUNS.md`
- CI refuses a release PR with no drill record at `drills/<version>.md`
### Changed

91
CONTRIBUTING.md
View file

@ -163,39 +163,70 @@ A release is a PR, and merging it ships it
real hardware and the better part of an hour, so it runs on the release
PR's branch and nowhere else.
Record it in [drill/RUNS.md](drill/RUNS.md) under its own heading:
Record it in its own file, named exactly for the version:
```markdown
## Release drill — X.Y.Z — DATE
```
drills/X.Y.Z.md
```
...with the run under it — what it measured, what it found, what it cost.
...with the run in it — what it measured, what it found, what it cost.
See [drills/README.md](drills/README.md) for the shape.
[.github/scripts/drill-recorded.sh](.github/scripts/drill-recorded.sh)
asserts exactly that on every tree with a bare `VERSION`, which is every
`release` PR and the merge that publishes it; a `-dev` tree passes with
nothing to assert. It is **no longer a thing a reviewer has to remember**.
**`drills/` is release evidence; [`drill/RUNS.md`](drill/RUNS.md) is the
harness's own log.** Two artifacts, two purposes. `drill/RUNS.md` keeps
being what it has always been — every run of the harness, the traps table,
the lore — and is not release-scoped. `drills/X.Y.Z.md` is the per-release
record the gate reads. Appending to `drill/RUNS.md` does not satisfy the
gate, and is not meant to.
One file per version is also why the guard is short. When records shared a
file it had to match em-dash heading fields, tolerate an optional date
tail, and compare versions *whole* so `0.9.0-rc1` could not satisfy
`0.9.0` — and two separate defects were found in review because of that
complexity. Now `0.9.0.md` and `0.9.0-rc1.md` are simply different files
and none of it is representable.
It became CI's job because remembering did not work. The sentence this
paragraph replaces described a step **no release had ever performed** —
`drill/RUNS.md` carried no `## Release drill` section at all — and #95,
#114 and #148 all shipped through the gap as a `VERSION` bump plus a
`CHANGELOG.md` stamp. The one time it was caught was the one time somebody
happened to look, which is not a gate.
there was no drill record anywhere in the repo — and #95, #114 and #148
all shipped through the gap as a `VERSION` bump plus a `CHANGELOG.md`
stamp. The one time it was caught was the one time somebody happened to
look, which is not a gate.
**The drill is ONE orchestrated run over the whole stack**, not three
drills in a queue. box and rig are **mutually recursive**, so there is no
linear order to put them in: rig sits *below* box as the host-builder
**The three drills are INDEPENDENT — there is no fixed order.** box, rig
and cast each drill separately, in any order, on any schedule, in separate
sittings if you like. What makes that safe is not sequencing, it is that
every drill **pins the same fixed set of candidate refs**: rig's drill
runs `--host yes` with `BOX_REF=release/<box-version>`, so it exercises the
box that will actually ship; box's drill mints with
`RIG_REF=release/<rig-version>`, so it exercises the rig that will actually
ship. Both measure the same pair.
That is what dissolves the box↔rig recursion. box and rig really are
**mutually recursive** — rig sits *below* box as the host-builder
(`rig bootstrap --host yes` installs box and runs `setup-host`) and
*above* it as the guest-converger (a `box new` seed's cloud-init curls
rig's installer and runs `rig bootstrap <tenant>-box`). box's own source
says as much — the seeds "invert the rig→box install edge (rig#28: rig
installs box on hosts; now box guests install rig)". The run goes:
rig's installer and runs `rig bootstrap <tenant>-box`); box's own source
says as much, the seeds "invert the rig→box install edge (rig#28: rig
installs box on hosts; now box guests install rig)". A cycle at *runtime*
becomes two independent tests against one fixed pair, because the refs are
**static identifiers that exist as soon as the release branches do**, long
before any drill runs. Nothing has to be released, or drilled, first.
1. `rig bootstrap --host yes` on a bare Debian host — installs box, runs
`setup-host`
2. `box new` mints a creds-free seed
3. the seed converges itself via `rig bootstrap <tenant>-box`
4. cast on top
Within a single drill you of course bring the substrate up before probing
it — a host before a guest. That is how you run a drill; it is not an
ordering rule between repos.
Each repo drills in a **different way** and asserts a different thing: box
asserts the **isolation contract** (the 85-probe VM trust boundary), rig
asserts **convergence** (a machine reaches its role, idempotently), cast
asserts **promotion** (A→B reproduces, the diff is idempotent). Three
different exercises sharing a substrate — not three phases of one script,
which is exactly why the records are per-repo.
It drills **candidate refs, not released artifacts.** `RIG_REPO` and
`RIG_REF` are mint-time environment variables (defaults
@ -206,11 +237,15 @@ A release is a PR, and merging it ships it
`CHANGELOG.md` and nothing else, so no executable byte differs between
the tree that was drilled and the tree that ships.
One run emits **one shared run ID**. Each repo records its own legs under
its own `## Release drill — X.Y.Z — DATE`, citing that run ID and the
other two repos' commit SHAs, so the three records reconcile into a single
run afterwards. The guard reads only this repo's file — it asserts box's
record exists, never the other two.
The drills of a release set share **one run ID**. Each repo records its own
legs in its own `drills/X.Y.Z.md`, citing that run ID and the other two
repos' commit SHAs, so the three records reconcile afterwards. The guard
reads only this repo's file — it asserts box's record exists, never the
other two.
If a defect shows up only in the combination: **patch, re-drill,
re-record.** The three releases converge on a set that holds together;
they are not required to be right in one pass.
**A known gap, and box is where it belongs.** A *released* box still
defaults `RIG_REF` to `main`, so what a user mints a week after a drill is
@ -224,10 +259,10 @@ A release is a PR, and merging it ships it
A maintainer **waiver** is possible, and it is still written down. The
guard requires a *record*, not a passing result, so a release that must
ship without a full drill puts the section under the same heading and says
plainly that the drill was waived and why. Skipping then costs a
deliberate, reviewable line in the diff — which is precisely what the
three silent skips above did not.
ship without a full drill creates `drills/X.Y.Z.md` anyway and says plainly
that the drill was waived and why. Skipping then costs a deliberate,
reviewable file in the diff — which is precisely what the three silent
skips above did not.
2. **The maintainer's merge IS the release.**
[release.yml](.github/workflows/release.yml) fires on the merged,
`release`-labeled PR and asserts before creating anything: `VERSION` at

2
LABELS.md
View file

@ -36,7 +36,7 @@ PR carries as many as apply.
| `blocker:conflict` | `#B60205` | GitHub says `CONFLICTING` — the agent owes a **rebase** | it merges cleanly |
| `blocker:ci-red` | `#B60205` | a check failed — the agent owes a **fix**, which a rebase will not provide | checks are green |
| `blocker:unrequested` | `#E99695` | this head has no verdict from somebody — never reviewed, or staled by a push — and **nobody was asked** for one | reviews are requested |
| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record in [drill/RUNS.md](drill/RUNS.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) under `## Release drill — X.Y.Z` |
| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record at [`drills/X.Y.Z.md`](drills/README.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) at `drills/X.Y.Z.md` |
`blocker:drill-pending` is the one blocker that says the diff is *fine*. The
version is stamped, the changelog is right, CI's other guards are green — what

82
drills/README.md Normal file
View file

@ -0,0 +1,82 @@
# drills/ — release evidence, one file per version
This directory holds the **evidence that a release was proven on real
hardware**. One file per shipped version, named exactly for the version:
```
drills/0.9.0.md
drills/0.9.0-rc1.md
```
The name must match the contents of `VERSION` exactly.
[.github/scripts/drill-recorded.sh](../.github/scripts/drill-recorded.sh)
refuses any tree with a bare `VERSION` that has no such file, or whose file is
blank. A `-dev` tree passes with nothing to assert.
Because each version owns a file, `0.9.0` and `0.9.0-rc1` can never be
confused for one another — they are simply different paths. That used to take
careful whole-version field matching inside one shared file; now it is free.
## This is not `drill/RUNS.md`
Two different artifacts, and the distinction is load-bearing:
| | what it is |
|---|---|
| [`drill/RUNS.md`](../drill/RUNS.md) | the **harness's own history** — every run of `drill/drill.sh`, the traps table, the lore about what broke and why. It is not release-scoped and it is not going anywhere. |
| `drills/<version>.md` | **release evidence** — the record that *this version* was drilled before it shipped. Release-scoped, one file, gated by CI. |
Appending to `drill/RUNS.md` does not satisfy the release gate, and is not
meant to. Keep using it for what it has always been for.
## What a record should contain
- **What ran** — which drill, how many probes, `drill/drill.sh` invocation.
- **On what host** — the machine, the OS, the Incus version. "Real hardware"
is the claim; name the hardware.
- **The pinned candidate refs** — the exact `BOX_REF` / `RIG_REF` /
`CAST_REF` under test, and the other repos' commit SHAs. A drill that does
not say what it drilled proves nothing later.
- **The shared run ID**, so this record reconciles with the sibling repos'.
- **The numbers** — passed, failed, how long it took.
- **What failed**, plainly.
**A failed drill is still a valid record.** The gate wants *evidence*, not
success. A record saying "83/85, criterion (m) regressed, here is the issue"
is a good record. So is a maintainer's written waiver explaining why this
release shipped without a full drill. What the gate refuses is silence — #95,
#114 and #148 all shipped unproven because a skip left no trace.
## Worked example
The version below is a **placeholder that can never be a real release**.
Copy the shape, not the number.
```markdown
# Release drill — 9.9.9
- **Run ID:** `drill-9.9.9-20260721-01` (shared with rig, cast)
- **Host:** bare Debian 13, Ryzen 7 5800X / 64 GB, Incus 6.0.2
- **Date:** 2026-07-21
- **Candidate refs:**
- box `release/9.9.9` @ `abc1234`
- rig `release/4.4.4` @ `def5678` (minted with `RIG_REF=release/4.4.4`)
- cast `release/2.2.2` @ `9abcdef`
## What ran
`bash drill/drill.sh --ref release/9.9.9` — the full end-to-end: install the
stack, mint every template cold, snapshot and restore, uninstall to zero
residue. Then `drill/multiuser.sh` for the two-user grant matrix.
## Result
**84/85 passed, 1 failed.** 41 minutes wall clock.
- Failed: `multiuser.sh` criterion (m) — the raw instance kept a stale route
after teardown. Filed as #999. Judged not release-blocking: it affects
teardown residue on a host that is about to be wiped, not the trust
boundary itself.
- The VM boundary probes (the 85-probe isolation contract) passed clean,
which is the assertion this repo's drill exists to make.
```

218
test/release.sh
View file

@ -279,7 +279,7 @@ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \
grep -qF 'changelog-armed.sh' "$ROOT/CONTRIBUTING.md"
# ---------------------------------------------------------------------------
# drill-recorded.sh — a RELEASE tree has a drill record in drill/RUNS.md.
# drill-recorded.sh — a RELEASE tree has a drill record at drills/<ver>.md.
#
# The gap this closes is not a drift or a race; it is a rule that was only
# ever a sentence. CONTRIBUTING.md has said since #96 that the release PR is
@ -292,108 +292,169 @@ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \
# that fired on every tree would be red on every ordinary PR in the repo and
# would be switched off inside a day.
#
# Every fixture carries its OWN VERSION and its OWN RUNS.md. Reaching for
# Every fixture carries its OWN VERSION and its OWN drills/ dir. Reaching for
# $ROOT/VERSION instead is exactly the coupling #146 had to fix: the suite
# then passes or fails on whichever state the repo happens to be in, so it
# goes red on the ceremony tree — the one tree where the release suite most
# needs to be trustworthy.
#
# Records are now ONE FILE PER VERSION, which is why this block is shorter
# than the one it replaces. The heading-parsing cases are gone because there
# is no heading to parse: em-dash fields, the optional ' — DATE' tail, and
# "an empty section must not borrow a neighbour's prose" were all artifacts
# of records sharing drill/RUNS.md. Deleting a test is only safe when the
# failure it described became unrepresentable, and that is the case here —
# except for the whitespace rule, which was never about headings and is
# pinned harder below.
# ---------------------------------------------------------------------------
DRILLED="$ROOT/.github/scripts/drill-recorded.sh"
check "drill-recorded: runnable bash" 0 "" bash -n "$DRILLED"
check "drill-recorded: executable" 0 "" test -x "$DRILLED"
# dtree <dir> <version> <runs-body...> — a two-file tree, self-contained
# dtree <dir> <version> [<record-version> <record-body...>] — a self-contained
# tree: its own VERSION, and its own drills/ dir created only when asked for.
# With 2 args there is NO drills/ dir at all, which is a case in its own right.
dtree() {
local d="$WORK/$1" v="$2"; shift 2
mkdir -p "$d"
printf '%s\n' "$v" > "$d/VERSION"
{ echo "# Drill run log"; echo; printf '%s\n' "$@"; } > "$d/RUNS.md"
if [ "$#" -gt 0 ]; then
local rec="$1"; shift
mkdir -p "$d/drills"
printf '%s\n' "$@" > "$d/drills/$rec.md"
fi
echo "$d"
}
drilled() { bash "$DRILLED" "$1/RUNS.md" "$1/VERSION"; }
drilled() { bash "$DRILLED" "$1/drills" "$1/VERSION"; }
# --- the -dev steady state: nothing to assert ------------------------------
# No record at all, and that must PASS. This is the case that makes the guard
# survivable on an ordinary PR.
T="$(dtree drill-dev 0.8.1-dev 'No drill sections in here at all.')"
check "drill-recorded: a -dev tree with NO record at all passes" 0 "nothing to assert" drilled "$T"
T="$(dtree drill-dev 0.8.1-dev)"
check "drill-recorded: a -dev tree with NO drills dir at all passes" 0 "nothing to assert" drilled "$T"
check "drill-recorded: ...and says why it declined to judge" 0 "development tree" drilled "$T"
# --- the ceremony tree: a record is required and must carry prose ----------
T="$(dtree drill-ok 0.9.0 \
'## Release drill — 0.9.0 — 2026-07-21' '' '- 47/47 on real hardware' '' \
'## Run history' '' '- older prose')"
check "drill-recorded: a bare VERSION with a matching non-empty record passes" \
# --- the ceremony tree: a record is required and must not be blank ---------
T="$(dtree drill-ok 0.9.0 0.9.0 '# Release drill — 0.9.0' '' '- 47/47 on real hardware')"
check "drill-recorded: a bare VERSION with a non-empty record for it passes" \
0 "has a drill record" drilled "$T"
# The date tail is optional; the heading without it is still a record.
T="$(dtree drill-nodate 0.9.0 '## Release drill — 0.9.0' '' '- ran it, 47/47')"
check "drill-recorded: ...and the ' — DATE' tail is optional" 0 "has a drill record" drilled "$T"
T="$(dtree drill-none 0.9.0 '## Run history' '' '- prose about earlier runs')"
check "drill-recorded: a bare VERSION with NO record fails" 1 "no drill record" drilled "$T"
# No drills/ dir AT ALL is the commonest way to fail this: the release branch
# was cut and nobody ran anything. It must fail loudly, not error out on a
# missing directory in some way that reads like a broken guard.
T="$(dtree drill-nodir 0.9.0)"
check "drill-recorded: a bare VERSION with NO drills dir fails" 1 "no drill record" drilled "$T"
check "drill-recorded: ...and the failure names the version" 1 "VERSION is '0.9.0'" drilled "$T"
# The dir exists — some other release left its record here — but this version
# has none. A directory listing is not evidence about a particular version.
T="$(dtree drill-otherver 0.9.0 0.8.0 '# Release drill — 0.8.0' '' '- the previous release')"
check "drill-recorded: a drills dir with no file for THIS version fails" \
1 "no drill record" drilled "$T"
check "drill-recorded: ...and names the unblock — run the drill and record it" \
1 "RUN THE DRILL" drilled "$T"
check "drill-recorded: ...and names the path it wanted" 1 "drills/0.9.0.md" drilled "$T"
check "drill-recorded: ...and offers the recorded maintainer waiver as the other way out" \
1 "WAIVED" drilled "$T"
check "drill-recorded: ...and names the three releases that shipped through the gap" \
1 "#95, #114 and #148" drilled "$T"
# A heading with nothing under it is the same failure as no heading: the
# ceremony wrote the shape and never wrote the evidence.
T="$(dtree drill-empty 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' '' '' '## Run history' '' '- older prose')"
check "drill-recorded: a bare VERSION whose record is present but EMPTY fails" \
1 "no drill record" drilled "$T"
# A file that exists and says nothing is the same failure as no file: the
# ceremony created the shape and never wrote the evidence. `touch` must not
# be able to satisfy a gate whose entire job is to demand evidence.
mkdir -p "$WORK/drill-emptyfile/drills"
printf '0.9.0\n' > "$WORK/drill-emptyfile/VERSION"
: > "$WORK/drill-emptyfile/drills/0.9.0.md"
check "drill-recorded: a record file that is present but EMPTY fails" \
1 "no drill record" drilled "$WORK/drill-emptyfile"
# ...and WHITESPACE is not evidence either. The first cut extracted with
# `sed '/./,$!d'`, where `.` matches a space — so a heading followed by one
# tab passed the guard while the failure text promised "at least one non-blank
# line". An evidence-free release for the price of an invisible character, on
# the one check whose entire job is to demand evidence. All three reviewers on
# #149 found it independently, which is the level of scrutiny it deserved and
# not a level it should ever need again.
T="$(dtree drill-blank 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' ' ' ' ' '' '## Run history' '' '- older prose')"
check "drill-recorded: a record body of only spaces and tabs fails (#149)" \
1 "no drill record" drilled "$T"
# ...and WHITESPACE is not evidence either. This is the ONE rule carried over
# from the heading-parsing guard, because it was never a heading problem. The
# first cut extracted with `sed '/./,$!d'`, where `.` matches a space — so a
# record whose body was one tab passed a guard that promised "at least one
# non-blank line". An evidence-free release for the price of an invisible
# character. All three reviewers on #149 found it independently, which is the
# level of scrutiny it deserved and not a level it should ever need again.
# Splitting the file per version removed the parsing; it did not remove this.
mkdir -p "$WORK/drill-blank/drills"
printf '0.9.0\n' > "$WORK/drill-blank/VERSION"
printf ' \n\t\n\n' > "$WORK/drill-blank/drills/0.9.0.md"
check "drill-recorded: a record of only spaces, tabs and newlines fails (#149)" \
1 "no drill record" drilled "$WORK/drill-blank"
# --- the version is matched WHOLE, both directions -------------------------
# release-notes.sh's trap, in a new file: a substring match would let an rc
# drill stand in for the release it was a candidate for, which is the one
# thing a release-candidate drill by definition did not measure.
T="$(dtree drill-rc-only 0.9.0 '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill')"
check "drill-recorded: a 0.9.0-rc1 record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T"
T="$(dtree drill-rel-only 0.9.0-rc1 '## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')"
check "drill-recorded: ...and a 0.9.0 record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T"
# ...and with both present, each resolves to its own section and neither
# borrows the other's prose.
T="$(dtree drill-both 0.9.0 \
'## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \
'## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')"
check "drill-recorded: with both present, the release resolves to its own" \
0 "has a drill record" drilled "$T"
# The sharp version of the same claim: the rc section is full, the release's
# own section is EMPTY. If emptiness were judged over anything but this
# version's own lines, this tree would pass on the rc's prose.
T="$(dtree drill-borrow 0.9.0 \
'## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \
'## Release drill — 0.9.0 — 2026-07-21' '')"
check "drill-recorded: ...and an empty release section cannot borrow the rc's prose" \
1 "no drill record" drilled "$T"
# Under the old scheme this took deliberate field-matching against a heading,
# and getting it wrong would have let a release-candidate drill stand in for
# the release it was a candidate for — the one thing an rc drill by
# definition did not measure. One file per version makes it free: these are
# simply different paths. Pinned anyway, in both directions, so that a future
# rewrite reaching for a prefix or glob match cannot quietly reintroduce it.
T="$(dtree drill-rc-only 0.9.0 0.9.0-rc1 '# Release drill — 0.9.0-rc1' '' '- the rc drill')"
check "drill-recorded: a 0.9.0-rc1.md record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T"
T="$(dtree drill-rel-only 0.9.0-rc1 0.9.0 '# Release drill — 0.9.0' '' '- the release drill')"
check "drill-recorded: ...and a 0.9.0.md record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T"
# --- degenerate trees refuse rather than pass by accident ------------------
# A guard that cannot read the version cannot know whether this tree is its
# business, and "could not tell" must never resolve to "allowed".
check "drill-recorded: a missing VERSION refuses by path" 1 "no such file" \
bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/nope-version"
bash "$DRILLED" "$WORK/drill-ok/drills" "$WORK/nope-version"
mkdir -p "$WORK/drill-empty-ver"; : > "$WORK/drill-empty-ver/VERSION"
check "drill-recorded: an empty VERSION refuses" 1 "is empty" \
bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/drill-empty-ver/VERSION"
# The runs file is only needed once the tree is a release — a -dev tree must
# not be refused for a file it has no business reading.
check "drill-recorded: a release tree with a missing runs file refuses by path" \
1 "no such file" bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-ok/VERSION"
check "drill-recorded: ...but a -dev tree does not even look for it" 0 "nothing to assert" \
bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-dev/VERSION"
bash "$DRILLED" "$WORK/drill-ok/drills" "$WORK/drill-empty-ver/VERSION"
# The drills dir is only consulted once the tree is a release — a -dev tree
# must not be refused for a directory it has no business reading.
check "drill-recorded: a -dev tree does not even look for the drills dir" 0 "nothing to assert" \
bash "$DRILLED" "$WORK/nope-drills" "$WORK/drill-dev/VERSION"
# --- and the tree under test ----------------------------------------------
check "drill-recorded: THIS tree passes its own guard" 0 "" \
bash "$DRILLED" "$ROOT/drill/RUNS.md" "$ROOT/VERSION"
# The property is that the guard's VERDICT IS CORRECT FOR THIS TREE — not that
# it always passes. Those come apart on a ceremony tree, and demanding success
# is wrong there in a way that took a red release PR to notice.
#
# A -dev tree is vacuous, so it must pass. A ceremony tree passes only once a
# human has run the drill and written the record — which is the entire point of
# the gate. Asserting exit 0 unconditionally therefore made test/release.sh
# UN-GREENABLE on every release branch before its drill, and reported it as a
# `release-flow tests` failure rather than as the gate doing its job: the same
# noise-and-misattribution shape as #146, where a fixture read the repo's real
# VERSION and only misbehaved on the ceremony tree.
#
# So: assert the verdict that this tree's own state entails. Green in all three
# states — dev, ceremony-without-record, ceremony-with-record — and still red if
# the guard ever disagrees with the tree in front of it.
THIS_VER="$(tr -d '[:space:]' < "$ROOT/VERSION")"
case "$THIS_VER" in
*-dev)
check "drill-recorded: THIS tree is -dev, and the guard is vacuous on it" 0 "nothing to assert" \
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION" ;;
*)
if [ -s "$ROOT/drills/$THIS_VER.md" ]; then
check "drill-recorded: THIS ceremony tree HAS its record, and the guard accepts it" 0 "" \
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION"
else
check "drill-recorded: THIS ceremony tree has NO record yet, and the guard refuses it" 1 "no drill record" \
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION"
fi ;;
esac
# drills/ is release evidence and drill/RUNS.md is the harness's own log. The
# rewrite is only coherent if both keep existing and the docs say which is
# which; a reader who appends to the wrong one gets a red gate and no clue.
check "drills/: the directory documents itself" 0 "" test -f "$ROOT/drills/README.md"
check "drills/: ...and is plain, not a dot-directory invisible to globs (#116)" 1 "" \
test -d "$ROOT/.drills"
check "drills/: ...and distinguishes itself from the harness log" 0 "" \
grep -qF 'drill/RUNS.md' "$ROOT/drills/README.md"
check "drills/: ...and its worked example uses a version that can never ship" 0 "" \
grep -qF '9.9.9' "$ROOT/drills/README.md"
check "drill/RUNS.md: the harness log is untouched and still present" 0 "" \
test -f "$ROOT/drill/RUNS.md"
# No real record may be invented to make the suite green. The repo is
# 0.8.1-dev; fabricating drills/0.8.1.md would defeat the whole gate.
check "drills/: no fabricated record for an unshipped version" 1 "" \
test -f "$ROOT/drills/0.8.1.md"
# The guard is only a guard if CI runs it, and only a rule if the document
# that used to carry the rule now points at it.
@ -401,21 +462,38 @@ check "ci.yml: runs the drill-recorded guard" 0 "" \
grep -qF 'drill-recorded.sh' "$ROOT/.github/workflows/ci.yml"
check "CONTRIBUTING: names the guard that enforces the drill" 0 "" \
grep -qF 'drill-recorded.sh' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: documents the drill record's heading format" 0 "" \
grep -qF '## Release drill — X.Y.Z — DATE' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: the drill is ONE run over the whole stack, not a queue" 0 "" \
check "CONTRIBUTING: documents the drill record's path" 0 "" \
grep -qF 'drills/X.Y.Z.md' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and distinguishes it from the harness's own log" 0 "" \
grep -qF "harness's own log" "$ROOT/CONTRIBUTING.md"
# The three drills are INDEPENDENT. An earlier draft of this section said the
# drill was "ONE orchestrated run over the whole stack", which over-constrains
# it into a fixed sequence the repos cannot satisfy. Pinning the corrected
# claim, and negatively pinning the fixed-order language, because "surely you
# drill rig before box" is exactly the simplification a future editor makes.
check "CONTRIBUTING: the three drills are independent, in any order" 0 "" \
grep -qF 'The three drills are INDEPENDENT' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and no fixed-order framing survives" 1 "" \
grep -qF 'ONE orchestrated run' "$ROOT/CONTRIBUTING.md"
# The load-bearing correction: box and rig are mutually recursive, so a
# "release A before you can drill B" rule is not merely bureaucratic, it is
# unsatisfiable in principle. Pinning the mechanism that dissolves it —
# mint-time candidate refs — because that is the claim a future editor is
# most likely to "simplify" back into a linear order.
# unsatisfiable in principle. What dissolves it is that every drill pins the
# same fixed candidate refs — static identifiers that exist as soon as the
# release branches do — turning a runtime cycle into two independent tests.
check "CONTRIBUTING: ...drilling candidate refs, not released artifacts" 0 "" \
grep -qF 'candidate refs, not released artifacts' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and the same fixed refs are what dissolve the recursion" 0 "" \
grep -qF 'pins the same fixed set of candidate refs' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and each repo asserts a different thing" 0 "" \
grep -qF 'isolation contract' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and a combination defect means patch, re-drill, re-record" 0 "" \
grep -qF 're-drill' "$ROOT/CONTRIBUTING.md"
check "CONTRIBUTING: ...and the RIG_REF gap is stated as outstanding (#81)" 0 "" \
grep -qF 'RIG_REF' "$ROOT/CONTRIBUTING.md"
check "LABELS: documents blocker:drill-pending" 0 "" \
grep -qF 'blocker:drill-pending' "$ROOT/LABELS.md"
check "LABELS: ...and points at the per-version record, not the harness log" 0 "" \
grep -qF 'drills/X.Y.Z.md' "$ROOT/LABELS.md"
# ---------------------------------------------------------------------------
# changelog-monotonic.sh (#122) — no SHIPPED release heading may be DELETED.