box/.github/scripts/drill-recorded.sh
dan-claude-bot 0678b22e01 feat: CI refuses a release PR with no drill record
CONTRIBUTING.md has said since #96 that the release PR is where the full
real-hardware drill hangs, recorded in drill/RUNS.md. No release has ever
done it: #95, #114 and #148 all shipped as a VERSION bump plus a CHANGELOG.md
stamp, and RUNS.md carries no '## Release drill' section at all. A reviewer
bot finally blocked on it — which is the point. The one time it was caught is
the one time somebody happened to look, and that is not a gate.

So the rule moves out of the document and into CI.

.github/scripts/drill-recorded.sh, keyed on VERSION the same way
changelog-armed.sh is: a -dev tree passes with nothing to assert (which is
what keeps the guard installable — a version that fired on every PR would be
switched off inside a day), and a bare VERSION must carry a section headed
'## Release drill — <version>', optional ' — <date>' tail, with prose under
it. The version is compared as a whole awk field, never as a substring, so
0.9.0 cannot be satisfied by a 0.9.0-rc1 drill or vice versa —
release-notes.sh's trap, solved the same way so the two cannot disagree
about what "the section for X" means.

What it asserts is a RECORD, deliberately, not a passing drill: CI cannot run
the drill (real hardware, the better part of an hour — ci.yml says as much
about the rehearsal job it runs instead). That also keeps the maintainer
waiver honest — a release that must ship undrilled writes that under the same
heading, so the skip is a reviewable line in the diff rather than silence.

Wired into ci.yml as its own step, NOT pull-request-only, for the reasoning
#143 applied to the monotonic guard: the merge that publishes a release is a
push to main carrying the same bare VERSION, so a PR-only check would leave
the tree that actually ships unasserted.

test/release.sh grows 27 cases (134 -> 161). Every fixture carries its own
VERSION and its own RUNS.md — reaching for $ROOT/VERSION is the coupling #146
had to fix, and it goes red on the ceremony tree, the one tree where the
release suite most needs to be trustworthy.

CONTRIBUTING.md now states the flow (draft -> ready -> bot round -> drill ->
state:needs-human -> merge), the heading format, that three releases shipped
through the gap, and the recorded-waiver escape.

It also describes the drill as ONE orchestrated run over the whole stack,
because box and rig are mutually recursive and cannot be linearly ordered:
rig sits below box as the host-builder ('rig bootstrap --host yes' installs
box and runs setup-host) and above it as the guest-converger (a box new seed
curls rig's installer and runs 'rig bootstrap <tenant>-box'), the inverted
edge bin/box already documents as rig#28. The run is host bootstrap -> box
new -> tenant converge -> cast. It drills CANDIDATE REFS, not released
artifacts: RIG_REPO/RIG_REF are mint-time environment variables defaulting
to heavy-duty/rig@main (bin/box:1116-1117), so a run pins the exact commits
under test and no repo must be released before another can be drilled.
Drilling the candidate is drilling the release — a release diff is VERSION +
CHANGELOG.md, so nothing executable differs. One run, one shared run ID; each
repo records its own legs citing that ID and the other two SHAs, and the
guard reads only this repo's file.

Recorded as a known gap, not fixed here: a released box still defaults
RIG_REF to main, so a box minted a week after a drill is not the drilled
combination. Pinning RIG_REF to a released rig tag in the templates is the
outstanding step from #81 (rig#32 step 5).

LABELS.md documents blocker:drill-pending — ceremony correct but unevidenced,
maintainer-created because the bot account gets a 403 on label creation, with
`blocked` standing in until it exists.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 15:24:15 +00:00

117 lines
5.6 KiB
Bash
Executable file

#!/usr/bin/env bash
set -euo pipefail
# drill-recorded.sh [<runs-file>] [<version-file>] — assert that a RELEASE
# tree carries a drill record: that the full real-hardware drill this repo
# says a release rests on was actually run for this version, and written
# down in drill/RUNS.md.
#
# CONTRIBUTING.md has said since #96 that "this PR is where the release
# ritual hangs: the full drill on real hardware, recorded in drill/RUNS.md".
# No release has ever done it. #95, #114 and #148 all shipped as a VERSION
# bump plus a CHANGELOG.md stamp and nothing else, and the file they were
# supposed to append to has no '## Release drill' section anywhere in it.
# That is three releases through the same gap, because the gate was a
# sentence in a document and the only thing standing on it was a reviewer
# remembering to ask. A reviewer finally did — which is the point: the ONE
# time it was caught is the time somebody happened to look, and that is not a
# gate, it is luck with good manners.
#
# So the rule moves into CI, where it fires on every release PR whether or
# not anyone is paying attention. The rule, keyed on VERSION for the same
# reason changelog-armed.sh is — the two states are genuinely different:
#
# VERSION ends in -dev -> PASS. A development tree ships nothing, so
# there is nothing for it to have proven. Almost
# every PR in this repo is this case, and a guard
# that nagged all of them would be turned off.
# VERSION is bare -> the ceremony tree, the one about to ship.
# drill/RUNS.md MUST carry a section headed
# '## Release drill — <version>' (optionally with
# ' — <date>' after it), and that section must
# have prose in it.
#
# What this guard asserts is a RECORD, deliberately — not a passing drill.
# CI cannot run the drill: it wants real hardware, a real Incus, and the
# better part of an hour (see ci.yml, which says exactly this about the
# rehearsal job it runs instead). What CI can do is refuse to let a release
# claim a ritual it left no evidence of. That also leaves the maintainer
# waiver intact and honest: a release that must ship without a full drill
# records WHY under its own heading, which is a deliberate, reviewable commit
# in the diff — rather than the silent skip that got us here.
#
# A file of its own (not inlined in ci.yml) so test/release.sh can drive it
# against constructed trees for both states — the same discipline as
# changelog-armed.sh and release-notes.sh.
runs="${1:-drill/RUNS.md}"
version_file="${2:-VERSION}"
[ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; }
ver="$(tr -d '[:space:]' < "$version_file")"
[ -n "$ver" ] || { echo "drill-recorded: $version_file is empty" >&2; exit 1; }
case "$ver" in
*-dev)
# Nothing to assert, and saying so is the point: the operator reading a
# green log should be able to tell "the guard passed" from "the guard
# decided this tree was not its business".
echo "drill-recorded: VERSION '$ver' is a development tree — nothing to assert; only ceremony trees ship"
exit 0
;;
esac
[ -f "$runs" ] || { echo "drill-recorded: no such file: $runs" >&2; exit 1; }
# The section for this version: everything between its own heading and the
# next '## '. The version is compared WHOLE — as a field, never as a
# substring or a regex — so '0.9.0' can never be satisfied by a
# '0.9.0-rc1' section (or vice versa), and there are no dots to escape.
# release-notes.sh solves the identical trap the identical way; the two
# scripts must not disagree about what "the section for X" means.
#
# The heading shape is '## Release drill — <ver>' with an OPTIONAL
# ' — <date>' tail, i.e. fields: '##' 'Release' 'drill' '—' '<ver>' ['—' ...].
# Pinning the leading fields as well as the version keeps some other '## '
# heading that merely mentions the number from counting as a record.
#
# The em dash is passed IN as a variable rather than written into the awk
# program, because '\x' escapes in an awk string are a gawk extension and CI
# runs on ubuntu-latest, where awk is mawk.
record="$(awk -v ver="$ver" -v dash="—" '
/^## / {
grab = ($2 == "Release" && $3 == "drill" && $4 == dash && $5 == ver \
&& (NF == 5 || $6 == dash))
next
}
grab { print }
' "$runs" | sed '/./,$!d')"
if [ -z "$record" ]; then
cat >&2 <<EOF
drill-recorded: VERSION is '$ver' — a release — and $runs has no drill record
for it. The heading this looks for is:
## Release drill — $ver — DATE
...with at least one non-blank line under it. Either the section is absent
entirely, or it is present and empty; both mean the same thing, which is
that this release is asserting a ritual it has left no evidence of.
The unblock is to RUN THE DRILL (drill/drill.sh, on real hardware) and
record it in $runs under that heading — what it measured, what it found,
what it cost. CI cannot run the drill for you; it can only refuse a release
that never ran one.
If this release must ship without a full drill, that is a maintainer's call
to make and it is still recorded: write the section under the same heading
and say plainly that the drill was WAIVED and why. The guard requires a
record, not a passing result — so a skip is a visible, reviewable line in
the diff rather than the silent gap that let #95, #114 and #148 all ship
unproven. See CONTRIBUTING.md, "Releases".
EOF
exit 1
fi
echo "drill-recorded: VERSION '$ver' has a drill record in $runs"