forked from heavy-duty/box
CONTRIBUTING.md has said since #96 that the release PR is where the full real-hardware drill hangs, recorded in drill/RUNS.md. No release has ever done it: #95, #114 and #148 all shipped as a VERSION bump plus a CHANGELOG.md stamp, and RUNS.md carries no '## Release drill' section at all. A reviewer bot finally blocked on it — which is the point. The one time it was caught is the one time somebody happened to look, and that is not a gate. So the rule moves out of the document and into CI. .github/scripts/drill-recorded.sh, keyed on VERSION the same way changelog-armed.sh is: a -dev tree passes with nothing to assert (which is what keeps the guard installable — a version that fired on every PR would be switched off inside a day), and a bare VERSION must carry a section headed '## Release drill — <version>', optional ' — <date>' tail, with prose under it. The version is compared as a whole awk field, never as a substring, so 0.9.0 cannot be satisfied by a 0.9.0-rc1 drill or vice versa — release-notes.sh's trap, solved the same way so the two cannot disagree about what "the section for X" means. What it asserts is a RECORD, deliberately, not a passing drill: CI cannot run the drill (real hardware, the better part of an hour — ci.yml says as much about the rehearsal job it runs instead). That also keeps the maintainer waiver honest — a release that must ship undrilled writes that under the same heading, so the skip is a reviewable line in the diff rather than silence. Wired into ci.yml as its own step, NOT pull-request-only, for the reasoning #143 applied to the monotonic guard: the merge that publishes a release is a push to main carrying the same bare VERSION, so a PR-only check would leave the tree that actually ships unasserted. test/release.sh grows 27 cases (134 -> 161). Every fixture carries its own VERSION and its own RUNS.md — reaching for $ROOT/VERSION is the coupling #146 had to fix, and it goes red on the ceremony tree, the one tree where the release suite most needs to be trustworthy. CONTRIBUTING.md now states the flow (draft -> ready -> bot round -> drill -> state:needs-human -> merge), the heading format, that three releases shipped through the gap, and the recorded-waiver escape. It also describes the drill as ONE orchestrated run over the whole stack, because box and rig are mutually recursive and cannot be linearly ordered: rig sits below box as the host-builder ('rig bootstrap --host yes' installs box and runs setup-host) and above it as the guest-converger (a box new seed curls rig's installer and runs 'rig bootstrap <tenant>-box'), the inverted edge bin/box already documents as rig#28. The run is host bootstrap -> box new -> tenant converge -> cast. It drills CANDIDATE REFS, not released artifacts: RIG_REPO/RIG_REF are mint-time environment variables defaulting to heavy-duty/rig@main (bin/box:1116-1117), so a run pins the exact commits under test and no repo must be released before another can be drilled. Drilling the candidate is drilling the release — a release diff is VERSION + CHANGELOG.md, so nothing executable differs. One run, one shared run ID; each repo records its own legs citing that ID and the other two SHAs, and the guard reads only this repo's file. Recorded as a known gap, not fixed here: a released box still defaults RIG_REF to main, so a box minted a week after a drill is not the drilled combination. Pinning RIG_REF to a released rig tag in the templates is the outstanding step from #81 (rig#32 step 5). LABELS.md documents blocker:drill-pending — ceremony correct but unevidenced, maintainer-created because the bot account gets a 403 on label creation, with `blocked` standing in until it exists. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
117 lines
5.6 KiB
Bash
Executable file
117 lines
5.6 KiB
Bash
Executable file
#!/usr/bin/env bash
|
|
set -euo pipefail
|
|
|
|
# drill-recorded.sh [<runs-file>] [<version-file>] — assert that a RELEASE
|
|
# tree carries a drill record: that the full real-hardware drill this repo
|
|
# says a release rests on was actually run for this version, and written
|
|
# down in drill/RUNS.md.
|
|
#
|
|
# CONTRIBUTING.md has said since #96 that "this PR is where the release
|
|
# ritual hangs: the full drill on real hardware, recorded in drill/RUNS.md".
|
|
# No release has ever done it. #95, #114 and #148 all shipped as a VERSION
|
|
# bump plus a CHANGELOG.md stamp and nothing else, and the file they were
|
|
# supposed to append to has no '## Release drill' section anywhere in it.
|
|
# That is three releases through the same gap, because the gate was a
|
|
# sentence in a document and the only thing standing on it was a reviewer
|
|
# remembering to ask. A reviewer finally did — which is the point: the ONE
|
|
# time it was caught is the time somebody happened to look, and that is not a
|
|
# gate, it is luck with good manners.
|
|
#
|
|
# So the rule moves into CI, where it fires on every release PR whether or
|
|
# not anyone is paying attention. The rule, keyed on VERSION for the same
|
|
# reason changelog-armed.sh is — the two states are genuinely different:
|
|
#
|
|
# VERSION ends in -dev -> PASS. A development tree ships nothing, so
|
|
# there is nothing for it to have proven. Almost
|
|
# every PR in this repo is this case, and a guard
|
|
# that nagged all of them would be turned off.
|
|
# VERSION is bare -> the ceremony tree, the one about to ship.
|
|
# drill/RUNS.md MUST carry a section headed
|
|
# '## Release drill — <version>' (optionally with
|
|
# ' — <date>' after it), and that section must
|
|
# have prose in it.
|
|
#
|
|
# What this guard asserts is a RECORD, deliberately — not a passing drill.
|
|
# CI cannot run the drill: it wants real hardware, a real Incus, and the
|
|
# better part of an hour (see ci.yml, which says exactly this about the
|
|
# rehearsal job it runs instead). What CI can do is refuse to let a release
|
|
# claim a ritual it left no evidence of. That also leaves the maintainer
|
|
# waiver intact and honest: a release that must ship without a full drill
|
|
# records WHY under its own heading, which is a deliberate, reviewable commit
|
|
# in the diff — rather than the silent skip that got us here.
|
|
#
|
|
# A file of its own (not inlined in ci.yml) so test/release.sh can drive it
|
|
# against constructed trees for both states — the same discipline as
|
|
# changelog-armed.sh and release-notes.sh.
|
|
|
|
runs="${1:-drill/RUNS.md}"
|
|
version_file="${2:-VERSION}"
|
|
|
|
[ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; }
|
|
|
|
ver="$(tr -d '[:space:]' < "$version_file")"
|
|
[ -n "$ver" ] || { echo "drill-recorded: $version_file is empty" >&2; exit 1; }
|
|
|
|
case "$ver" in
|
|
*-dev)
|
|
# Nothing to assert, and saying so is the point: the operator reading a
|
|
# green log should be able to tell "the guard passed" from "the guard
|
|
# decided this tree was not its business".
|
|
echo "drill-recorded: VERSION '$ver' is a development tree — nothing to assert; only ceremony trees ship"
|
|
exit 0
|
|
;;
|
|
esac
|
|
|
|
[ -f "$runs" ] || { echo "drill-recorded: no such file: $runs" >&2; exit 1; }
|
|
|
|
# The section for this version: everything between its own heading and the
|
|
# next '## '. The version is compared WHOLE — as a field, never as a
|
|
# substring or a regex — so '0.9.0' can never be satisfied by a
|
|
# '0.9.0-rc1' section (or vice versa), and there are no dots to escape.
|
|
# release-notes.sh solves the identical trap the identical way; the two
|
|
# scripts must not disagree about what "the section for X" means.
|
|
#
|
|
# The heading shape is '## Release drill — <ver>' with an OPTIONAL
|
|
# ' — <date>' tail, i.e. fields: '##' 'Release' 'drill' '—' '<ver>' ['—' ...].
|
|
# Pinning the leading fields as well as the version keeps some other '## '
|
|
# heading that merely mentions the number from counting as a record.
|
|
#
|
|
# The em dash is passed IN as a variable rather than written into the awk
|
|
# program, because '\x' escapes in an awk string are a gawk extension and CI
|
|
# runs on ubuntu-latest, where awk is mawk.
|
|
record="$(awk -v ver="$ver" -v dash="—" '
|
|
/^## / {
|
|
grab = ($2 == "Release" && $3 == "drill" && $4 == dash && $5 == ver \
|
|
&& (NF == 5 || $6 == dash))
|
|
next
|
|
}
|
|
grab { print }
|
|
' "$runs" | sed '/./,$!d')"
|
|
|
|
if [ -z "$record" ]; then
|
|
cat >&2 <<EOF
|
|
drill-recorded: VERSION is '$ver' — a release — and $runs has no drill record
|
|
for it. The heading this looks for is:
|
|
|
|
## Release drill — $ver — DATE
|
|
|
|
...with at least one non-blank line under it. Either the section is absent
|
|
entirely, or it is present and empty; both mean the same thing, which is
|
|
that this release is asserting a ritual it has left no evidence of.
|
|
|
|
The unblock is to RUN THE DRILL (drill/drill.sh, on real hardware) and
|
|
record it in $runs under that heading — what it measured, what it found,
|
|
what it cost. CI cannot run the drill for you; it can only refuse a release
|
|
that never ran one.
|
|
|
|
If this release must ship without a full drill, that is a maintainer's call
|
|
to make and it is still recorded: write the section under the same heading
|
|
and say plainly that the drill was WAIVED and why. The guard requires a
|
|
record, not a passing result — so a skip is a visible, reviewable line in
|
|
the diff rather than the silent gap that let #95, #114 and #148 all ship
|
|
unproven. See CONTRIBUTING.md, "Releases".
|
|
EOF
|
|
exit 1
|
|
fi
|
|
|
|
echo "drill-recorded: VERSION '$ver' has a drill record in $runs"
|