From 0678b22e01ccdb3f8c9d8bf07f161629983c355c Mon Sep 17 00:00:00 2001 From: dan-claude-bot Date: Tue, 21 Jul 2026 15:20:37 +0000 Subject: [PATCH 1/2] feat: CI refuses a release PR with no drill record MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CONTRIBUTING.md has said since #96 that the release PR is where the full real-hardware drill hangs, recorded in drill/RUNS.md. No release has ever done it: #95, #114 and #148 all shipped as a VERSION bump plus a CHANGELOG.md stamp, and RUNS.md carries no '## Release drill' section at all. A reviewer bot finally blocked on it — which is the point. The one time it was caught is the one time somebody happened to look, and that is not a gate. So the rule moves out of the document and into CI. .github/scripts/drill-recorded.sh, keyed on VERSION the same way changelog-armed.sh is: a -dev tree passes with nothing to assert (which is what keeps the guard installable — a version that fired on every PR would be switched off inside a day), and a bare VERSION must carry a section headed '## Release drill — ', optional ' — ' tail, with prose under it. The version is compared as a whole awk field, never as a substring, so 0.9.0 cannot be satisfied by a 0.9.0-rc1 drill or vice versa — release-notes.sh's trap, solved the same way so the two cannot disagree about what "the section for X" means. What it asserts is a RECORD, deliberately, not a passing drill: CI cannot run the drill (real hardware, the better part of an hour — ci.yml says as much about the rehearsal job it runs instead). That also keeps the maintainer waiver honest — a release that must ship undrilled writes that under the same heading, so the skip is a reviewable line in the diff rather than silence. Wired into ci.yml as its own step, NOT pull-request-only, for the reasoning #143 applied to the monotonic guard: the merge that publishes a release is a push to main carrying the same bare VERSION, so a PR-only check would leave the tree that actually ships unasserted. test/release.sh grows 27 cases (134 -> 161). Every fixture carries its own VERSION and its own RUNS.md — reaching for $ROOT/VERSION is the coupling #146 had to fix, and it goes red on the ceremony tree, the one tree where the release suite most needs to be trustworthy. CONTRIBUTING.md now states the flow (draft -> ready -> bot round -> drill -> state:needs-human -> merge), the heading format, that three releases shipped through the gap, and the recorded-waiver escape. It also describes the drill as ONE orchestrated run over the whole stack, because box and rig are mutually recursive and cannot be linearly ordered: rig sits below box as the host-builder ('rig bootstrap --host yes' installs box and runs setup-host) and above it as the guest-converger (a box new seed curls rig's installer and runs 'rig bootstrap -box'), the inverted edge bin/box already documents as rig#28. The run is host bootstrap -> box new -> tenant converge -> cast. It drills CANDIDATE REFS, not released artifacts: RIG_REPO/RIG_REF are mint-time environment variables defaulting to heavy-duty/rig@main (bin/box:1116-1117), so a run pins the exact commits under test and no repo must be released before another can be drilled. Drilling the candidate is drilling the release — a release diff is VERSION + CHANGELOG.md, so nothing executable differs. One run, one shared run ID; each repo records its own legs citing that ID and the other two SHAs, and the guard reads only this repo's file. Recorded as a known gap, not fixed here: a released box still defaults RIG_REF to main, so a box minted a week after a drill is not the drilled combination. Pinning RIG_REF to a released rig tag in the templates is the outstanding step from #81 (rig#32 step 5). LABELS.md documents blocker:drill-pending — ceremony correct but unevidenced, maintainer-created because the bot account gets a 403 on label creation, with `blocked` standing in until it exists. Co-Authored-By: Claude Opus 4.8 --- .github/scripts/drill-recorded.sh | 117 +++++++++++++++++++++++++++ .github/workflows/ci.yml | 17 ++++ CHANGELOG.md | 1 + CONTRIBUTING.md | 76 +++++++++++++++++- LABELS.md | 16 ++++ test/release.sh | 128 ++++++++++++++++++++++++++++++ 6 files changed, 351 insertions(+), 4 deletions(-) create mode 100755 .github/scripts/drill-recorded.sh diff --git a/.github/scripts/drill-recorded.sh b/.github/scripts/drill-recorded.sh new file mode 100755 index 0000000..b51614c --- /dev/null +++ b/.github/scripts/drill-recorded.sh @@ -0,0 +1,117 @@ +#!/usr/bin/env bash +set -euo pipefail + +# drill-recorded.sh [] [] — assert that a RELEASE +# tree carries a drill record: that the full real-hardware drill this repo +# says a release rests on was actually run for this version, and written +# down in drill/RUNS.md. +# +# CONTRIBUTING.md has said since #96 that "this PR is where the release +# ritual hangs: the full drill on real hardware, recorded in drill/RUNS.md". +# No release has ever done it. #95, #114 and #148 all shipped as a VERSION +# bump plus a CHANGELOG.md stamp and nothing else, and the file they were +# supposed to append to has no '## Release drill' section anywhere in it. +# That is three releases through the same gap, because the gate was a +# sentence in a document and the only thing standing on it was a reviewer +# remembering to ask. A reviewer finally did — which is the point: the ONE +# time it was caught is the time somebody happened to look, and that is not a +# gate, it is luck with good manners. +# +# So the rule moves into CI, where it fires on every release PR whether or +# not anyone is paying attention. The rule, keyed on VERSION for the same +# reason changelog-armed.sh is — the two states are genuinely different: +# +# VERSION ends in -dev -> PASS. A development tree ships nothing, so +# there is nothing for it to have proven. Almost +# every PR in this repo is this case, and a guard +# that nagged all of them would be turned off. +# VERSION is bare -> the ceremony tree, the one about to ship. +# drill/RUNS.md MUST carry a section headed +# '## Release drill — ' (optionally with +# ' — ' after it), and that section must +# have prose in it. +# +# What this guard asserts is a RECORD, deliberately — not a passing drill. +# CI cannot run the drill: it wants real hardware, a real Incus, and the +# better part of an hour (see ci.yml, which says exactly this about the +# rehearsal job it runs instead). What CI can do is refuse to let a release +# claim a ritual it left no evidence of. That also leaves the maintainer +# waiver intact and honest: a release that must ship without a full drill +# records WHY under its own heading, which is a deliberate, reviewable commit +# in the diff — rather than the silent skip that got us here. +# +# A file of its own (not inlined in ci.yml) so test/release.sh can drive it +# against constructed trees for both states — the same discipline as +# changelog-armed.sh and release-notes.sh. + +runs="${1:-drill/RUNS.md}" +version_file="${2:-VERSION}" + +[ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; } + +ver="$(tr -d '[:space:]' < "$version_file")" +[ -n "$ver" ] || { echo "drill-recorded: $version_file is empty" >&2; exit 1; } + +case "$ver" in + *-dev) + # Nothing to assert, and saying so is the point: the operator reading a + # green log should be able to tell "the guard passed" from "the guard + # decided this tree was not its business". + echo "drill-recorded: VERSION '$ver' is a development tree — nothing to assert; only ceremony trees ship" + exit 0 + ;; +esac + +[ -f "$runs" ] || { echo "drill-recorded: no such file: $runs" >&2; exit 1; } + +# The section for this version: everything between its own heading and the +# next '## '. The version is compared WHOLE — as a field, never as a +# substring or a regex — so '0.9.0' can never be satisfied by a +# '0.9.0-rc1' section (or vice versa), and there are no dots to escape. +# release-notes.sh solves the identical trap the identical way; the two +# scripts must not disagree about what "the section for X" means. +# +# The heading shape is '## Release drill — ' with an OPTIONAL +# ' — ' tail, i.e. fields: '##' 'Release' 'drill' '—' '' ['—' ...]. +# Pinning the leading fields as well as the version keeps some other '## ' +# heading that merely mentions the number from counting as a record. +# +# The em dash is passed IN as a variable rather than written into the awk +# program, because '\x' escapes in an awk string are a gawk extension and CI +# runs on ubuntu-latest, where awk is mawk. +record="$(awk -v ver="$ver" -v dash="—" ' + /^## / { + grab = ($2 == "Release" && $3 == "drill" && $4 == dash && $5 == ver \ + && (NF == 5 || $6 == dash)) + next + } + grab { print } +' "$runs" | sed '/./,$!d')" + +if [ -z "$record" ]; then + cat >&2 <-box`). box's own source + says as much — the seeds "invert the rig→box install edge (rig#28: rig + installs box on hosts; now box guests install rig)". The run goes: + + 1. `rig bootstrap --host yes` on a bare Debian host — installs box, runs + `setup-host` + 2. `box new` mints a creds-free seed + 3. the seed converges itself via `rig bootstrap -box` + 4. cast on top + + It drills **candidate refs, not released artifacts.** `RIG_REPO` and + `RIG_REF` are mint-time environment variables (defaults + `heavy-duty/rig` and `main`, `bin/box`), so a run pins the exact commits + under test. That dissolves the chicken-and-egg: **no repo has to be + released before another can be drilled.** And drilling the candidate *is* + drilling the release — a release PR's diff is `VERSION` plus + `CHANGELOG.md` and nothing else, so no executable byte differs between + the tree that was drilled and the tree that ships. + + One run emits **one shared run ID**. Each repo records its own legs under + its own `## Release drill — X.Y.Z — DATE`, citing that run ID and the + other two repos' commit SHAs, so the three records reconcile into a single + run afterwards. The guard reads only this repo's file — it asserts box's + record exists, never the other two. + + **A known gap, and box is where it belongs.** A *released* box still + defaults `RIG_REF` to `main`, so what a user mints a week after a drill is + not the combination that was drilled — the guest converges against + whatever rig's main has become since. Pinning `RIG_REF` to a released rig + tag in the templates is the outstanding step from + [#81](https://github.com/heavy-duty/box/issues/81) (rig#32 step 5), and it + is what would make a drilled combination reproducible for users. This PR + does not fix it; box's source already says the two directions "track main + unpinned today, said honestly ... until a release flow exists". + + A maintainer **waiver** is possible, and it is still written down. The + guard requires a *record*, not a passing result, so a release that must + ship without a full drill puts the section under the same heading and says + plainly that the drill was waived and why. Skipping then costs a + deliberate, reviewable line in the diff — which is precisely what the + three silent skips above did not. 2. **The maintainer's merge IS the release.** [release.yml](.github/workflows/release.yml) fires on the merged, `release`-labeled PR and asserts before creating anything: `VERSION` at diff --git a/LABELS.md b/LABELS.md index 6806147..bc43d5d 100644 --- a/LABELS.md +++ b/LABELS.md @@ -36,6 +36,22 @@ PR carries as many as apply. | `blocker:conflict` | `#B60205` | GitHub says `CONFLICTING` — the agent owes a **rebase** | it merges cleanly | | `blocker:ci-red` | `#B60205` | a check failed — the agent owes a **fix**, which a rebase will not provide | checks are green | | `blocker:unrequested` | `#E99695` | this head has no verdict from somebody — never reviewed, or staled by a push — and **nobody was asked** for one | reviews are requested | +| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record in [drill/RUNS.md](drill/RUNS.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) under `## Release drill — X.Y.Z` | + +`blocker:drill-pending` is the one blocker that says the diff is *fine*. The +version is stamped, the changelog is right, CI's other guards are green — what +is missing is the evidence that the release was proven on real hardware, which +[.github/scripts/drill-recorded.sh](.github/scripts/drill-recorded.sh) refuses +to let a release ship without (CONTRIBUTING.md, "Releases"). Naming it +separately from `blocker:ci-red` matters because the two owe different work: a +red check is a fix in the branch, a pending drill is an hour on a real host. + +It must be **created by a maintainer account** — the bot account gets a `403` +creating labels, so the labels workflow's bootstrap dispatch cannot mint it. +Until it exists in the repo, `blocked` stands in: it is the closest true thing +(the PR is waiting on something outside itself) and, usefully, the staleness +sweep already skips it, so a release parked overnight on a drill does not +collect a `stale`. One rule joins the axes: **`state:needs-human` requires zero blockers.** Any blocker means the work is the agent's, whatever the review round says. diff --git a/test/release.sh b/test/release.sh index 4c9a593..df7bba5 100644 --- a/test/release.sh +++ b/test/release.sh @@ -278,6 +278,134 @@ check "CONTRIBUTING: the ceremony re-arms '## Unreleased' after stamping" 0 "" \ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \ grep -qF 'changelog-armed.sh' "$ROOT/CONTRIBUTING.md" +# --------------------------------------------------------------------------- +# drill-recorded.sh — a RELEASE tree has a drill record in drill/RUNS.md. +# +# The gap this closes is not a drift or a race; it is a rule that was only +# ever a sentence. CONTRIBUTING.md has said since #96 that the release PR is +# where the full real-hardware drill hangs — and #95, #114 and #148 all +# shipped without one, because nothing but a reviewer's memory stood on it. +# +# Both states are constructed as real trees, same discipline as the armed +# block above, and for a sharper reason here: the -dev pass is not a +# convenience, it is what keeps the guard installable. A version of this rule +# that fired on every tree would be red on every ordinary PR in the repo and +# would be switched off inside a day. +# +# Every fixture carries its OWN VERSION and its OWN RUNS.md. Reaching for +# $ROOT/VERSION instead is exactly the coupling #146 had to fix: the suite +# then passes or fails on whichever state the repo happens to be in, so it +# goes red on the ceremony tree — the one tree where the release suite most +# needs to be trustworthy. +# --------------------------------------------------------------------------- +DRILLED="$ROOT/.github/scripts/drill-recorded.sh" +check "drill-recorded: runnable bash" 0 "" bash -n "$DRILLED" +check "drill-recorded: executable" 0 "" test -x "$DRILLED" + +# dtree — a two-file tree, self-contained +dtree() { + local d="$WORK/$1" v="$2"; shift 2 + mkdir -p "$d" + printf '%s\n' "$v" > "$d/VERSION" + { echo "# Drill run log"; echo; printf '%s\n' "$@"; } > "$d/RUNS.md" + echo "$d" +} +drilled() { bash "$DRILLED" "$1/RUNS.md" "$1/VERSION"; } + +# --- the -dev steady state: nothing to assert ------------------------------ +# No record at all, and that must PASS. This is the case that makes the guard +# survivable on an ordinary PR. +T="$(dtree drill-dev 0.8.1-dev 'No drill sections in here at all.')" +check "drill-recorded: a -dev tree with NO record at all passes" 0 "nothing to assert" drilled "$T" +check "drill-recorded: ...and says why it declined to judge" 0 "development tree" drilled "$T" + +# --- the ceremony tree: a record is required and must carry prose ---------- +T="$(dtree drill-ok 0.9.0 \ + '## Release drill — 0.9.0 — 2026-07-21' '' '- 47/47 on real hardware' '' \ + '## Run history' '' '- older prose')" +check "drill-recorded: a bare VERSION with a matching non-empty record passes" \ + 0 "has a drill record" drilled "$T" +# The date tail is optional; the heading without it is still a record. +T="$(dtree drill-nodate 0.9.0 '## Release drill — 0.9.0' '' '- ran it, 47/47')" +check "drill-recorded: ...and the ' — DATE' tail is optional" 0 "has a drill record" drilled "$T" + +T="$(dtree drill-none 0.9.0 '## Run history' '' '- prose about earlier runs')" +check "drill-recorded: a bare VERSION with NO record fails" 1 "no drill record" drilled "$T" +check "drill-recorded: ...and the failure names the version" 1 "VERSION is '0.9.0'" drilled "$T" +check "drill-recorded: ...and names the unblock — run the drill and record it" \ + 1 "RUN THE DRILL" drilled "$T" +check "drill-recorded: ...and offers the recorded maintainer waiver as the other way out" \ + 1 "WAIVED" drilled "$T" + +# A heading with nothing under it is the same failure as no heading: the +# ceremony wrote the shape and never wrote the evidence. +T="$(dtree drill-empty 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' '' '' '## Run history' '' '- older prose')" +check "drill-recorded: a bare VERSION whose record is present but EMPTY fails" \ + 1 "no drill record" drilled "$T" + +# --- the version is matched WHOLE, both directions ------------------------- +# release-notes.sh's trap, in a new file: a substring match would let an rc +# drill stand in for the release it was a candidate for, which is the one +# thing a release-candidate drill by definition did not measure. +T="$(dtree drill-rc-only 0.9.0 '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill')" +check "drill-recorded: a 0.9.0-rc1 record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T" +T="$(dtree drill-rel-only 0.9.0-rc1 '## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')" +check "drill-recorded: ...and a 0.9.0 record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T" +# ...and with both present, each resolves to its own section and neither +# borrows the other's prose. +T="$(dtree drill-both 0.9.0 \ + '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \ + '## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')" +check "drill-recorded: with both present, the release resolves to its own" \ + 0 "has a drill record" drilled "$T" +# The sharp version of the same claim: the rc section is full, the release's +# own section is EMPTY. If emptiness were judged over anything but this +# version's own lines, this tree would pass on the rc's prose. +T="$(dtree drill-borrow 0.9.0 \ + '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \ + '## Release drill — 0.9.0 — 2026-07-21' '')" +check "drill-recorded: ...and an empty release section cannot borrow the rc's prose" \ + 1 "no drill record" drilled "$T" + +# --- degenerate trees refuse rather than pass by accident ------------------ +check "drill-recorded: a missing VERSION refuses by path" 1 "no such file" \ + bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/nope-version" +mkdir -p "$WORK/drill-empty-ver"; : > "$WORK/drill-empty-ver/VERSION" +check "drill-recorded: an empty VERSION refuses" 1 "is empty" \ + bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/drill-empty-ver/VERSION" +# The runs file is only needed once the tree is a release — a -dev tree must +# not be refused for a file it has no business reading. +check "drill-recorded: a release tree with a missing runs file refuses by path" \ + 1 "no such file" bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-ok/VERSION" +check "drill-recorded: ...but a -dev tree does not even look for it" 0 "nothing to assert" \ + bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-dev/VERSION" + +# --- and the tree under test ---------------------------------------------- +check "drill-recorded: THIS tree passes its own guard" 0 "" \ + bash "$DRILLED" "$ROOT/drill/RUNS.md" "$ROOT/VERSION" + +# The guard is only a guard if CI runs it, and only a rule if the document +# that used to carry the rule now points at it. +check "ci.yml: runs the drill-recorded guard" 0 "" \ + grep -qF 'drill-recorded.sh' "$ROOT/.github/workflows/ci.yml" +check "CONTRIBUTING: names the guard that enforces the drill" 0 "" \ + grep -qF 'drill-recorded.sh' "$ROOT/CONTRIBUTING.md" +check "CONTRIBUTING: documents the drill record's heading format" 0 "" \ + grep -qF '## Release drill — X.Y.Z — DATE' "$ROOT/CONTRIBUTING.md" +check "CONTRIBUTING: the drill is ONE run over the whole stack, not a queue" 0 "" \ + grep -qF 'ONE orchestrated run' "$ROOT/CONTRIBUTING.md" +# The load-bearing correction: box and rig are mutually recursive, so a +# "release A before you can drill B" rule is not merely bureaucratic, it is +# unsatisfiable in principle. Pinning the mechanism that dissolves it — +# mint-time candidate refs — because that is the claim a future editor is +# most likely to "simplify" back into a linear order. +check "CONTRIBUTING: ...drilling candidate refs, not released artifacts" 0 "" \ + grep -qF 'candidate refs, not released artifacts' "$ROOT/CONTRIBUTING.md" +check "CONTRIBUTING: ...and the RIG_REF gap is stated as outstanding (#81)" 0 "" \ + grep -qF 'RIG_REF' "$ROOT/CONTRIBUTING.md" +check "LABELS: documents blocker:drill-pending" 0 "" \ + grep -qF 'blocker:drill-pending' "$ROOT/LABELS.md" + # --------------------------------------------------------------------------- # changelog-monotonic.sh (#122) — no SHIPPED release heading may be DELETED. # -- 2.45.2 From 877d9ded5527ea56665945c8173acbf3db9acae8 Mon Sep 17 00:00:00 2001 From: dan-claude-bot Date: Tue, 21 Jul 2026 15:53:29 +0000 Subject: [PATCH 2/2] fix: whitespace is not a drill record --- .github/scripts/drill-recorded.sh | 15 +++++++++++++-- test/release.sh | 11 +++++++++++ 2 files changed, 24 insertions(+), 2 deletions(-) diff --git a/.github/scripts/drill-recorded.sh b/.github/scripts/drill-recorded.sh index b51614c..c7d1f73 100755 --- a/.github/scripts/drill-recorded.sh +++ b/.github/scripts/drill-recorded.sh @@ -79,14 +79,25 @@ esac # The em dash is passed IN as a variable rather than written into the awk # program, because '\x' escapes in an awk string are a gawk extension and CI # runs on ubuntu-latest, where awk is mawk. +# +# `grab && NF` is the non-blank rule, and it is load-bearing rather than +# tidiness. NF is 0 on a line that is empty OR contains only whitespace, so +# `record` ends up non-empty exactly when a line with actual content exists. +# The first cut of this piped through `sed '/./,$!d'` instead, which drops +# leading blank lines but keeps a line of spaces — `.` matches a space. A +# heading followed by nothing but a tab therefore satisfied the guard and +# shipped an evidence-free release, while the failure text below promised "at +# least one non-blank line". The implementation was looser than its own +# contract, which on a gate is the whole ballgame: the bypass costs one +# invisible character. Caught by all three reviewers on #149, independently. record="$(awk -v ver="$ver" -v dash="—" ' /^## / { grab = ($2 == "Release" && $3 == "drill" && $4 == dash && $5 == ver \ && (NF == 5 || $6 == dash)) next } - grab { print } -' "$runs" | sed '/./,$!d')" + grab && NF { print } +' "$runs")" if [ -z "$record" ]; then cat >&2 <