diff --git a/.github/scripts/drill-recorded.sh b/.github/scripts/drill-recorded.sh index c7d1f73..7618662 100755 --- a/.github/scripts/drill-recorded.sh +++ b/.github/scripts/drill-recorded.sh @@ -1,35 +1,53 @@ #!/usr/bin/env bash set -euo pipefail -# drill-recorded.sh [] [] — assert that a RELEASE +# drill-recorded.sh [] [] — assert that a RELEASE # tree carries a drill record: that the full real-hardware drill this repo # says a release rests on was actually run for this version, and written -# down in drill/RUNS.md. +# down as drills/.md. # # CONTRIBUTING.md has said since #96 that "this PR is where the release -# ritual hangs: the full drill on real hardware, recorded in drill/RUNS.md". -# No release has ever done it. #95, #114 and #148 all shipped as a VERSION -# bump plus a CHANGELOG.md stamp and nothing else, and the file they were -# supposed to append to has no '## Release drill' section anywhere in it. -# That is three releases through the same gap, because the gate was a -# sentence in a document and the only thing standing on it was a reviewer -# remembering to ask. A reviewer finally did — which is the point: the ONE -# time it was caught is the time somebody happened to look, and that is not a -# gate, it is luck with good manners. +# ritual hangs: the full drill on real hardware, recorded". No release ever +# did it. #95, #114 and #148 all shipped as a VERSION bump plus a +# CHANGELOG.md stamp and nothing else. That is three releases through the +# same gap, because the gate was a sentence in a document and the only thing +# standing on it was a reviewer remembering to ask. A reviewer finally did — +# which is the point: the ONE time it was caught is the time somebody +# happened to look, and that is not a gate, it is luck with good manners. # # So the rule moves into CI, where it fires on every release PR whether or -# not anyone is paying attention. The rule, keyed on VERSION for the same -# reason changelog-armed.sh is — the two states are genuinely different: +# not anyone is paying attention. It is keyed on VERSION for the same reason +# changelog-armed.sh is — the two states are genuinely different: # # VERSION ends in -dev -> PASS. A development tree ships nothing, so # there is nothing for it to have proven. Almost # every PR in this repo is this case, and a guard # that nagged all of them would be turned off. # VERSION is bare -> the ceremony tree, the one about to ship. -# drill/RUNS.md MUST carry a section headed -# '## Release drill — ' (optionally with -# ' — ' after it), and that section must -# have prose in it. +# /.md MUST exist and carry +# at least one non-whitespace character. +# +# ONE FILE PER VERSION, and that is the whole design. Records used to be +# sections sharing drill/RUNS.md, and every hard edge this script used to +# have existed only because of that sharing: em-dash field matching, an +# optional ' — DATE' tail, whole-version comparison so '0.9.0-rc1' could not +# satisfy '0.9.0', avoiding '\x' escapes because CI runs mawk not gawk, and a +# non-blank body rule to tell an empty section from a filled one. Two +# separate defects were found in review because of that complexity — a +# `sed '/./,$!d'` whitespace bypass, and heading-grammar drift from the +# sibling repos. Splitting the file makes almost all of it UNREPRESENTABLE: +# '0.9.0.md' and '0.9.0-rc1.md' are simply different files, so whole-version +# matching is free rather than a trap, and there is no grammar left to drift. +# +# The directory is plain 'drills', NOT '.drills'. A dot-directory is +# invisible to a glob without dotglob, which is exactly what caused #116 and +# #118 in this repo; evidence a sweep cannot see is evidence that goes +# missing quietly. +# +# drills/ is RELEASE EVIDENCE, one file per shipped version. It is NOT +# drill/RUNS.md, which stays exactly as it is: the harness's own run log, +# traps table and lore, a different artifact with a different purpose. This +# guard reads drills/ and never looks at drill/RUNS.md. # # What this guard asserts is a RECORD, deliberately — not a passing drill. # CI cannot run the drill: it wants real hardware, a real Incus, and the @@ -37,16 +55,19 @@ set -euo pipefail # rehearsal job it runs instead). What CI can do is refuse to let a release # claim a ritual it left no evidence of. That also leaves the maintainer # waiver intact and honest: a release that must ship without a full drill -# records WHY under its own heading, which is a deliberate, reviewable commit -# in the diff — rather than the silent skip that got us here. +# records WHY in its own file, which is a deliberate, reviewable commit in +# the diff — rather than the silent skip that got us here. # # A file of its own (not inlined in ci.yml) so test/release.sh can drive it # against constructed trees for both states — the same discipline as # changelog-armed.sh and release-notes.sh. -runs="${1:-drill/RUNS.md}" +drills="${1:-drills}" version_file="${2:-VERSION}" +# A missing or empty version file is an ERROR, never a silent pass. A guard +# that cannot read the version cannot know whether this tree is its business, +# and "could not tell" must not resolve to "allowed". [ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; } ver="$(tr -d '[:space:]' < "$version_file")" @@ -62,67 +83,43 @@ case "$ver" in ;; esac -[ -f "$runs" ] || { echo "drill-recorded: no such file: $runs" >&2; exit 1; } +record="$drills/$ver.md" -# The section for this version: everything between its own heading and the -# next '## '. The version is compared WHOLE — as a field, never as a -# substring or a regex — so '0.9.0' can never be satisfied by a -# '0.9.0-rc1' section (or vice versa), and there are no dots to escape. -# release-notes.sh solves the identical trap the identical way; the two -# scripts must not disagree about what "the section for X" means. -# -# The heading shape is '## Release drill — ' with an OPTIONAL -# ' — ' tail, i.e. fields: '##' 'Release' 'drill' '—' '' ['—' ...]. -# Pinning the leading fields as well as the version keeps some other '## ' -# heading that merely mentions the number from counting as a record. -# -# The em dash is passed IN as a variable rather than written into the awk -# program, because '\x' escapes in an awk string are a gawk extension and CI -# runs on ubuntu-latest, where awk is mawk. -# -# `grab && NF` is the non-blank rule, and it is load-bearing rather than -# tidiness. NF is 0 on a line that is empty OR contains only whitespace, so -# `record` ends up non-empty exactly when a line with actual content exists. -# The first cut of this piped through `sed '/./,$!d'` instead, which drops -# leading blank lines but keeps a line of spaces — `.` matches a space. A -# heading followed by nothing but a tab therefore satisfied the guard and -# shipped an evidence-free release, while the failure text below promised "at -# least one non-blank line". The implementation was looser than its own -# contract, which on a gate is the whole ballgame: the bypass costs one -# invisible character. Caught by all three reviewers on #149, independently. -record="$(awk -v ver="$ver" -v dash="—" ' - /^## / { - grab = ($2 == "Release" && $3 == "drill" && $4 == dash && $5 == ver \ - && (NF == 5 || $6 == dash)) - next - } - grab && NF { print } -' "$runs")" - -if [ -z "$record" ]; then +# The one rule that survives the rewrite, and it survives because it was +# never really about heading parsing: a file of only spaces, tabs and +# newlines is NOT a record. The first cut of the old guard extracted with +# `sed '/./,$!d'`, where `.` matches a space — so a record whose body was one +# tab satisfied a guard that promised "at least one non-blank line". An +# evidence-free release for the price of an invisible character, on the one +# check whose entire job is to demand evidence. Existence alone is a weaker +# claim than `touch` can defeat, so existence alone is not the test. +if [ ! -f "$record" ] || ! grep -q '[^[:space:]]' "$record"; then cat >&2 <.md` ### Changed diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e6579a0..ed9c95e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -163,39 +163,70 @@ A release is a PR, and merging it ships it real hardware and the better part of an hour, so it runs on the release PR's branch and nowhere else. - Record it in [drill/RUNS.md](drill/RUNS.md) under its own heading: + Record it in its own file, named exactly for the version: - ```markdown - ## Release drill — X.Y.Z — DATE + ``` + drills/X.Y.Z.md ``` - ...with the run under it — what it measured, what it found, what it cost. + ...with the run in it — what it measured, what it found, what it cost. + See [drills/README.md](drills/README.md) for the shape. [.github/scripts/drill-recorded.sh](.github/scripts/drill-recorded.sh) asserts exactly that on every tree with a bare `VERSION`, which is every `release` PR and the merge that publishes it; a `-dev` tree passes with nothing to assert. It is **no longer a thing a reviewer has to remember**. + **`drills/` is release evidence; [`drill/RUNS.md`](drill/RUNS.md) is the + harness's own log.** Two artifacts, two purposes. `drill/RUNS.md` keeps + being what it has always been — every run of the harness, the traps table, + the lore — and is not release-scoped. `drills/X.Y.Z.md` is the per-release + record the gate reads. Appending to `drill/RUNS.md` does not satisfy the + gate, and is not meant to. + + One file per version is also why the guard is short. When records shared a + file it had to match em-dash heading fields, tolerate an optional date + tail, and compare versions *whole* so `0.9.0-rc1` could not satisfy + `0.9.0` — and two separate defects were found in review because of that + complexity. Now `0.9.0.md` and `0.9.0-rc1.md` are simply different files + and none of it is representable. + It became CI's job because remembering did not work. The sentence this paragraph replaces described a step **no release had ever performed** — - `drill/RUNS.md` carried no `## Release drill` section at all — and #95, - #114 and #148 all shipped through the gap as a `VERSION` bump plus a - `CHANGELOG.md` stamp. The one time it was caught was the one time somebody - happened to look, which is not a gate. + there was no drill record anywhere in the repo — and #95, #114 and #148 + all shipped through the gap as a `VERSION` bump plus a `CHANGELOG.md` + stamp. The one time it was caught was the one time somebody happened to + look, which is not a gate. - **The drill is ONE orchestrated run over the whole stack**, not three - drills in a queue. box and rig are **mutually recursive**, so there is no - linear order to put them in: rig sits *below* box as the host-builder + **The three drills are INDEPENDENT — there is no fixed order.** box, rig + and cast each drill separately, in any order, on any schedule, in separate + sittings if you like. What makes that safe is not sequencing, it is that + every drill **pins the same fixed set of candidate refs**: rig's drill + runs `--host yes` with `BOX_REF=release/`, so it exercises the + box that will actually ship; box's drill mints with + `RIG_REF=release/`, so it exercises the rig that will actually + ship. Both measure the same pair. + + That is what dissolves the box↔rig recursion. box and rig really are + **mutually recursive** — rig sits *below* box as the host-builder (`rig bootstrap --host yes` installs box and runs `setup-host`) and *above* it as the guest-converger (a `box new` seed's cloud-init curls - rig's installer and runs `rig bootstrap -box`). box's own source - says as much — the seeds "invert the rig→box install edge (rig#28: rig - installs box on hosts; now box guests install rig)". The run goes: + rig's installer and runs `rig bootstrap -box`); box's own source + says as much, the seeds "invert the rig→box install edge (rig#28: rig + installs box on hosts; now box guests install rig)". A cycle at *runtime* + becomes two independent tests against one fixed pair, because the refs are + **static identifiers that exist as soon as the release branches do**, long + before any drill runs. Nothing has to be released, or drilled, first. - 1. `rig bootstrap --host yes` on a bare Debian host — installs box, runs - `setup-host` - 2. `box new` mints a creds-free seed - 3. the seed converges itself via `rig bootstrap -box` - 4. cast on top + Within a single drill you of course bring the substrate up before probing + it — a host before a guest. That is how you run a drill; it is not an + ordering rule between repos. + + Each repo drills in a **different way** and asserts a different thing: box + asserts the **isolation contract** (the 85-probe VM trust boundary), rig + asserts **convergence** (a machine reaches its role, idempotently), cast + asserts **promotion** (A→B reproduces, the diff is idempotent). Three + different exercises sharing a substrate — not three phases of one script, + which is exactly why the records are per-repo. It drills **candidate refs, not released artifacts.** `RIG_REPO` and `RIG_REF` are mint-time environment variables (defaults @@ -206,11 +237,15 @@ A release is a PR, and merging it ships it `CHANGELOG.md` and nothing else, so no executable byte differs between the tree that was drilled and the tree that ships. - One run emits **one shared run ID**. Each repo records its own legs under - its own `## Release drill — X.Y.Z — DATE`, citing that run ID and the - other two repos' commit SHAs, so the three records reconcile into a single - run afterwards. The guard reads only this repo's file — it asserts box's - record exists, never the other two. + The drills of a release set share **one run ID**. Each repo records its own + legs in its own `drills/X.Y.Z.md`, citing that run ID and the other two + repos' commit SHAs, so the three records reconcile afterwards. The guard + reads only this repo's file — it asserts box's record exists, never the + other two. + + If a defect shows up only in the combination: **patch, re-drill, + re-record.** The three releases converge on a set that holds together; + they are not required to be right in one pass. **A known gap, and box is where it belongs.** A *released* box still defaults `RIG_REF` to `main`, so what a user mints a week after a drill is @@ -224,10 +259,10 @@ A release is a PR, and merging it ships it A maintainer **waiver** is possible, and it is still written down. The guard requires a *record*, not a passing result, so a release that must - ship without a full drill puts the section under the same heading and says - plainly that the drill was waived and why. Skipping then costs a - deliberate, reviewable line in the diff — which is precisely what the - three silent skips above did not. + ship without a full drill creates `drills/X.Y.Z.md` anyway and says plainly + that the drill was waived and why. Skipping then costs a deliberate, + reviewable file in the diff — which is precisely what the three silent + skips above did not. 2. **The maintainer's merge IS the release.** [release.yml](.github/workflows/release.yml) fires on the merged, `release`-labeled PR and asserts before creating anything: `VERSION` at diff --git a/LABELS.md b/LABELS.md index bc43d5d..ddfc75f 100644 --- a/LABELS.md +++ b/LABELS.md @@ -36,7 +36,7 @@ PR carries as many as apply. | `blocker:conflict` | `#B60205` | GitHub says `CONFLICTING` — the agent owes a **rebase** | it merges cleanly | | `blocker:ci-red` | `#B60205` | a check failed — the agent owes a **fix**, which a rebase will not provide | checks are green | | `blocker:unrequested` | `#E99695` | this head has no verdict from somebody — never reviewed, or staled by a push — and **nobody was asked** for one | reviews are requested | -| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record in [drill/RUNS.md](drill/RUNS.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) under `## Release drill — X.Y.Z` | +| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record at [`drills/X.Y.Z.md`](drills/README.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) at `drills/X.Y.Z.md` | `blocker:drill-pending` is the one blocker that says the diff is *fine*. The version is stamped, the changelog is right, CI's other guards are green — what diff --git a/drills/README.md b/drills/README.md new file mode 100644 index 0000000..bdb2659 --- /dev/null +++ b/drills/README.md @@ -0,0 +1,82 @@ +# drills/ — release evidence, one file per version + +This directory holds the **evidence that a release was proven on real +hardware**. One file per shipped version, named exactly for the version: + +``` +drills/0.9.0.md +drills/0.9.0-rc1.md +``` + +The name must match the contents of `VERSION` exactly. +[.github/scripts/drill-recorded.sh](../.github/scripts/drill-recorded.sh) +refuses any tree with a bare `VERSION` that has no such file, or whose file is +blank. A `-dev` tree passes with nothing to assert. + +Because each version owns a file, `0.9.0` and `0.9.0-rc1` can never be +confused for one another — they are simply different paths. That used to take +careful whole-version field matching inside one shared file; now it is free. + +## This is not `drill/RUNS.md` + +Two different artifacts, and the distinction is load-bearing: + +| | what it is | +|---|---| +| [`drill/RUNS.md`](../drill/RUNS.md) | the **harness's own history** — every run of `drill/drill.sh`, the traps table, the lore about what broke and why. It is not release-scoped and it is not going anywhere. | +| `drills/.md` | **release evidence** — the record that *this version* was drilled before it shipped. Release-scoped, one file, gated by CI. | + +Appending to `drill/RUNS.md` does not satisfy the release gate, and is not +meant to. Keep using it for what it has always been for. + +## What a record should contain + +- **What ran** — which drill, how many probes, `drill/drill.sh` invocation. +- **On what host** — the machine, the OS, the Incus version. "Real hardware" + is the claim; name the hardware. +- **The pinned candidate refs** — the exact `BOX_REF` / `RIG_REF` / + `CAST_REF` under test, and the other repos' commit SHAs. A drill that does + not say what it drilled proves nothing later. +- **The shared run ID**, so this record reconciles with the sibling repos'. +- **The numbers** — passed, failed, how long it took. +- **What failed**, plainly. + +**A failed drill is still a valid record.** The gate wants *evidence*, not +success. A record saying "83/85, criterion (m) regressed, here is the issue" +is a good record. So is a maintainer's written waiver explaining why this +release shipped without a full drill. What the gate refuses is silence — #95, +#114 and #148 all shipped unproven because a skip left no trace. + +## Worked example + +The version below is a **placeholder that can never be a real release**. +Copy the shape, not the number. + +```markdown +# Release drill — 9.9.9 + +- **Run ID:** `drill-9.9.9-20260721-01` (shared with rig, cast) +- **Host:** bare Debian 13, Ryzen 7 5800X / 64 GB, Incus 6.0.2 +- **Date:** 2026-07-21 +- **Candidate refs:** + - box `release/9.9.9` @ `abc1234` + - rig `release/4.4.4` @ `def5678` (minted with `RIG_REF=release/4.4.4`) + - cast `release/2.2.2` @ `9abcdef` + +## What ran + +`bash drill/drill.sh --ref release/9.9.9` — the full end-to-end: install the +stack, mint every template cold, snapshot and restore, uninstall to zero +residue. Then `drill/multiuser.sh` for the two-user grant matrix. + +## Result + +**84/85 passed, 1 failed.** 41 minutes wall clock. + +- Failed: `multiuser.sh` criterion (m) — the raw instance kept a stale route + after teardown. Filed as #999. Judged not release-blocking: it affects + teardown residue on a host that is about to be wiped, not the trust + boundary itself. +- The VM boundary probes (the 85-probe isolation contract) passed clean, + which is the assertion this repo's drill exists to make. +``` diff --git a/test/release.sh b/test/release.sh index 731df7f..da3df3f 100644 --- a/test/release.sh +++ b/test/release.sh @@ -279,7 +279,7 @@ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \ grep -qF 'changelog-armed.sh' "$ROOT/CONTRIBUTING.md" # --------------------------------------------------------------------------- -# drill-recorded.sh — a RELEASE tree has a drill record in drill/RUNS.md. +# drill-recorded.sh — a RELEASE tree has a drill record at drills/.md. # # The gap this closes is not a drift or a race; it is a rule that was only # ever a sentence. CONTRIBUTING.md has said since #96 that the release PR is @@ -292,108 +292,169 @@ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \ # that fired on every tree would be red on every ordinary PR in the repo and # would be switched off inside a day. # -# Every fixture carries its OWN VERSION and its OWN RUNS.md. Reaching for +# Every fixture carries its OWN VERSION and its OWN drills/ dir. Reaching for # $ROOT/VERSION instead is exactly the coupling #146 had to fix: the suite # then passes or fails on whichever state the repo happens to be in, so it # goes red on the ceremony tree — the one tree where the release suite most # needs to be trustworthy. +# +# Records are now ONE FILE PER VERSION, which is why this block is shorter +# than the one it replaces. The heading-parsing cases are gone because there +# is no heading to parse: em-dash fields, the optional ' — DATE' tail, and +# "an empty section must not borrow a neighbour's prose" were all artifacts +# of records sharing drill/RUNS.md. Deleting a test is only safe when the +# failure it described became unrepresentable, and that is the case here — +# except for the whitespace rule, which was never about headings and is +# pinned harder below. # --------------------------------------------------------------------------- DRILLED="$ROOT/.github/scripts/drill-recorded.sh" check "drill-recorded: runnable bash" 0 "" bash -n "$DRILLED" check "drill-recorded: executable" 0 "" test -x "$DRILLED" -# dtree — a two-file tree, self-contained +# dtree [ ] — a self-contained +# tree: its own VERSION, and its own drills/ dir created only when asked for. +# With 2 args there is NO drills/ dir at all, which is a case in its own right. dtree() { local d="$WORK/$1" v="$2"; shift 2 mkdir -p "$d" printf '%s\n' "$v" > "$d/VERSION" - { echo "# Drill run log"; echo; printf '%s\n' "$@"; } > "$d/RUNS.md" + if [ "$#" -gt 0 ]; then + local rec="$1"; shift + mkdir -p "$d/drills" + printf '%s\n' "$@" > "$d/drills/$rec.md" + fi echo "$d" } -drilled() { bash "$DRILLED" "$1/RUNS.md" "$1/VERSION"; } +drilled() { bash "$DRILLED" "$1/drills" "$1/VERSION"; } # --- the -dev steady state: nothing to assert ------------------------------ # No record at all, and that must PASS. This is the case that makes the guard # survivable on an ordinary PR. -T="$(dtree drill-dev 0.8.1-dev 'No drill sections in here at all.')" -check "drill-recorded: a -dev tree with NO record at all passes" 0 "nothing to assert" drilled "$T" +T="$(dtree drill-dev 0.8.1-dev)" +check "drill-recorded: a -dev tree with NO drills dir at all passes" 0 "nothing to assert" drilled "$T" check "drill-recorded: ...and says why it declined to judge" 0 "development tree" drilled "$T" -# --- the ceremony tree: a record is required and must carry prose ---------- -T="$(dtree drill-ok 0.9.0 \ - '## Release drill — 0.9.0 — 2026-07-21' '' '- 47/47 on real hardware' '' \ - '## Run history' '' '- older prose')" -check "drill-recorded: a bare VERSION with a matching non-empty record passes" \ +# --- the ceremony tree: a record is required and must not be blank --------- +T="$(dtree drill-ok 0.9.0 0.9.0 '# Release drill — 0.9.0' '' '- 47/47 on real hardware')" +check "drill-recorded: a bare VERSION with a non-empty record for it passes" \ 0 "has a drill record" drilled "$T" -# The date tail is optional; the heading without it is still a record. -T="$(dtree drill-nodate 0.9.0 '## Release drill — 0.9.0' '' '- ran it, 47/47')" -check "drill-recorded: ...and the ' — DATE' tail is optional" 0 "has a drill record" drilled "$T" -T="$(dtree drill-none 0.9.0 '## Run history' '' '- prose about earlier runs')" -check "drill-recorded: a bare VERSION with NO record fails" 1 "no drill record" drilled "$T" +# No drills/ dir AT ALL is the commonest way to fail this: the release branch +# was cut and nobody ran anything. It must fail loudly, not error out on a +# missing directory in some way that reads like a broken guard. +T="$(dtree drill-nodir 0.9.0)" +check "drill-recorded: a bare VERSION with NO drills dir fails" 1 "no drill record" drilled "$T" check "drill-recorded: ...and the failure names the version" 1 "VERSION is '0.9.0'" drilled "$T" + +# The dir exists — some other release left its record here — but this version +# has none. A directory listing is not evidence about a particular version. +T="$(dtree drill-otherver 0.9.0 0.8.0 '# Release drill — 0.8.0' '' '- the previous release')" +check "drill-recorded: a drills dir with no file for THIS version fails" \ + 1 "no drill record" drilled "$T" + check "drill-recorded: ...and names the unblock — run the drill and record it" \ 1 "RUN THE DRILL" drilled "$T" +check "drill-recorded: ...and names the path it wanted" 1 "drills/0.9.0.md" drilled "$T" check "drill-recorded: ...and offers the recorded maintainer waiver as the other way out" \ 1 "WAIVED" drilled "$T" +check "drill-recorded: ...and names the three releases that shipped through the gap" \ + 1 "#95, #114 and #148" drilled "$T" -# A heading with nothing under it is the same failure as no heading: the -# ceremony wrote the shape and never wrote the evidence. -T="$(dtree drill-empty 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' '' '' '## Run history' '' '- older prose')" -check "drill-recorded: a bare VERSION whose record is present but EMPTY fails" \ - 1 "no drill record" drilled "$T" +# A file that exists and says nothing is the same failure as no file: the +# ceremony created the shape and never wrote the evidence. `touch` must not +# be able to satisfy a gate whose entire job is to demand evidence. +mkdir -p "$WORK/drill-emptyfile/drills" +printf '0.9.0\n' > "$WORK/drill-emptyfile/VERSION" +: > "$WORK/drill-emptyfile/drills/0.9.0.md" +check "drill-recorded: a record file that is present but EMPTY fails" \ + 1 "no drill record" drilled "$WORK/drill-emptyfile" -# ...and WHITESPACE is not evidence either. The first cut extracted with -# `sed '/./,$!d'`, where `.` matches a space — so a heading followed by one -# tab passed the guard while the failure text promised "at least one non-blank -# line". An evidence-free release for the price of an invisible character, on -# the one check whose entire job is to demand evidence. All three reviewers on -# #149 found it independently, which is the level of scrutiny it deserved and -# not a level it should ever need again. -T="$(dtree drill-blank 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' ' ' ' ' '' '## Run history' '' '- older prose')" -check "drill-recorded: a record body of only spaces and tabs fails (#149)" \ - 1 "no drill record" drilled "$T" +# ...and WHITESPACE is not evidence either. This is the ONE rule carried over +# from the heading-parsing guard, because it was never a heading problem. The +# first cut extracted with `sed '/./,$!d'`, where `.` matches a space — so a +# record whose body was one tab passed a guard that promised "at least one +# non-blank line". An evidence-free release for the price of an invisible +# character. All three reviewers on #149 found it independently, which is the +# level of scrutiny it deserved and not a level it should ever need again. +# Splitting the file per version removed the parsing; it did not remove this. +mkdir -p "$WORK/drill-blank/drills" +printf '0.9.0\n' > "$WORK/drill-blank/VERSION" +printf ' \n\t\n\n' > "$WORK/drill-blank/drills/0.9.0.md" +check "drill-recorded: a record of only spaces, tabs and newlines fails (#149)" \ + 1 "no drill record" drilled "$WORK/drill-blank" # --- the version is matched WHOLE, both directions ------------------------- -# release-notes.sh's trap, in a new file: a substring match would let an rc -# drill stand in for the release it was a candidate for, which is the one -# thing a release-candidate drill by definition did not measure. -T="$(dtree drill-rc-only 0.9.0 '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill')" -check "drill-recorded: a 0.9.0-rc1 record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T" -T="$(dtree drill-rel-only 0.9.0-rc1 '## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')" -check "drill-recorded: ...and a 0.9.0 record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T" -# ...and with both present, each resolves to its own section and neither -# borrows the other's prose. -T="$(dtree drill-both 0.9.0 \ - '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \ - '## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')" -check "drill-recorded: with both present, the release resolves to its own" \ - 0 "has a drill record" drilled "$T" -# The sharp version of the same claim: the rc section is full, the release's -# own section is EMPTY. If emptiness were judged over anything but this -# version's own lines, this tree would pass on the rc's prose. -T="$(dtree drill-borrow 0.9.0 \ - '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \ - '## Release drill — 0.9.0 — 2026-07-21' '')" -check "drill-recorded: ...and an empty release section cannot borrow the rc's prose" \ - 1 "no drill record" drilled "$T" +# Under the old scheme this took deliberate field-matching against a heading, +# and getting it wrong would have let a release-candidate drill stand in for +# the release it was a candidate for — the one thing an rc drill by +# definition did not measure. One file per version makes it free: these are +# simply different paths. Pinned anyway, in both directions, so that a future +# rewrite reaching for a prefix or glob match cannot quietly reintroduce it. +T="$(dtree drill-rc-only 0.9.0 0.9.0-rc1 '# Release drill — 0.9.0-rc1' '' '- the rc drill')" +check "drill-recorded: a 0.9.0-rc1.md record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T" +T="$(dtree drill-rel-only 0.9.0-rc1 0.9.0 '# Release drill — 0.9.0' '' '- the release drill')" +check "drill-recorded: ...and a 0.9.0.md record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T" # --- degenerate trees refuse rather than pass by accident ------------------ +# A guard that cannot read the version cannot know whether this tree is its +# business, and "could not tell" must never resolve to "allowed". check "drill-recorded: a missing VERSION refuses by path" 1 "no such file" \ - bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/nope-version" + bash "$DRILLED" "$WORK/drill-ok/drills" "$WORK/nope-version" mkdir -p "$WORK/drill-empty-ver"; : > "$WORK/drill-empty-ver/VERSION" check "drill-recorded: an empty VERSION refuses" 1 "is empty" \ - bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/drill-empty-ver/VERSION" -# The runs file is only needed once the tree is a release — a -dev tree must -# not be refused for a file it has no business reading. -check "drill-recorded: a release tree with a missing runs file refuses by path" \ - 1 "no such file" bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-ok/VERSION" -check "drill-recorded: ...but a -dev tree does not even look for it" 0 "nothing to assert" \ - bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-dev/VERSION" + bash "$DRILLED" "$WORK/drill-ok/drills" "$WORK/drill-empty-ver/VERSION" +# The drills dir is only consulted once the tree is a release — a -dev tree +# must not be refused for a directory it has no business reading. +check "drill-recorded: a -dev tree does not even look for the drills dir" 0 "nothing to assert" \ + bash "$DRILLED" "$WORK/nope-drills" "$WORK/drill-dev/VERSION" # --- and the tree under test ---------------------------------------------- -check "drill-recorded: THIS tree passes its own guard" 0 "" \ - bash "$DRILLED" "$ROOT/drill/RUNS.md" "$ROOT/VERSION" +# The property is that the guard's VERDICT IS CORRECT FOR THIS TREE — not that +# it always passes. Those come apart on a ceremony tree, and demanding success +# is wrong there in a way that took a red release PR to notice. +# +# A -dev tree is vacuous, so it must pass. A ceremony tree passes only once a +# human has run the drill and written the record — which is the entire point of +# the gate. Asserting exit 0 unconditionally therefore made test/release.sh +# UN-GREENABLE on every release branch before its drill, and reported it as a +# `release-flow tests` failure rather than as the gate doing its job: the same +# noise-and-misattribution shape as #146, where a fixture read the repo's real +# VERSION and only misbehaved on the ceremony tree. +# +# So: assert the verdict that this tree's own state entails. Green in all three +# states — dev, ceremony-without-record, ceremony-with-record — and still red if +# the guard ever disagrees with the tree in front of it. +THIS_VER="$(tr -d '[:space:]' < "$ROOT/VERSION")" +case "$THIS_VER" in + *-dev) + check "drill-recorded: THIS tree is -dev, and the guard is vacuous on it" 0 "nothing to assert" \ + bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION" ;; + *) + if [ -s "$ROOT/drills/$THIS_VER.md" ]; then + check "drill-recorded: THIS ceremony tree HAS its record, and the guard accepts it" 0 "" \ + bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION" + else + check "drill-recorded: THIS ceremony tree has NO record yet, and the guard refuses it" 1 "no drill record" \ + bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION" + fi ;; +esac + +# drills/ is release evidence and drill/RUNS.md is the harness's own log. The +# rewrite is only coherent if both keep existing and the docs say which is +# which; a reader who appends to the wrong one gets a red gate and no clue. +check "drills/: the directory documents itself" 0 "" test -f "$ROOT/drills/README.md" +check "drills/: ...and is plain, not a dot-directory invisible to globs (#116)" 1 "" \ + test -d "$ROOT/.drills" +check "drills/: ...and distinguishes itself from the harness log" 0 "" \ + grep -qF 'drill/RUNS.md' "$ROOT/drills/README.md" +check "drills/: ...and its worked example uses a version that can never ship" 0 "" \ + grep -qF '9.9.9' "$ROOT/drills/README.md" +check "drill/RUNS.md: the harness log is untouched and still present" 0 "" \ + test -f "$ROOT/drill/RUNS.md" +# No real record may be invented to make the suite green. The repo is +# 0.8.1-dev; fabricating drills/0.8.1.md would defeat the whole gate. +check "drills/: no fabricated record for an unshipped version" 1 "" \ + test -f "$ROOT/drills/0.8.1.md" # The guard is only a guard if CI runs it, and only a rule if the document # that used to carry the rule now points at it. @@ -401,21 +462,38 @@ check "ci.yml: runs the drill-recorded guard" 0 "" \ grep -qF 'drill-recorded.sh' "$ROOT/.github/workflows/ci.yml" check "CONTRIBUTING: names the guard that enforces the drill" 0 "" \ grep -qF 'drill-recorded.sh' "$ROOT/CONTRIBUTING.md" -check "CONTRIBUTING: documents the drill record's heading format" 0 "" \ - grep -qF '## Release drill — X.Y.Z — DATE' "$ROOT/CONTRIBUTING.md" -check "CONTRIBUTING: the drill is ONE run over the whole stack, not a queue" 0 "" \ +check "CONTRIBUTING: documents the drill record's path" 0 "" \ + grep -qF 'drills/X.Y.Z.md' "$ROOT/CONTRIBUTING.md" +check "CONTRIBUTING: ...and distinguishes it from the harness's own log" 0 "" \ + grep -qF "harness's own log" "$ROOT/CONTRIBUTING.md" +# The three drills are INDEPENDENT. An earlier draft of this section said the +# drill was "ONE orchestrated run over the whole stack", which over-constrains +# it into a fixed sequence the repos cannot satisfy. Pinning the corrected +# claim, and negatively pinning the fixed-order language, because "surely you +# drill rig before box" is exactly the simplification a future editor makes. +check "CONTRIBUTING: the three drills are independent, in any order" 0 "" \ + grep -qF 'The three drills are INDEPENDENT' "$ROOT/CONTRIBUTING.md" +check "CONTRIBUTING: ...and no fixed-order framing survives" 1 "" \ grep -qF 'ONE orchestrated run' "$ROOT/CONTRIBUTING.md" # The load-bearing correction: box and rig are mutually recursive, so a # "release A before you can drill B" rule is not merely bureaucratic, it is -# unsatisfiable in principle. Pinning the mechanism that dissolves it — -# mint-time candidate refs — because that is the claim a future editor is -# most likely to "simplify" back into a linear order. +# unsatisfiable in principle. What dissolves it is that every drill pins the +# same fixed candidate refs — static identifiers that exist as soon as the +# release branches do — turning a runtime cycle into two independent tests. check "CONTRIBUTING: ...drilling candidate refs, not released artifacts" 0 "" \ grep -qF 'candidate refs, not released artifacts' "$ROOT/CONTRIBUTING.md" +check "CONTRIBUTING: ...and the same fixed refs are what dissolve the recursion" 0 "" \ + grep -qF 'pins the same fixed set of candidate refs' "$ROOT/CONTRIBUTING.md" +check "CONTRIBUTING: ...and each repo asserts a different thing" 0 "" \ + grep -qF 'isolation contract' "$ROOT/CONTRIBUTING.md" +check "CONTRIBUTING: ...and a combination defect means patch, re-drill, re-record" 0 "" \ + grep -qF 're-drill' "$ROOT/CONTRIBUTING.md" check "CONTRIBUTING: ...and the RIG_REF gap is stated as outstanding (#81)" 0 "" \ grep -qF 'RIG_REF' "$ROOT/CONTRIBUTING.md" check "LABELS: documents blocker:drill-pending" 0 "" \ grep -qF 'blocker:drill-pending' "$ROOT/LABELS.md" +check "LABELS: ...and points at the per-version record, not the harness log" 0 "" \ + grep -qF 'drills/X.Y.Z.md' "$ROOT/LABELS.md" # --------------------------------------------------------------------------- # changelog-monotonic.sh (#122) — no SHIPPED release heading may be DELETED.