refactor: one drill record per version, in drills/ #151
6 changed files with 364 additions and 172 deletions
141
.github/scripts/drill-recorded.sh
vendored
141
.github/scripts/drill-recorded.sh
vendored
|
|
@ -1,35 +1,53 @@
|
||||||
#!/usr/bin/env bash
|
#!/usr/bin/env bash
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
# drill-recorded.sh [<runs-file>] [<version-file>] — assert that a RELEASE
|
# drill-recorded.sh [<drills-dir>] [<version-file>] — assert that a RELEASE
|
||||||
# tree carries a drill record: that the full real-hardware drill this repo
|
# tree carries a drill record: that the full real-hardware drill this repo
|
||||||
# says a release rests on was actually run for this version, and written
|
# says a release rests on was actually run for this version, and written
|
||||||
# down in drill/RUNS.md.
|
# down as drills/<version>.md.
|
||||||
#
|
#
|
||||||
# CONTRIBUTING.md has said since #96 that "this PR is where the release
|
# CONTRIBUTING.md has said since #96 that "this PR is where the release
|
||||||
# ritual hangs: the full drill on real hardware, recorded in drill/RUNS.md".
|
# ritual hangs: the full drill on real hardware, recorded". No release ever
|
||||||
# No release has ever done it. #95, #114 and #148 all shipped as a VERSION
|
# did it. #95, #114 and #148 all shipped as a VERSION bump plus a
|
||||||
# bump plus a CHANGELOG.md stamp and nothing else, and the file they were
|
# CHANGELOG.md stamp and nothing else. That is three releases through the
|
||||||
# supposed to append to has no '## Release drill' section anywhere in it.
|
# same gap, because the gate was a sentence in a document and the only thing
|
||||||
# That is three releases through the same gap, because the gate was a
|
# standing on it was a reviewer remembering to ask. A reviewer finally did —
|
||||||
# sentence in a document and the only thing standing on it was a reviewer
|
# which is the point: the ONE time it was caught is the time somebody
|
||||||
# remembering to ask. A reviewer finally did — which is the point: the ONE
|
# happened to look, and that is not a gate, it is luck with good manners.
|
||||||
# time it was caught is the time somebody happened to look, and that is not a
|
|
||||||
# gate, it is luck with good manners.
|
|
||||||
#
|
#
|
||||||
# So the rule moves into CI, where it fires on every release PR whether or
|
# So the rule moves into CI, where it fires on every release PR whether or
|
||||||
# not anyone is paying attention. The rule, keyed on VERSION for the same
|
# not anyone is paying attention. It is keyed on VERSION for the same reason
|
||||||
# reason changelog-armed.sh is — the two states are genuinely different:
|
# changelog-armed.sh is — the two states are genuinely different:
|
||||||
#
|
#
|
||||||
# VERSION ends in -dev -> PASS. A development tree ships nothing, so
|
# VERSION ends in -dev -> PASS. A development tree ships nothing, so
|
||||||
# there is nothing for it to have proven. Almost
|
# there is nothing for it to have proven. Almost
|
||||||
# every PR in this repo is this case, and a guard
|
# every PR in this repo is this case, and a guard
|
||||||
# that nagged all of them would be turned off.
|
# that nagged all of them would be turned off.
|
||||||
# VERSION is bare -> the ceremony tree, the one about to ship.
|
# VERSION is bare -> the ceremony tree, the one about to ship.
|
||||||
# drill/RUNS.md MUST carry a section headed
|
# <drills-dir>/<version>.md MUST exist and carry
|
||||||
# '## Release drill — <version>' (optionally with
|
# at least one non-whitespace character.
|
||||||
# ' — <date>' after it), and that section must
|
#
|
||||||
# have prose in it.
|
# ONE FILE PER VERSION, and that is the whole design. Records used to be
|
||||||
|
# sections sharing drill/RUNS.md, and every hard edge this script used to
|
||||||
|
# have existed only because of that sharing: em-dash field matching, an
|
||||||
|
# optional ' — DATE' tail, whole-version comparison so '0.9.0-rc1' could not
|
||||||
|
# satisfy '0.9.0', avoiding '\x' escapes because CI runs mawk not gawk, and a
|
||||||
|
# non-blank body rule to tell an empty section from a filled one. Two
|
||||||
|
# separate defects were found in review because of that complexity — a
|
||||||
|
# `sed '/./,$!d'` whitespace bypass, and heading-grammar drift from the
|
||||||
|
# sibling repos. Splitting the file makes almost all of it UNREPRESENTABLE:
|
||||||
|
# '0.9.0.md' and '0.9.0-rc1.md' are simply different files, so whole-version
|
||||||
|
# matching is free rather than a trap, and there is no grammar left to drift.
|
||||||
|
#
|
||||||
|
# The directory is plain 'drills', NOT '.drills'. A dot-directory is
|
||||||
|
# invisible to a glob without dotglob, which is exactly what caused #116 and
|
||||||
|
# #118 in this repo; evidence a sweep cannot see is evidence that goes
|
||||||
|
# missing quietly.
|
||||||
|
#
|
||||||
|
# drills/ is RELEASE EVIDENCE, one file per shipped version. It is NOT
|
||||||
|
# drill/RUNS.md, which stays exactly as it is: the harness's own run log,
|
||||||
|
# traps table and lore, a different artifact with a different purpose. This
|
||||||
|
# guard reads drills/ and never looks at drill/RUNS.md.
|
||||||
#
|
#
|
||||||
# What this guard asserts is a RECORD, deliberately — not a passing drill.
|
# What this guard asserts is a RECORD, deliberately — not a passing drill.
|
||||||
# CI cannot run the drill: it wants real hardware, a real Incus, and the
|
# CI cannot run the drill: it wants real hardware, a real Incus, and the
|
||||||
|
|
@ -37,16 +55,19 @@ set -euo pipefail
|
||||||
# rehearsal job it runs instead). What CI can do is refuse to let a release
|
# rehearsal job it runs instead). What CI can do is refuse to let a release
|
||||||
# claim a ritual it left no evidence of. That also leaves the maintainer
|
# claim a ritual it left no evidence of. That also leaves the maintainer
|
||||||
# waiver intact and honest: a release that must ship without a full drill
|
# waiver intact and honest: a release that must ship without a full drill
|
||||||
# records WHY under its own heading, which is a deliberate, reviewable commit
|
# records WHY in its own file, which is a deliberate, reviewable commit in
|
||||||
# in the diff — rather than the silent skip that got us here.
|
# the diff — rather than the silent skip that got us here.
|
||||||
#
|
#
|
||||||
# A file of its own (not inlined in ci.yml) so test/release.sh can drive it
|
# A file of its own (not inlined in ci.yml) so test/release.sh can drive it
|
||||||
# against constructed trees for both states — the same discipline as
|
# against constructed trees for both states — the same discipline as
|
||||||
# changelog-armed.sh and release-notes.sh.
|
# changelog-armed.sh and release-notes.sh.
|
||||||
|
|
||||||
runs="${1:-drill/RUNS.md}"
|
drills="${1:-drills}"
|
||||||
version_file="${2:-VERSION}"
|
version_file="${2:-VERSION}"
|
||||||
|
|
||||||
|
# A missing or empty version file is an ERROR, never a silent pass. A guard
|
||||||
|
# that cannot read the version cannot know whether this tree is its business,
|
||||||
|
# and "could not tell" must not resolve to "allowed".
|
||||||
[ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; }
|
[ -f "$version_file" ] || { echo "drill-recorded: no such file: $version_file" >&2; exit 1; }
|
||||||
|
|
||||||
ver="$(tr -d '[:space:]' < "$version_file")"
|
ver="$(tr -d '[:space:]' < "$version_file")"
|
||||||
|
|
@ -62,67 +83,43 @@ case "$ver" in
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
|
|
||||||
[ -f "$runs" ] || { echo "drill-recorded: no such file: $runs" >&2; exit 1; }
|
record="$drills/$ver.md"
|
||||||
|
|
||||||
# The section for this version: everything between its own heading and the
|
# The one rule that survives the rewrite, and it survives because it was
|
||||||
# next '## '. The version is compared WHOLE — as a field, never as a
|
# never really about heading parsing: a file of only spaces, tabs and
|
||||||
# substring or a regex — so '0.9.0' can never be satisfied by a
|
# newlines is NOT a record. The first cut of the old guard extracted with
|
||||||
# '0.9.0-rc1' section (or vice versa), and there are no dots to escape.
|
# `sed '/./,$!d'`, where `.` matches a space — so a record whose body was one
|
||||||
# release-notes.sh solves the identical trap the identical way; the two
|
# tab satisfied a guard that promised "at least one non-blank line". An
|
||||||
# scripts must not disagree about what "the section for X" means.
|
# evidence-free release for the price of an invisible character, on the one
|
||||||
#
|
# check whose entire job is to demand evidence. Existence alone is a weaker
|
||||||
# The heading shape is '## Release drill — <ver>' with an OPTIONAL
|
# claim than `touch` can defeat, so existence alone is not the test.
|
||||||
# ' — <date>' tail, i.e. fields: '##' 'Release' 'drill' '—' '<ver>' ['—' ...].
|
if [ ! -f "$record" ] || ! grep -q '[^[:space:]]' "$record"; then
|
||||||
# Pinning the leading fields as well as the version keeps some other '## '
|
|
||||||
# heading that merely mentions the number from counting as a record.
|
|
||||||
#
|
|
||||||
# The em dash is passed IN as a variable rather than written into the awk
|
|
||||||
# program, because '\x' escapes in an awk string are a gawk extension and CI
|
|
||||||
# runs on ubuntu-latest, where awk is mawk.
|
|
||||||
#
|
|
||||||
# `grab && NF` is the non-blank rule, and it is load-bearing rather than
|
|
||||||
# tidiness. NF is 0 on a line that is empty OR contains only whitespace, so
|
|
||||||
# `record` ends up non-empty exactly when a line with actual content exists.
|
|
||||||
# The first cut of this piped through `sed '/./,$!d'` instead, which drops
|
|
||||||
# leading blank lines but keeps a line of spaces — `.` matches a space. A
|
|
||||||
# heading followed by nothing but a tab therefore satisfied the guard and
|
|
||||||
# shipped an evidence-free release, while the failure text below promised "at
|
|
||||||
# least one non-blank line". The implementation was looser than its own
|
|
||||||
# contract, which on a gate is the whole ballgame: the bypass costs one
|
|
||||||
# invisible character. Caught by all three reviewers on #149, independently.
|
|
||||||
record="$(awk -v ver="$ver" -v dash="—" '
|
|
||||||
/^## / {
|
|
||||||
grab = ($2 == "Release" && $3 == "drill" && $4 == dash && $5 == ver \
|
|
||||||
&& (NF == 5 || $6 == dash))
|
|
||||||
next
|
|
||||||
}
|
|
||||||
grab && NF { print }
|
|
||||||
' "$runs")"
|
|
||||||
|
|
||||||
if [ -z "$record" ]; then
|
|
||||||
cat >&2 <<EOF
|
cat >&2 <<EOF
|
||||||
drill-recorded: VERSION is '$ver' — a release — and $runs has no drill record
|
drill-recorded: VERSION is '$ver' — a release — and there is no drill record
|
||||||
for it. The heading this looks for is:
|
for it. The file this looks for is:
|
||||||
|
|
||||||
## Release drill — $ver — DATE
|
$record
|
||||||
|
|
||||||
...with at least one non-blank line under it. Either the section is absent
|
...with something written in it. Either the file is absent entirely, or it
|
||||||
entirely, or it is present and empty; both mean the same thing, which is
|
is present and blank; both mean the same thing, which is that this release
|
||||||
that this release is asserting a ritual it has left no evidence of.
|
is asserting a ritual it has left no evidence of.
|
||||||
|
|
||||||
The unblock is to RUN THE DRILL (drill/drill.sh, on real hardware) and
|
The unblock is to RUN THE DRILL (drill/drill.sh, on real hardware) and
|
||||||
record it in $runs under that heading — what it measured, what it found,
|
record it at that path — what it measured, what it found, what it cost.
|
||||||
what it cost. CI cannot run the drill for you; it can only refuse a release
|
See $drills/README.md for what a record should contain. CI cannot run the
|
||||||
that never ran one.
|
drill for you; it can only refuse a release that never ran one.
|
||||||
|
|
||||||
|
(drill/RUNS.md is a different artifact — the harness's own run log and
|
||||||
|
traps. Appending there does not satisfy this guard, and is not meant to.)
|
||||||
|
|
||||||
If this release must ship without a full drill, that is a maintainer's call
|
If this release must ship without a full drill, that is a maintainer's call
|
||||||
to make and it is still recorded: write the section under the same heading
|
to make and it is still recorded: create the same file and say plainly that
|
||||||
and say plainly that the drill was WAIVED and why. The guard requires a
|
the drill was WAIVED and why. The guard requires a record, not a passing
|
||||||
record, not a passing result — so a skip is a visible, reviewable line in
|
result — so a skip is a visible, reviewable file in the diff rather than
|
||||||
the diff rather than the silent gap that let #95, #114 and #148 all ship
|
the silent gap that let #95, #114 and #148 all ship unproven. See
|
||||||
unproven. See CONTRIBUTING.md, "Releases".
|
CONTRIBUTING.md, "Releases".
|
||||||
EOF
|
EOF
|
||||||
exit 1
|
exit 1
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo "drill-recorded: VERSION '$ver' has a drill record in $runs"
|
echo "drill-recorded: VERSION '$ver' has a drill record at $record"
|
||||||
|
|
|
||||||
|
|
@ -17,7 +17,7 @@ which records not just what changed but what each drill run proved.
|
||||||
- Every fresh mint marks a `pristine` snapshot, before rig converges anything
|
- Every fresh mint marks a `pristine` snapshot, before rig converges anything
|
||||||
(#104, heavy-duty/rig#62)
|
(#104, heavy-duty/rig#62)
|
||||||
- A mint that converges a tenant role marks a `bootstrapped` snapshot (#130)
|
- A mint that converges a tenant role marks a `bootstrapped` snapshot (#130)
|
||||||
- CI refuses a release PR with no drill record in `drill/RUNS.md`
|
- CI refuses a release PR with no drill record at `drills/<version>.md`
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -163,39 +163,70 @@ A release is a PR, and merging it ships it
|
||||||
real hardware and the better part of an hour, so it runs on the release
|
real hardware and the better part of an hour, so it runs on the release
|
||||||
PR's branch and nowhere else.
|
PR's branch and nowhere else.
|
||||||
|
|
||||||
Record it in [drill/RUNS.md](drill/RUNS.md) under its own heading:
|
Record it in its own file, named exactly for the version:
|
||||||
|
|
||||||
```markdown
|
```
|
||||||
## Release drill — X.Y.Z — DATE
|
drills/X.Y.Z.md
|
||||||
```
|
```
|
||||||
|
|
||||||
...with the run under it — what it measured, what it found, what it cost.
|
...with the run in it — what it measured, what it found, what it cost.
|
||||||
|
See [drills/README.md](drills/README.md) for the shape.
|
||||||
[.github/scripts/drill-recorded.sh](.github/scripts/drill-recorded.sh)
|
[.github/scripts/drill-recorded.sh](.github/scripts/drill-recorded.sh)
|
||||||
asserts exactly that on every tree with a bare `VERSION`, which is every
|
asserts exactly that on every tree with a bare `VERSION`, which is every
|
||||||
`release` PR and the merge that publishes it; a `-dev` tree passes with
|
`release` PR and the merge that publishes it; a `-dev` tree passes with
|
||||||
nothing to assert. It is **no longer a thing a reviewer has to remember**.
|
nothing to assert. It is **no longer a thing a reviewer has to remember**.
|
||||||
|
|
||||||
|
**`drills/` is release evidence; [`drill/RUNS.md`](drill/RUNS.md) is the
|
||||||
|
harness's own log.** Two artifacts, two purposes. `drill/RUNS.md` keeps
|
||||||
|
being what it has always been — every run of the harness, the traps table,
|
||||||
|
the lore — and is not release-scoped. `drills/X.Y.Z.md` is the per-release
|
||||||
|
record the gate reads. Appending to `drill/RUNS.md` does not satisfy the
|
||||||
|
gate, and is not meant to.
|
||||||
|
|
||||||
|
One file per version is also why the guard is short. When records shared a
|
||||||
|
file it had to match em-dash heading fields, tolerate an optional date
|
||||||
|
tail, and compare versions *whole* so `0.9.0-rc1` could not satisfy
|
||||||
|
`0.9.0` — and two separate defects were found in review because of that
|
||||||
|
complexity. Now `0.9.0.md` and `0.9.0-rc1.md` are simply different files
|
||||||
|
and none of it is representable.
|
||||||
|
|
||||||
It became CI's job because remembering did not work. The sentence this
|
It became CI's job because remembering did not work. The sentence this
|
||||||
paragraph replaces described a step **no release had ever performed** —
|
paragraph replaces described a step **no release had ever performed** —
|
||||||
`drill/RUNS.md` carried no `## Release drill` section at all — and #95,
|
there was no drill record anywhere in the repo — and #95, #114 and #148
|
||||||
#114 and #148 all shipped through the gap as a `VERSION` bump plus a
|
all shipped through the gap as a `VERSION` bump plus a `CHANGELOG.md`
|
||||||
`CHANGELOG.md` stamp. The one time it was caught was the one time somebody
|
stamp. The one time it was caught was the one time somebody happened to
|
||||||
happened to look, which is not a gate.
|
look, which is not a gate.
|
||||||
|
|
||||||
**The drill is ONE orchestrated run over the whole stack**, not three
|
**The three drills are INDEPENDENT — there is no fixed order.** box, rig
|
||||||
drills in a queue. box and rig are **mutually recursive**, so there is no
|
and cast each drill separately, in any order, on any schedule, in separate
|
||||||
linear order to put them in: rig sits *below* box as the host-builder
|
sittings if you like. What makes that safe is not sequencing, it is that
|
||||||
|
every drill **pins the same fixed set of candidate refs**: rig's drill
|
||||||
|
runs `--host yes` with `BOX_REF=release/<box-version>`, so it exercises the
|
||||||
|
box that will actually ship; box's drill mints with
|
||||||
|
`RIG_REF=release/<rig-version>`, so it exercises the rig that will actually
|
||||||
|
ship. Both measure the same pair.
|
||||||
|
|
||||||
|
That is what dissolves the box↔rig recursion. box and rig really are
|
||||||
|
**mutually recursive** — rig sits *below* box as the host-builder
|
||||||
(`rig bootstrap --host yes` installs box and runs `setup-host`) and
|
(`rig bootstrap --host yes` installs box and runs `setup-host`) and
|
||||||
*above* it as the guest-converger (a `box new` seed's cloud-init curls
|
*above* it as the guest-converger (a `box new` seed's cloud-init curls
|
||||||
rig's installer and runs `rig bootstrap <tenant>-box`). box's own source
|
rig's installer and runs `rig bootstrap <tenant>-box`); box's own source
|
||||||
says as much — the seeds "invert the rig→box install edge (rig#28: rig
|
says as much, the seeds "invert the rig→box install edge (rig#28: rig
|
||||||
installs box on hosts; now box guests install rig)". The run goes:
|
installs box on hosts; now box guests install rig)". A cycle at *runtime*
|
||||||
|
becomes two independent tests against one fixed pair, because the refs are
|
||||||
|
**static identifiers that exist as soon as the release branches do**, long
|
||||||
|
before any drill runs. Nothing has to be released, or drilled, first.
|
||||||
|
|
||||||
1. `rig bootstrap --host yes` on a bare Debian host — installs box, runs
|
Within a single drill you of course bring the substrate up before probing
|
||||||
`setup-host`
|
it — a host before a guest. That is how you run a drill; it is not an
|
||||||
2. `box new` mints a creds-free seed
|
ordering rule between repos.
|
||||||
3. the seed converges itself via `rig bootstrap <tenant>-box`
|
|
||||||
4. cast on top
|
Each repo drills in a **different way** and asserts a different thing: box
|
||||||
|
asserts the **isolation contract** (the 85-probe VM trust boundary), rig
|
||||||
|
asserts **convergence** (a machine reaches its role, idempotently), cast
|
||||||
|
asserts **promotion** (A→B reproduces, the diff is idempotent). Three
|
||||||
|
different exercises sharing a substrate — not three phases of one script,
|
||||||
|
which is exactly why the records are per-repo.
|
||||||
|
|
||||||
It drills **candidate refs, not released artifacts.** `RIG_REPO` and
|
It drills **candidate refs, not released artifacts.** `RIG_REPO` and
|
||||||
`RIG_REF` are mint-time environment variables (defaults
|
`RIG_REF` are mint-time environment variables (defaults
|
||||||
|
|
@ -206,11 +237,15 @@ A release is a PR, and merging it ships it
|
||||||
`CHANGELOG.md` and nothing else, so no executable byte differs between
|
`CHANGELOG.md` and nothing else, so no executable byte differs between
|
||||||
the tree that was drilled and the tree that ships.
|
the tree that was drilled and the tree that ships.
|
||||||
|
|
||||||
One run emits **one shared run ID**. Each repo records its own legs under
|
The drills of a release set share **one run ID**. Each repo records its own
|
||||||
its own `## Release drill — X.Y.Z — DATE`, citing that run ID and the
|
legs in its own `drills/X.Y.Z.md`, citing that run ID and the other two
|
||||||
other two repos' commit SHAs, so the three records reconcile into a single
|
repos' commit SHAs, so the three records reconcile afterwards. The guard
|
||||||
run afterwards. The guard reads only this repo's file — it asserts box's
|
reads only this repo's file — it asserts box's record exists, never the
|
||||||
record exists, never the other two.
|
other two.
|
||||||
|
|
||||||
|
If a defect shows up only in the combination: **patch, re-drill,
|
||||||
|
re-record.** The three releases converge on a set that holds together;
|
||||||
|
they are not required to be right in one pass.
|
||||||
|
|
||||||
**A known gap, and box is where it belongs.** A *released* box still
|
**A known gap, and box is where it belongs.** A *released* box still
|
||||||
defaults `RIG_REF` to `main`, so what a user mints a week after a drill is
|
defaults `RIG_REF` to `main`, so what a user mints a week after a drill is
|
||||||
|
|
@ -224,10 +259,10 @@ A release is a PR, and merging it ships it
|
||||||
|
|
||||||
A maintainer **waiver** is possible, and it is still written down. The
|
A maintainer **waiver** is possible, and it is still written down. The
|
||||||
guard requires a *record*, not a passing result, so a release that must
|
guard requires a *record*, not a passing result, so a release that must
|
||||||
ship without a full drill puts the section under the same heading and says
|
ship without a full drill creates `drills/X.Y.Z.md` anyway and says plainly
|
||||||
plainly that the drill was waived and why. Skipping then costs a
|
that the drill was waived and why. Skipping then costs a deliberate,
|
||||||
deliberate, reviewable line in the diff — which is precisely what the
|
reviewable file in the diff — which is precisely what the three silent
|
||||||
three silent skips above did not.
|
skips above did not.
|
||||||
2. **The maintainer's merge IS the release.**
|
2. **The maintainer's merge IS the release.**
|
||||||
[release.yml](.github/workflows/release.yml) fires on the merged,
|
[release.yml](.github/workflows/release.yml) fires on the merged,
|
||||||
`release`-labeled PR and asserts before creating anything: `VERSION` at
|
`release`-labeled PR and asserts before creating anything: `VERSION` at
|
||||||
|
|
|
||||||
|
|
@ -36,7 +36,7 @@ PR carries as many as apply.
|
||||||
| `blocker:conflict` | `#B60205` | GitHub says `CONFLICTING` — the agent owes a **rebase** | it merges cleanly |
|
| `blocker:conflict` | `#B60205` | GitHub says `CONFLICTING` — the agent owes a **rebase** | it merges cleanly |
|
||||||
| `blocker:ci-red` | `#B60205` | a check failed — the agent owes a **fix**, which a rebase will not provide | checks are green |
|
| `blocker:ci-red` | `#B60205` | a check failed — the agent owes a **fix**, which a rebase will not provide | checks are green |
|
||||||
| `blocker:unrequested` | `#E99695` | this head has no verdict from somebody — never reviewed, or staled by a push — and **nobody was asked** for one | reviews are requested |
|
| `blocker:unrequested` | `#E99695` | this head has no verdict from somebody — never reviewed, or staled by a push — and **nobody was asked** for one | reviews are requested |
|
||||||
| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record in [drill/RUNS.md](drill/RUNS.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) under `## Release drill — X.Y.Z` |
|
| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no drill record at [`drills/X.Y.Z.md`](drills/README.md) — the ceremony is **correct but unevidenced** | the drill is run and recorded (or a waiver is recorded) at `drills/X.Y.Z.md` |
|
||||||
|
|
||||||
`blocker:drill-pending` is the one blocker that says the diff is *fine*. The
|
`blocker:drill-pending` is the one blocker that says the diff is *fine*. The
|
||||||
version is stamped, the changelog is right, CI's other guards are green — what
|
version is stamped, the changelog is right, CI's other guards are green — what
|
||||||
|
|
|
||||||
82
drills/README.md
Normal file
82
drills/README.md
Normal file
|
|
@ -0,0 +1,82 @@
|
||||||
|
# drills/ — release evidence, one file per version
|
||||||
|
|
||||||
|
This directory holds the **evidence that a release was proven on real
|
||||||
|
hardware**. One file per shipped version, named exactly for the version:
|
||||||
|
|
||||||
|
```
|
||||||
|
drills/0.9.0.md
|
||||||
|
drills/0.9.0-rc1.md
|
||||||
|
```
|
||||||
|
|
||||||
|
The name must match the contents of `VERSION` exactly.
|
||||||
|
[.github/scripts/drill-recorded.sh](../.github/scripts/drill-recorded.sh)
|
||||||
|
refuses any tree with a bare `VERSION` that has no such file, or whose file is
|
||||||
|
blank. A `-dev` tree passes with nothing to assert.
|
||||||
|
|
||||||
|
Because each version owns a file, `0.9.0` and `0.9.0-rc1` can never be
|
||||||
|
confused for one another — they are simply different paths. That used to take
|
||||||
|
careful whole-version field matching inside one shared file; now it is free.
|
||||||
|
|
||||||
|
## This is not `drill/RUNS.md`
|
||||||
|
|
||||||
|
Two different artifacts, and the distinction is load-bearing:
|
||||||
|
|
||||||
|
| | what it is |
|
||||||
|
|---|---|
|
||||||
|
| [`drill/RUNS.md`](../drill/RUNS.md) | the **harness's own history** — every run of `drill/drill.sh`, the traps table, the lore about what broke and why. It is not release-scoped and it is not going anywhere. |
|
||||||
|
| `drills/<version>.md` | **release evidence** — the record that *this version* was drilled before it shipped. Release-scoped, one file, gated by CI. |
|
||||||
|
|
||||||
|
Appending to `drill/RUNS.md` does not satisfy the release gate, and is not
|
||||||
|
meant to. Keep using it for what it has always been for.
|
||||||
|
|
||||||
|
## What a record should contain
|
||||||
|
|
||||||
|
- **What ran** — which drill, how many probes, `drill/drill.sh` invocation.
|
||||||
|
- **On what host** — the machine, the OS, the Incus version. "Real hardware"
|
||||||
|
is the claim; name the hardware.
|
||||||
|
- **The pinned candidate refs** — the exact `BOX_REF` / `RIG_REF` /
|
||||||
|
`CAST_REF` under test, and the other repos' commit SHAs. A drill that does
|
||||||
|
not say what it drilled proves nothing later.
|
||||||
|
- **The shared run ID**, so this record reconciles with the sibling repos'.
|
||||||
|
- **The numbers** — passed, failed, how long it took.
|
||||||
|
- **What failed**, plainly.
|
||||||
|
|
||||||
|
**A failed drill is still a valid record.** The gate wants *evidence*, not
|
||||||
|
success. A record saying "83/85, criterion (m) regressed, here is the issue"
|
||||||
|
is a good record. So is a maintainer's written waiver explaining why this
|
||||||
|
release shipped without a full drill. What the gate refuses is silence — #95,
|
||||||
|
#114 and #148 all shipped unproven because a skip left no trace.
|
||||||
|
|
||||||
|
## Worked example
|
||||||
|
|
||||||
|
The version below is a **placeholder that can never be a real release**.
|
||||||
|
Copy the shape, not the number.
|
||||||
|
|
||||||
|
```markdown
|
||||||
|
# Release drill — 9.9.9
|
||||||
|
|
||||||
|
- **Run ID:** `drill-9.9.9-20260721-01` (shared with rig, cast)
|
||||||
|
- **Host:** bare Debian 13, Ryzen 7 5800X / 64 GB, Incus 6.0.2
|
||||||
|
- **Date:** 2026-07-21
|
||||||
|
- **Candidate refs:**
|
||||||
|
- box `release/9.9.9` @ `abc1234`
|
||||||
|
- rig `release/4.4.4` @ `def5678` (minted with `RIG_REF=release/4.4.4`)
|
||||||
|
- cast `release/2.2.2` @ `9abcdef`
|
||||||
|
|
||||||
|
## What ran
|
||||||
|
|
||||||
|
`bash drill/drill.sh --ref release/9.9.9` — the full end-to-end: install the
|
||||||
|
stack, mint every template cold, snapshot and restore, uninstall to zero
|
||||||
|
residue. Then `drill/multiuser.sh` for the two-user grant matrix.
|
||||||
|
|
||||||
|
## Result
|
||||||
|
|
||||||
|
**84/85 passed, 1 failed.** 41 minutes wall clock.
|
||||||
|
|
||||||
|
- Failed: `multiuser.sh` criterion (m) — the raw instance kept a stale route
|
||||||
|
after teardown. Filed as #999. Judged not release-blocking: it affects
|
||||||
|
teardown residue on a host that is about to be wiped, not the trust
|
||||||
|
boundary itself.
|
||||||
|
- The VM boundary probes (the 85-probe isolation contract) passed clean,
|
||||||
|
which is the assertion this repo's drill exists to make.
|
||||||
|
```
|
||||||
218
test/release.sh
218
test/release.sh
|
|
@ -279,7 +279,7 @@ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \
|
||||||
grep -qF 'changelog-armed.sh' "$ROOT/CONTRIBUTING.md"
|
grep -qF 'changelog-armed.sh' "$ROOT/CONTRIBUTING.md"
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# drill-recorded.sh — a RELEASE tree has a drill record in drill/RUNS.md.
|
# drill-recorded.sh — a RELEASE tree has a drill record at drills/<ver>.md.
|
||||||
#
|
#
|
||||||
# The gap this closes is not a drift or a race; it is a rule that was only
|
# The gap this closes is not a drift or a race; it is a rule that was only
|
||||||
# ever a sentence. CONTRIBUTING.md has said since #96 that the release PR is
|
# ever a sentence. CONTRIBUTING.md has said since #96 that the release PR is
|
||||||
|
|
@ -292,108 +292,169 @@ check "CONTRIBUTING: ...and names the guard that enforces it" 0 "" \
|
||||||
# that fired on every tree would be red on every ordinary PR in the repo and
|
# that fired on every tree would be red on every ordinary PR in the repo and
|
||||||
# would be switched off inside a day.
|
# would be switched off inside a day.
|
||||||
#
|
#
|
||||||
# Every fixture carries its OWN VERSION and its OWN RUNS.md. Reaching for
|
# Every fixture carries its OWN VERSION and its OWN drills/ dir. Reaching for
|
||||||
# $ROOT/VERSION instead is exactly the coupling #146 had to fix: the suite
|
# $ROOT/VERSION instead is exactly the coupling #146 had to fix: the suite
|
||||||
# then passes or fails on whichever state the repo happens to be in, so it
|
# then passes or fails on whichever state the repo happens to be in, so it
|
||||||
# goes red on the ceremony tree — the one tree where the release suite most
|
# goes red on the ceremony tree — the one tree where the release suite most
|
||||||
# needs to be trustworthy.
|
# needs to be trustworthy.
|
||||||
|
#
|
||||||
|
# Records are now ONE FILE PER VERSION, which is why this block is shorter
|
||||||
|
# than the one it replaces. The heading-parsing cases are gone because there
|
||||||
|
# is no heading to parse: em-dash fields, the optional ' — DATE' tail, and
|
||||||
|
# "an empty section must not borrow a neighbour's prose" were all artifacts
|
||||||
|
# of records sharing drill/RUNS.md. Deleting a test is only safe when the
|
||||||
|
# failure it described became unrepresentable, and that is the case here —
|
||||||
|
# except for the whitespace rule, which was never about headings and is
|
||||||
|
# pinned harder below.
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
DRILLED="$ROOT/.github/scripts/drill-recorded.sh"
|
DRILLED="$ROOT/.github/scripts/drill-recorded.sh"
|
||||||
check "drill-recorded: runnable bash" 0 "" bash -n "$DRILLED"
|
check "drill-recorded: runnable bash" 0 "" bash -n "$DRILLED"
|
||||||
check "drill-recorded: executable" 0 "" test -x "$DRILLED"
|
check "drill-recorded: executable" 0 "" test -x "$DRILLED"
|
||||||
|
|
||||||
# dtree <dir> <version> <runs-body...> — a two-file tree, self-contained
|
# dtree <dir> <version> [<record-version> <record-body...>] — a self-contained
|
||||||
|
# tree: its own VERSION, and its own drills/ dir created only when asked for.
|
||||||
|
# With 2 args there is NO drills/ dir at all, which is a case in its own right.
|
||||||
dtree() {
|
dtree() {
|
||||||
local d="$WORK/$1" v="$2"; shift 2
|
local d="$WORK/$1" v="$2"; shift 2
|
||||||
mkdir -p "$d"
|
mkdir -p "$d"
|
||||||
printf '%s\n' "$v" > "$d/VERSION"
|
printf '%s\n' "$v" > "$d/VERSION"
|
||||||
{ echo "# Drill run log"; echo; printf '%s\n' "$@"; } > "$d/RUNS.md"
|
if [ "$#" -gt 0 ]; then
|
||||||
|
local rec="$1"; shift
|
||||||
|
mkdir -p "$d/drills"
|
||||||
|
printf '%s\n' "$@" > "$d/drills/$rec.md"
|
||||||
|
fi
|
||||||
echo "$d"
|
echo "$d"
|
||||||
}
|
}
|
||||||
drilled() { bash "$DRILLED" "$1/RUNS.md" "$1/VERSION"; }
|
drilled() { bash "$DRILLED" "$1/drills" "$1/VERSION"; }
|
||||||
|
|
||||||
# --- the -dev steady state: nothing to assert ------------------------------
|
# --- the -dev steady state: nothing to assert ------------------------------
|
||||||
# No record at all, and that must PASS. This is the case that makes the guard
|
# No record at all, and that must PASS. This is the case that makes the guard
|
||||||
# survivable on an ordinary PR.
|
# survivable on an ordinary PR.
|
||||||
T="$(dtree drill-dev 0.8.1-dev 'No drill sections in here at all.')"
|
T="$(dtree drill-dev 0.8.1-dev)"
|
||||||
check "drill-recorded: a -dev tree with NO record at all passes" 0 "nothing to assert" drilled "$T"
|
check "drill-recorded: a -dev tree with NO drills dir at all passes" 0 "nothing to assert" drilled "$T"
|
||||||
check "drill-recorded: ...and says why it declined to judge" 0 "development tree" drilled "$T"
|
check "drill-recorded: ...and says why it declined to judge" 0 "development tree" drilled "$T"
|
||||||
|
|
||||||
# --- the ceremony tree: a record is required and must carry prose ----------
|
# --- the ceremony tree: a record is required and must not be blank ---------
|
||||||
T="$(dtree drill-ok 0.9.0 \
|
T="$(dtree drill-ok 0.9.0 0.9.0 '# Release drill — 0.9.0' '' '- 47/47 on real hardware')"
|
||||||
'## Release drill — 0.9.0 — 2026-07-21' '' '- 47/47 on real hardware' '' \
|
check "drill-recorded: a bare VERSION with a non-empty record for it passes" \
|
||||||
'## Run history' '' '- older prose')"
|
|
||||||
check "drill-recorded: a bare VERSION with a matching non-empty record passes" \
|
|
||||||
0 "has a drill record" drilled "$T"
|
0 "has a drill record" drilled "$T"
|
||||||
# The date tail is optional; the heading without it is still a record.
|
|
||||||
T="$(dtree drill-nodate 0.9.0 '## Release drill — 0.9.0' '' '- ran it, 47/47')"
|
|
||||||
check "drill-recorded: ...and the ' — DATE' tail is optional" 0 "has a drill record" drilled "$T"
|
|
||||||
|
|
||||||
T="$(dtree drill-none 0.9.0 '## Run history' '' '- prose about earlier runs')"
|
# No drills/ dir AT ALL is the commonest way to fail this: the release branch
|
||||||
check "drill-recorded: a bare VERSION with NO record fails" 1 "no drill record" drilled "$T"
|
# was cut and nobody ran anything. It must fail loudly, not error out on a
|
||||||
|
# missing directory in some way that reads like a broken guard.
|
||||||
|
T="$(dtree drill-nodir 0.9.0)"
|
||||||
|
check "drill-recorded: a bare VERSION with NO drills dir fails" 1 "no drill record" drilled "$T"
|
||||||
check "drill-recorded: ...and the failure names the version" 1 "VERSION is '0.9.0'" drilled "$T"
|
check "drill-recorded: ...and the failure names the version" 1 "VERSION is '0.9.0'" drilled "$T"
|
||||||
|
|
||||||
|
# The dir exists — some other release left its record here — but this version
|
||||||
|
# has none. A directory listing is not evidence about a particular version.
|
||||||
|
T="$(dtree drill-otherver 0.9.0 0.8.0 '# Release drill — 0.8.0' '' '- the previous release')"
|
||||||
|
check "drill-recorded: a drills dir with no file for THIS version fails" \
|
||||||
|
1 "no drill record" drilled "$T"
|
||||||
|
|
||||||
check "drill-recorded: ...and names the unblock — run the drill and record it" \
|
check "drill-recorded: ...and names the unblock — run the drill and record it" \
|
||||||
1 "RUN THE DRILL" drilled "$T"
|
1 "RUN THE DRILL" drilled "$T"
|
||||||
|
check "drill-recorded: ...and names the path it wanted" 1 "drills/0.9.0.md" drilled "$T"
|
||||||
check "drill-recorded: ...and offers the recorded maintainer waiver as the other way out" \
|
check "drill-recorded: ...and offers the recorded maintainer waiver as the other way out" \
|
||||||
1 "WAIVED" drilled "$T"
|
1 "WAIVED" drilled "$T"
|
||||||
|
check "drill-recorded: ...and names the three releases that shipped through the gap" \
|
||||||
|
1 "#95, #114 and #148" drilled "$T"
|
||||||
|
|
||||||
# A heading with nothing under it is the same failure as no heading: the
|
# A file that exists and says nothing is the same failure as no file: the
|
||||||
# ceremony wrote the shape and never wrote the evidence.
|
# ceremony created the shape and never wrote the evidence. `touch` must not
|
||||||
T="$(dtree drill-empty 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' '' '' '## Run history' '' '- older prose')"
|
# be able to satisfy a gate whose entire job is to demand evidence.
|
||||||
check "drill-recorded: a bare VERSION whose record is present but EMPTY fails" \
|
mkdir -p "$WORK/drill-emptyfile/drills"
|
||||||
1 "no drill record" drilled "$T"
|
printf '0.9.0\n' > "$WORK/drill-emptyfile/VERSION"
|
||||||
|
: > "$WORK/drill-emptyfile/drills/0.9.0.md"
|
||||||
|
check "drill-recorded: a record file that is present but EMPTY fails" \
|
||||||
|
1 "no drill record" drilled "$WORK/drill-emptyfile"
|
||||||
|
|
||||||
# ...and WHITESPACE is not evidence either. The first cut extracted with
|
# ...and WHITESPACE is not evidence either. This is the ONE rule carried over
|
||||||
# `sed '/./,$!d'`, where `.` matches a space — so a heading followed by one
|
# from the heading-parsing guard, because it was never a heading problem. The
|
||||||
# tab passed the guard while the failure text promised "at least one non-blank
|
# first cut extracted with `sed '/./,$!d'`, where `.` matches a space — so a
|
||||||
# line". An evidence-free release for the price of an invisible character, on
|
# record whose body was one tab passed a guard that promised "at least one
|
||||||
# the one check whose entire job is to demand evidence. All three reviewers on
|
# non-blank line". An evidence-free release for the price of an invisible
|
||||||
# #149 found it independently, which is the level of scrutiny it deserved and
|
# character. All three reviewers on #149 found it independently, which is the
|
||||||
# not a level it should ever need again.
|
# level of scrutiny it deserved and not a level it should ever need again.
|
||||||
T="$(dtree drill-blank 0.9.0 '## Release drill — 0.9.0 — 2026-07-21' ' ' ' ' '' '## Run history' '' '- older prose')"
|
# Splitting the file per version removed the parsing; it did not remove this.
|
||||||
check "drill-recorded: a record body of only spaces and tabs fails (#149)" \
|
mkdir -p "$WORK/drill-blank/drills"
|
||||||
1 "no drill record" drilled "$T"
|
printf '0.9.0\n' > "$WORK/drill-blank/VERSION"
|
||||||
|
printf ' \n\t\n\n' > "$WORK/drill-blank/drills/0.9.0.md"
|
||||||
|
check "drill-recorded: a record of only spaces, tabs and newlines fails (#149)" \
|
||||||
|
1 "no drill record" drilled "$WORK/drill-blank"
|
||||||
|
|
||||||
# --- the version is matched WHOLE, both directions -------------------------
|
# --- the version is matched WHOLE, both directions -------------------------
|
||||||
# release-notes.sh's trap, in a new file: a substring match would let an rc
|
# Under the old scheme this took deliberate field-matching against a heading,
|
||||||
# drill stand in for the release it was a candidate for, which is the one
|
# and getting it wrong would have let a release-candidate drill stand in for
|
||||||
# thing a release-candidate drill by definition did not measure.
|
# the release it was a candidate for — the one thing an rc drill by
|
||||||
T="$(dtree drill-rc-only 0.9.0 '## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill')"
|
# definition did not measure. One file per version makes it free: these are
|
||||||
check "drill-recorded: a 0.9.0-rc1 record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T"
|
# simply different paths. Pinned anyway, in both directions, so that a future
|
||||||
T="$(dtree drill-rel-only 0.9.0-rc1 '## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')"
|
# rewrite reaching for a prefix or glob match cannot quietly reintroduce it.
|
||||||
check "drill-recorded: ...and a 0.9.0 record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T"
|
T="$(dtree drill-rc-only 0.9.0 0.9.0-rc1 '# Release drill — 0.9.0-rc1' '' '- the rc drill')"
|
||||||
# ...and with both present, each resolves to its own section and neither
|
check "drill-recorded: a 0.9.0-rc1.md record does NOT satisfy 0.9.0" 1 "no drill record" drilled "$T"
|
||||||
# borrows the other's prose.
|
T="$(dtree drill-rel-only 0.9.0-rc1 0.9.0 '# Release drill — 0.9.0' '' '- the release drill')"
|
||||||
T="$(dtree drill-both 0.9.0 \
|
check "drill-recorded: ...and a 0.9.0.md record does NOT satisfy 0.9.0-rc1" 1 "no drill record" drilled "$T"
|
||||||
'## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \
|
|
||||||
'## Release drill — 0.9.0 — 2026-07-21' '' '- the release drill')"
|
|
||||||
check "drill-recorded: with both present, the release resolves to its own" \
|
|
||||||
0 "has a drill record" drilled "$T"
|
|
||||||
# The sharp version of the same claim: the rc section is full, the release's
|
|
||||||
# own section is EMPTY. If emptiness were judged over anything but this
|
|
||||||
# version's own lines, this tree would pass on the rc's prose.
|
|
||||||
T="$(dtree drill-borrow 0.9.0 \
|
|
||||||
'## Release drill — 0.9.0-rc1 — 2026-07-20' '' '- the rc drill' '' \
|
|
||||||
'## Release drill — 0.9.0 — 2026-07-21' '')"
|
|
||||||
check "drill-recorded: ...and an empty release section cannot borrow the rc's prose" \
|
|
||||||
1 "no drill record" drilled "$T"
|
|
||||||
|
|
||||||
# --- degenerate trees refuse rather than pass by accident ------------------
|
# --- degenerate trees refuse rather than pass by accident ------------------
|
||||||
|
# A guard that cannot read the version cannot know whether this tree is its
|
||||||
|
# business, and "could not tell" must never resolve to "allowed".
|
||||||
check "drill-recorded: a missing VERSION refuses by path" 1 "no such file" \
|
check "drill-recorded: a missing VERSION refuses by path" 1 "no such file" \
|
||||||
bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/nope-version"
|
bash "$DRILLED" "$WORK/drill-ok/drills" "$WORK/nope-version"
|
||||||
mkdir -p "$WORK/drill-empty-ver"; : > "$WORK/drill-empty-ver/VERSION"
|
mkdir -p "$WORK/drill-empty-ver"; : > "$WORK/drill-empty-ver/VERSION"
|
||||||
check "drill-recorded: an empty VERSION refuses" 1 "is empty" \
|
check "drill-recorded: an empty VERSION refuses" 1 "is empty" \
|
||||||
bash "$DRILLED" "$WORK/drill-ok/RUNS.md" "$WORK/drill-empty-ver/VERSION"
|
bash "$DRILLED" "$WORK/drill-ok/drills" "$WORK/drill-empty-ver/VERSION"
|
||||||
# The runs file is only needed once the tree is a release — a -dev tree must
|
# The drills dir is only consulted once the tree is a release — a -dev tree
|
||||||
# not be refused for a file it has no business reading.
|
# must not be refused for a directory it has no business reading.
|
||||||
check "drill-recorded: a release tree with a missing runs file refuses by path" \
|
check "drill-recorded: a -dev tree does not even look for the drills dir" 0 "nothing to assert" \
|
||||||
1 "no such file" bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-ok/VERSION"
|
bash "$DRILLED" "$WORK/nope-drills" "$WORK/drill-dev/VERSION"
|
||||||
check "drill-recorded: ...but a -dev tree does not even look for it" 0 "nothing to assert" \
|
|
||||||
bash "$DRILLED" "$WORK/nope-runs.md" "$WORK/drill-dev/VERSION"
|
|
||||||
|
|
||||||
# --- and the tree under test ----------------------------------------------
|
# --- and the tree under test ----------------------------------------------
|
||||||
check "drill-recorded: THIS tree passes its own guard" 0 "" \
|
# The property is that the guard's VERDICT IS CORRECT FOR THIS TREE — not that
|
||||||
bash "$DRILLED" "$ROOT/drill/RUNS.md" "$ROOT/VERSION"
|
# it always passes. Those come apart on a ceremony tree, and demanding success
|
||||||
|
# is wrong there in a way that took a red release PR to notice.
|
||||||
|
#
|
||||||
|
# A -dev tree is vacuous, so it must pass. A ceremony tree passes only once a
|
||||||
|
# human has run the drill and written the record — which is the entire point of
|
||||||
|
# the gate. Asserting exit 0 unconditionally therefore made test/release.sh
|
||||||
|
# UN-GREENABLE on every release branch before its drill, and reported it as a
|
||||||
|
# `release-flow tests` failure rather than as the gate doing its job: the same
|
||||||
|
# noise-and-misattribution shape as #146, where a fixture read the repo's real
|
||||||
|
# VERSION and only misbehaved on the ceremony tree.
|
||||||
|
#
|
||||||
|
# So: assert the verdict that this tree's own state entails. Green in all three
|
||||||
|
# states — dev, ceremony-without-record, ceremony-with-record — and still red if
|
||||||
|
# the guard ever disagrees with the tree in front of it.
|
||||||
|
THIS_VER="$(tr -d '[:space:]' < "$ROOT/VERSION")"
|
||||||
|
case "$THIS_VER" in
|
||||||
|
*-dev)
|
||||||
|
check "drill-recorded: THIS tree is -dev, and the guard is vacuous on it" 0 "nothing to assert" \
|
||||||
|
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION" ;;
|
||||||
|
*)
|
||||||
|
if [ -s "$ROOT/drills/$THIS_VER.md" ]; then
|
||||||
|
check "drill-recorded: THIS ceremony tree HAS its record, and the guard accepts it" 0 "" \
|
||||||
|
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION"
|
||||||
|
else
|
||||||
|
check "drill-recorded: THIS ceremony tree has NO record yet, and the guard refuses it" 1 "no drill record" \
|
||||||
|
bash "$DRILLED" "$ROOT/drills" "$ROOT/VERSION"
|
||||||
|
fi ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
# drills/ is release evidence and drill/RUNS.md is the harness's own log. The
|
||||||
|
# rewrite is only coherent if both keep existing and the docs say which is
|
||||||
|
# which; a reader who appends to the wrong one gets a red gate and no clue.
|
||||||
|
check "drills/: the directory documents itself" 0 "" test -f "$ROOT/drills/README.md"
|
||||||
|
check "drills/: ...and is plain, not a dot-directory invisible to globs (#116)" 1 "" \
|
||||||
|
test -d "$ROOT/.drills"
|
||||||
|
check "drills/: ...and distinguishes itself from the harness log" 0 "" \
|
||||||
|
grep -qF 'drill/RUNS.md' "$ROOT/drills/README.md"
|
||||||
|
check "drills/: ...and its worked example uses a version that can never ship" 0 "" \
|
||||||
|
grep -qF '9.9.9' "$ROOT/drills/README.md"
|
||||||
|
check "drill/RUNS.md: the harness log is untouched and still present" 0 "" \
|
||||||
|
test -f "$ROOT/drill/RUNS.md"
|
||||||
|
# No real record may be invented to make the suite green. The repo is
|
||||||
|
# 0.8.1-dev; fabricating drills/0.8.1.md would defeat the whole gate.
|
||||||
|
check "drills/: no fabricated record for an unshipped version" 1 "" \
|
||||||
|
test -f "$ROOT/drills/0.8.1.md"
|
||||||
|
|
||||||
# The guard is only a guard if CI runs it, and only a rule if the document
|
# The guard is only a guard if CI runs it, and only a rule if the document
|
||||||
# that used to carry the rule now points at it.
|
# that used to carry the rule now points at it.
|
||||||
|
|
@ -401,21 +462,38 @@ check "ci.yml: runs the drill-recorded guard" 0 "" \
|
||||||
grep -qF 'drill-recorded.sh' "$ROOT/.github/workflows/ci.yml"
|
grep -qF 'drill-recorded.sh' "$ROOT/.github/workflows/ci.yml"
|
||||||
check "CONTRIBUTING: names the guard that enforces the drill" 0 "" \
|
check "CONTRIBUTING: names the guard that enforces the drill" 0 "" \
|
||||||
grep -qF 'drill-recorded.sh' "$ROOT/CONTRIBUTING.md"
|
grep -qF 'drill-recorded.sh' "$ROOT/CONTRIBUTING.md"
|
||||||
check "CONTRIBUTING: documents the drill record's heading format" 0 "" \
|
check "CONTRIBUTING: documents the drill record's path" 0 "" \
|
||||||
grep -qF '## Release drill — X.Y.Z — DATE' "$ROOT/CONTRIBUTING.md"
|
grep -qF 'drills/X.Y.Z.md' "$ROOT/CONTRIBUTING.md"
|
||||||
check "CONTRIBUTING: the drill is ONE run over the whole stack, not a queue" 0 "" \
|
check "CONTRIBUTING: ...and distinguishes it from the harness's own log" 0 "" \
|
||||||
|
grep -qF "harness's own log" "$ROOT/CONTRIBUTING.md"
|
||||||
|
# The three drills are INDEPENDENT. An earlier draft of this section said the
|
||||||
|
# drill was "ONE orchestrated run over the whole stack", which over-constrains
|
||||||
|
# it into a fixed sequence the repos cannot satisfy. Pinning the corrected
|
||||||
|
# claim, and negatively pinning the fixed-order language, because "surely you
|
||||||
|
# drill rig before box" is exactly the simplification a future editor makes.
|
||||||
|
check "CONTRIBUTING: the three drills are independent, in any order" 0 "" \
|
||||||
|
grep -qF 'The three drills are INDEPENDENT' "$ROOT/CONTRIBUTING.md"
|
||||||
|
check "CONTRIBUTING: ...and no fixed-order framing survives" 1 "" \
|
||||||
grep -qF 'ONE orchestrated run' "$ROOT/CONTRIBUTING.md"
|
grep -qF 'ONE orchestrated run' "$ROOT/CONTRIBUTING.md"
|
||||||
# The load-bearing correction: box and rig are mutually recursive, so a
|
# The load-bearing correction: box and rig are mutually recursive, so a
|
||||||
# "release A before you can drill B" rule is not merely bureaucratic, it is
|
# "release A before you can drill B" rule is not merely bureaucratic, it is
|
||||||
# unsatisfiable in principle. Pinning the mechanism that dissolves it —
|
# unsatisfiable in principle. What dissolves it is that every drill pins the
|
||||||
# mint-time candidate refs — because that is the claim a future editor is
|
# same fixed candidate refs — static identifiers that exist as soon as the
|
||||||
# most likely to "simplify" back into a linear order.
|
# release branches do — turning a runtime cycle into two independent tests.
|
||||||
check "CONTRIBUTING: ...drilling candidate refs, not released artifacts" 0 "" \
|
check "CONTRIBUTING: ...drilling candidate refs, not released artifacts" 0 "" \
|
||||||
grep -qF 'candidate refs, not released artifacts' "$ROOT/CONTRIBUTING.md"
|
grep -qF 'candidate refs, not released artifacts' "$ROOT/CONTRIBUTING.md"
|
||||||
|
check "CONTRIBUTING: ...and the same fixed refs are what dissolve the recursion" 0 "" \
|
||||||
|
grep -qF 'pins the same fixed set of candidate refs' "$ROOT/CONTRIBUTING.md"
|
||||||
|
check "CONTRIBUTING: ...and each repo asserts a different thing" 0 "" \
|
||||||
|
grep -qF 'isolation contract' "$ROOT/CONTRIBUTING.md"
|
||||||
|
check "CONTRIBUTING: ...and a combination defect means patch, re-drill, re-record" 0 "" \
|
||||||
|
grep -qF 're-drill' "$ROOT/CONTRIBUTING.md"
|
||||||
check "CONTRIBUTING: ...and the RIG_REF gap is stated as outstanding (#81)" 0 "" \
|
check "CONTRIBUTING: ...and the RIG_REF gap is stated as outstanding (#81)" 0 "" \
|
||||||
grep -qF 'RIG_REF' "$ROOT/CONTRIBUTING.md"
|
grep -qF 'RIG_REF' "$ROOT/CONTRIBUTING.md"
|
||||||
check "LABELS: documents blocker:drill-pending" 0 "" \
|
check "LABELS: documents blocker:drill-pending" 0 "" \
|
||||||
grep -qF 'blocker:drill-pending' "$ROOT/LABELS.md"
|
grep -qF 'blocker:drill-pending' "$ROOT/LABELS.md"
|
||||||
|
check "LABELS: ...and points at the per-version record, not the harness log" 0 "" \
|
||||||
|
grep -qF 'drills/X.Y.Z.md' "$ROOT/LABELS.md"
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# changelog-monotonic.sh (#122) — no SHIPPED release heading may be DELETED.
|
# changelog-monotonic.sh (#122) — no SHIPPED release heading may be DELETED.
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue