refactor: one drill record per version, in drills/
Drill records move from sections inside drill/RUNS.md to one file per
version: drills/<version>.md. drill/RUNS.md is untouched — it stays the
harness's own run log, traps table and lore. drills/ is release evidence
only, and the docs now say which is which.
The old guard parsed headings: em-dash field matching, an optional
' — DATE' tail, whole-version comparison so 0.9.0-rc1 could not satisfy
0.9.0, avoiding \x escapes because CI runs mawk not gawk, and a non-blank
body rule. Every one of those existed only because records shared one
file, and two separate defects were found in review because of that
complexity — the sed '/./,$!d' whitespace bypass, and heading-grammar
drift from the sibling repos.
One file per version makes almost all of it unrepresentable: 0.9.0.md and
0.9.0-rc1.md are simply different files, so whole-version matching is
free rather than a trap. The guard keeps only the rule that was never
about headings — a record of pure whitespace is not a record — and keeps
the failure message that names the version, the path, the unblock, and
the recorded-waiver escape hatch.
Plain drills/, not .drills/ — a dot-directory is invisible to globs
without dotglob, which is what caused #116 and #118.
Also corrects the release-drill framing: the three repos' drills are
INDEPENDENT and run in any order. What dissolves the box<->rig recursion
is that every drill pins the same fixed candidate refs, not sequencing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 16:26:47 +00:00
# drills/ — release evidence, one file per version
This directory holds the **evidence that a release was proven on real
hardware**. One file per shipped version, named exactly for the version:
```
drills/0.9.0.md
drills/0.9.0-rc1.md
```
The name must match the contents of `VERSION` exactly.
2026-07-23 13:45:08 +00:00
[the pinned ceremony drill-recorded action ](https://github.com/heavy-duty/ceremony/tree/0.1.0/actions/drill-recorded )
refactor: one drill record per version, in drills/
Drill records move from sections inside drill/RUNS.md to one file per
version: drills/<version>.md. drill/RUNS.md is untouched — it stays the
harness's own run log, traps table and lore. drills/ is release evidence
only, and the docs now say which is which.
The old guard parsed headings: em-dash field matching, an optional
' — DATE' tail, whole-version comparison so 0.9.0-rc1 could not satisfy
0.9.0, avoiding \x escapes because CI runs mawk not gawk, and a non-blank
body rule. Every one of those existed only because records shared one
file, and two separate defects were found in review because of that
complexity — the sed '/./,$!d' whitespace bypass, and heading-grammar
drift from the sibling repos.
One file per version makes almost all of it unrepresentable: 0.9.0.md and
0.9.0-rc1.md are simply different files, so whole-version matching is
free rather than a trap. The guard keeps only the rule that was never
about headings — a record of pure whitespace is not a record — and keeps
the failure message that names the version, the path, the unblock, and
the recorded-waiver escape hatch.
Plain drills/, not .drills/ — a dot-directory is invisible to globs
without dotglob, which is what caused #116 and #118.
Also corrects the release-drill framing: the three repos' drills are
INDEPENDENT and run in any order. What dissolves the box<->rig recursion
is that every drill pins the same fixed candidate refs, not sequencing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 16:26:47 +00:00
refuses any tree with a bare `VERSION` that has no such file, or whose file is
blank. A `-dev` tree passes with nothing to assert.
Because each version owns a file, `0.9.0` and `0.9.0-rc1` can never be
confused for one another — they are simply different paths. That used to take
careful whole-version field matching inside one shared file; now it is free.
## This is not `drill/RUNS.md`
Two different artifacts, and the distinction is load-bearing:
| | what it is |
|---|---|
| [`drill/RUNS.md` ](../drill/RUNS.md ) | the **harness's own history** — every run of `drill/drill.sh` , the traps table, the lore about what broke and why. It is not release-scoped and it is not going anywhere. |
| `drills/<version>.md` | **release evidence** — the record that *this version* was drilled before it shipped. Release-scoped, one file, gated by CI. |
Appending to `drill/RUNS.md` does not satisfy the release gate, and is not
meant to. Keep using it for what it has always been for.
## What a record should contain
- **What ran** — which drill, how many probes, `drill/drill.sh` invocation.
- **On what host** — the machine, the OS, the Incus version. "Real hardware"
is the claim; name the hardware.
- **The pinned candidate refs** — the exact `BOX_REF` / `RIG_REF` /
`CAST_REF` under test, and the other repos' commit SHAs. A drill that does
not say what it drilled proves nothing later.
- **The shared run ID**, so this record reconciles with the sibling repos'.
- **The numbers** — passed, failed, how long it took.
- **What failed**, plainly.
**A failed drill is still a valid record.** The gate wants *evidence* , not
success. A record saying "83/85, criterion (m) regressed, here is the issue"
is a good record. So is a maintainer's written waiver explaining why this
release shipped without a full drill. What the gate refuses is silence — #95 ,
#114 and #148 all shipped unproven because a skip left no trace.
2026-08-18 06:56:56 +00:00
## Generate the factual draft
The harness can write the facts it already knows directly into a new record:
```sh
bash drill/drill.sh --ref release/0.9.1 \
--run-id drill-0.9.1-20260818-01 \
--emit-record drills/0.9.1.md
```
`--run-id` wins over `DRILL_RUN_ID` ; if neither is set, the harness generates
`drill-<version>-<UTC date>-01` . Supply the same ID explicitly to each family
drill so their records reconcile. The emitter refuses to overwrite an existing
file, and its Markdown contains no terminal colour escapes.
2026-08-18 07:04:33 +00:00
When record emission is requested, the harness resolves the requested box and
rig refs before their installers run and feeds those installers the resulting
full commit SHAs. The draft keeps both the human-facing refs and the immutable
SHAs. If either ref cannot be pinned, the drill refuses before mutating the host
and still writes the failed preflight record.
2026-08-18 06:56:56 +00:00
The generated file is a **draft, not finished release evidence** . It records
the host, invocation, elapsed time, findings, and the exact box and rig refs and
SHAs the run selected. Before committing it, verify those facts and add the
operator judgement that a script cannot make: hardware detail, the meaning of
any failures or skips, and whether they block release.
refactor: one drill record per version, in drills/
Drill records move from sections inside drill/RUNS.md to one file per
version: drills/<version>.md. drill/RUNS.md is untouched — it stays the
harness's own run log, traps table and lore. drills/ is release evidence
only, and the docs now say which is which.
The old guard parsed headings: em-dash field matching, an optional
' — DATE' tail, whole-version comparison so 0.9.0-rc1 could not satisfy
0.9.0, avoiding \x escapes because CI runs mawk not gawk, and a non-blank
body rule. Every one of those existed only because records shared one
file, and two separate defects were found in review because of that
complexity — the sed '/./,$!d' whitespace bypass, and heading-grammar
drift from the sibling repos.
One file per version makes almost all of it unrepresentable: 0.9.0.md and
0.9.0-rc1.md are simply different files, so whole-version matching is
free rather than a trap. The guard keeps only the rule that was never
about headings — a record of pure whitespace is not a record — and keeps
the failure message that names the version, the path, the unblock, and
the recorded-waiver escape hatch.
Plain drills/, not .drills/ — a dot-directory is invisible to globs
without dotglob, which is what caused #116 and #118.
Also corrects the release-drill framing: the three repos' drills are
INDEPENDENT and run in any order. What dissolves the box<->rig recursion
is that every drill pins the same fixed candidate refs, not sequencing.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 16:26:47 +00:00
## Worked example
The version below is a **placeholder that can never be a real release** .
Copy the shape, not the number.
```markdown
# Release drill — 9.9.9
- **Run ID:** `drill-9.9.9-20260721-01` (shared with rig, cast)
- **Host:** bare Debian 13, Ryzen 7 5800X / 64 GB, Incus 6.0.2
- **Date:** 2026-07-21
- **Candidate refs:**
- box `release/9.9.9` @ `abc1234`
- rig `release/4.4.4` @ `def5678` (minted with `RIG_REF=release/4.4.4` )
- cast `release/2.2.2` @ `9abcdef`
## What ran
`bash drill/drill.sh --ref release/9.9.9` — the full end-to-end: install the
stack, mint every template cold, snapshot and restore, uninstall to zero
residue. Then `drill/multiuser.sh` for the two-user grant matrix.
## Result
**84/85 passed, 1 failed.** 41 minutes wall clock.
- Failed: `multiuser.sh` criterion (m) — the raw instance kept a stale route
after teardown. Filed as #999 . Judged not release-blocking: it affects
teardown residue on a host that is about to be wiped, not the trust
boundary itself.
- The VM boundary probes (the 85-probe isolation contract) passed clean,
which is the assertion this repo's drill exists to make.
```
