heavy-duty/box
7
0
Fork 1
Code Issues 9 Pull requests Projects Releases 4 Packages Wiki Activity Actions
box/drills
History
dan-claude-bot f63fa952d4 refactor: one drill record per version, in drills/ 
Drill records move from sections inside drill/RUNS.md to one file per
version: drills/<version>.md. drill/RUNS.md is untouched — it stays the
harness's own run log, traps table and lore. drills/ is release evidence
only, and the docs now say which is which.

The old guard parsed headings: em-dash field matching, an optional
' — DATE' tail, whole-version comparison so 0.9.0-rc1 could not satisfy
0.9.0, avoiding \x escapes because CI runs mawk not gawk, and a non-blank
body rule. Every one of those existed only because records shared one
file, and two separate defects were found in review because of that
complexity — the sed '/./,$!d' whitespace bypass, and heading-grammar
drift from the sibling repos.

One file per version makes almost all of it unrepresentable: 0.9.0.md and
0.9.0-rc1.md are simply different files, so whole-version matching is
free rather than a trap. The guard keeps only the rule that was never
about headings — a record of pure whitespace is not a record — and keeps
the failure message that names the version, the path, the unblock, and
the recorded-waiver escape hatch.

Plain drills/, not .drills/ — a dot-directory is invisible to globs
without dotglob, which is what caused #116 and #118.

Also corrects the release-drill framing: the three repos' drills are
INDEPENDENT and run in any order. What dissolves the box<->rig recursion
is that every drill pins the same fixed candidate refs, not sequencing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 16:35:28 +00:00
..
README.md refactor: one drill record per version, in drills/ 2026-07-21 16:35:28 +00:00

README.md

drills/ — release evidence, one file per version

This directory holds the evidence that a release was proven on real hardware. One file per shipped version, named exactly for the version:

drills/0.9.0.md
drills/0.9.0-rc1.md

The name must match the contents of VERSION exactly. .github/scripts/drill-recorded.sh refuses any tree with a bare VERSION that has no such file, or whose file is blank. A -dev tree passes with nothing to assert.

Because each version owns a file, 0.9.0 and 0.9.0-rc1 can never be confused for one another — they are simply different paths. That used to take careful whole-version field matching inside one shared file; now it is free.

This is not drill/RUNS.md

Two different artifacts, and the distinction is load-bearing:

what it is
drill/RUNS.md the harness's own history — every run of drill/drill.sh, the traps table, the lore about what broke and why. It is not release-scoped and it is not going anywhere.
drills/<version>.md release evidence — the record that this version was drilled before it shipped. Release-scoped, one file, gated by CI.

Appending to drill/RUNS.md does not satisfy the release gate, and is not meant to. Keep using it for what it has always been for.

What a record should contain

  • What ran — which drill, how many probes, drill/drill.sh invocation.
  • On what host — the machine, the OS, the Incus version. "Real hardware" is the claim; name the hardware.
  • The pinned candidate refs — the exact BOX_REF / RIG_REF / CAST_REF under test, and the other repos' commit SHAs. A drill that does not say what it drilled proves nothing later.
  • The shared run ID, so this record reconciles with the sibling repos'.
  • The numbers — passed, failed, how long it took.
  • What failed, plainly.

A failed drill is still a valid record. The gate wants evidence, not success. A record saying "83/85, criterion (m) regressed, here is the issue" is a good record. So is a maintainer's written waiver explaining why this release shipped without a full drill. What the gate refuses is silence — #95, #114 and #148 all shipped unproven because a skip left no trace.

Worked example

The version below is a placeholder that can never be a real release. Copy the shape, not the number.

# Release drill — 9.9.9

- **Run ID:** `drill-9.9.9-20260721-01` (shared with rig, cast)
- **Host:** bare Debian 13, Ryzen 7 5800X / 64 GB, Incus 6.0.2
- **Date:** 2026-07-21
- **Candidate refs:**
  - box `release/9.9.9` @ `abc1234`
  - rig `release/4.4.4` @ `def5678` (minted with `RIG_REF=release/4.4.4`)
  - cast `release/2.2.2` @ `9abcdef`

## What ran

`bash drill/drill.sh --ref release/9.9.9` — the full end-to-end: install the
stack, mint every template cold, snapshot and restore, uninstall to zero
residue. Then `drill/multiuser.sh` for the two-user grant matrix.

## Result

**84/85 passed, 1 failed.** 41 minutes wall clock.

- Failed: `multiuser.sh` criterion (m) — the raw instance kept a stale route
  after teardown. Filed as #999. Judged not release-blocking: it affects
  teardown residue on a host that is about to be wiped, not the trust
  boundary itself.
- The VM boundary probes (the 85-probe isolation contract) passed clean,
  which is the assertion this repo's drill exists to make.