2026-07-22 22:19:50 +00:00
|
|
|
# Drills
|
|
|
|
|
|
|
|
|
|
What a drill means in this repo: an **end-to-end rehearsal of both doors of
|
|
|
|
|
the release workflow on a disposable repo**. The contract suite proves every
|
|
|
|
|
decision offline — facts → decide → notes against fixtures, the merge door's
|
|
|
|
|
step sequence replayed in release-exercise.yml — but the doors themselves
|
|
|
|
|
only ever run live: gating on a real push event, the tag create, the
|
|
|
|
|
publish, the `-dev` re-arm (release.yml's "what is honestly untested"). The
|
|
|
|
|
drill is where they run live *before* a version rests on them.
|
|
|
|
|
|
|
|
|
|
## The rehearsal
|
|
|
|
|
|
2026-07-24 13:16:42 +00:00
|
|
|
1. Create a scratch **private** repo. It is disposable by design — but the
|
|
|
|
|
disposal is split, because the builder cannot perform the delete: at the
|
|
|
|
|
end the builder **archives** it (`PATCH /repos/{owner}/{repo}` with
|
|
|
|
|
`archived: true`, inside the `repo` scope every fleet identity holds),
|
|
|
|
|
and **deleting it is the operator's step** — `delete_repo` is
|
|
|
|
|
deliberately absent from bot tokens, fleet doctrine and not a
|
|
|
|
|
misconfiguration, so no builder that will ever run a drill can do it.
|
|
|
|
|
Do not retry the delete and do not wait on it: both 0.2.0 drills ended
|
|
|
|
|
at that wall independently (#135) — one builder held its release draft
|
|
|
|
|
in `state:building` re-trying a 403 that cannot succeed, the other wrote
|
|
|
|
|
a record asserting a delete that had not happened. **Cleanup gates
|
|
|
|
|
nothing** — not ready-for-review, not the review panel, not the merge.
|
|
|
|
|
The archived leftover is safe to leave: private, no consumers, and
|
|
|
|
|
outside `heavy-duty/ceremony`'s ref namespace — the namespace the
|
|
|
|
|
"never a branch named like the tag" rule below protects.
|
2026-07-24 11:51:55 +00:00
|
|
|
2. Install the docs/CONSUMERS.md caller stubs, pinned to a fork ref carrying
|
|
|
|
|
the release candidate tree. The candidate's `CEREMONY_SELF_REF` is by
|
|
|
|
|
construction the tag this release has not created yet, so the consumer
|
|
|
|
|
path cannot resolve directly from the candidate. Rewrite that pin to a
|
|
|
|
|
canonical candidate SHA in every carrier on the fork ref, and record the
|
|
|
|
|
fork ref and rewritten pin in the drill record.
|
2026-07-22 23:40:38 +00:00
|
|
|
|
2026-07-24 11:51:55 +00:00
|
|
|
Never create a branch named like the tag on `heavy-duty/ceremony` to
|
|
|
|
|
paper over this deadlock: it would shadow the tag for every consumer
|
|
|
|
|
until someone remembers to delete it. The 0.1.0 drill (#11) is the
|
|
|
|
|
worked example of this standing fork-ref shape.
|
2026-07-24 10:24:31 +00:00
|
|
|
3. Give it a fixture `VERSION` / `CHANGELOG.md` / `changelog.d/` /
|
|
|
|
|
`drills/` in the armed state (`X.Y.Z-dev`, the fragments directory with
|
|
|
|
|
its `README.md` marker plus at least one fragment for the ceremony to
|
|
|
|
|
consume).
|
2026-07-22 22:19:50 +00:00
|
|
|
4. Exercise both doors, one probe at a time:
|
|
|
|
|
|
|
|
|
|
1. a merge-door ceremony publishes exactly one release and re-arms main
|
|
|
|
|
to `-dev`;
|
|
|
|
|
2. a mislabeled ordinary PR is a green NOTICE no-op;
|
|
|
|
|
3. a bare-version PR without the `release` label refuses;
|
|
|
|
|
4. a re-run of the completed ceremony refuses;
|
|
|
|
|
5. a tag-door release from a manual tag;
|
|
|
|
|
6. a mismatched tag refuses.
|
|
|
|
|
|
|
|
|
|
Every refusal must refuse **creating nothing** — a probe that leaves a
|
|
|
|
|
tag or a release behind on a refusal path is a failed probe.
|
|
|
|
|
|
|
|
|
|
## The record
|
|
|
|
|
|
|
|
|
|
One file per version, `drills/X.Y.Z.md` — the shape the siblings use: what
|
|
|
|
|
was run, where, the result of each probe, failures written down plainly. The
|
2026-07-24 13:16:42 +00:00
|
|
|
record is the evidence; the scratch repo is the evidence's scaffolding. The
|
|
|
|
|
record names the scratch repo by full `owner/name` and states its disposal
|
|
|
|
|
state **as its author observed it when the record was written** — archived
|
|
|
|
|
and pending the operator's delete, or deleted only if the author genuinely
|
|
|
|
|
performed the delete. Never a disposal the author did not observe: the
|
|
|
|
|
record is the only thing that survives the drill, and 0.2.0's record shipped
|
|
|
|
|
its first draft asserting a cleanup that had not happened (#135) — false
|
|
|
|
|
evidence in the one file whose job is to be evidence.
|
2026-07-22 22:19:50 +00:00
|
|
|
|
2026-08-03 21:41:34 +00:00
|
|
|
A record has one of three shapes. A **rehearsal** records the disposable-repo
|
|
|
|
|
run above. **Doors unchanged** records the mechanically checked claim below
|
|
|
|
|
when a new rehearsal would execute the same bytes as the last one. **WAIVED**
|
|
|
|
|
records a maintainer's judgement under the standing paragraph below. If the
|
|
|
|
|
doors-unchanged conditions do not all hold, the release owes a rehearsal or a
|
|
|
|
|
waiver; the narrower shape is never a substitute for either.
|
|
|
|
|
|
|
|
|
|
## Doors unchanged
|
|
|
|
|
|
|
|
|
|
The builder may assert that no disposable-repo rehearsal is owed only when
|
|
|
|
|
all three conditions below hold at the candidate head. The release PR's panel
|
|
|
|
|
verifies the claim like any other evidence, and if any reviewer rules a full
|
|
|
|
|
drill owed, that verdict wins.
|
|
|
|
|
|
|
|
|
|
1. `git diff <last-rehearsed-tag>..HEAD -- <release-path>` contains no change
|
|
|
|
|
except the `CEREMONY_SELF_REF` pin line in
|
|
|
|
|
`.github/workflows/release.yml`.
|
|
|
|
|
2. The release path is exactly the output of
|
2026-08-25 07:32:30 +00:00
|
|
|
`.github/scripts/release-path.sh`. Run the script and paste its output into
|
|
|
|
|
the record; measure the diff with
|
|
|
|
|
`git diff <last-rehearsed-tag>..HEAD -- $(sh .github/scripts/release-path.sh)`.
|
|
|
|
|
The script's contract test keeps its list and the workflow's direct and
|
2026-08-03 21:41:34 +00:00
|
|
|
transitive dependencies in agreement.
|
|
|
|
|
3. The last rehearsed tag's own record is a full rehearsal, its release is
|
|
|
|
|
published, and `main` was re-armed to `-dev` after it.
|
|
|
|
|
|
|
|
|
|
The baseline is the last **rehearsed** tag, never merely the previous tag. A
|
|
|
|
|
previous-tag baseline could chain one doors-unchanged assertion from another
|
|
|
|
|
while the doors drift a small diff at a time; the last-rehearsed anchor makes
|
|
|
|
|
any accumulated release-path change force a new rehearsal.
|
|
|
|
|
|
|
|
|
|
The record carries all three measurements as observed at its candidate head,
|
|
|
|
|
never copied from an earlier record. `drills/0.4.1.md` and
|
|
|
|
|
`drills/0.5.0.md` are the worked examples; the latter's amendment from a
|
|
|
|
|
predicted empty `lib/` diff to the observed `lib/ruling.sh` delta is why each
|
|
|
|
|
candidate is measured afresh (#233). Re-running its stricter baseline now is
|
|
|
|
|
also the path-enumeration proof: `git diff 0.4.0 0.5.0 -- <release-path>` is
|
|
|
|
|
only the `CEREMONY_SELF_REF` pin, while adding `lib/ruling.sh` makes the diff
|
|
|
|
|
non-empty even though neither release door reads that file (#217, #237).
|
|
|
|
|
|
2026-07-22 22:19:50 +00:00
|
|
|
`actions/drill-recorded` refuses any bare-version tree whose record is
|
|
|
|
|
missing or blank. A waived drill is still a record: the file says WAIVED and
|
|
|
|
|
why — a maintainer's call, visible and reviewable in the release PR's diff,
|
|
|
|
|
never a silent skip.
|
docs(drills): the standing runner-probe venue, and why it is not a drill (#202)
@andres ruled option A (#5631): one standing never-archived repo. This is the
runbook half.
The distinction the document exists to make: a drill is disposable by design
and ends with the builder archiving it. This venue is the opposite — it exists
so that runner-only facts can be measured on demand, and archiving it defeats
the purpose. That is not hypothetical: all three drill repos were archived
correctly, by the rule, and each then had to be un-archived or replaced. The
request came three times in two days across #192 and #198 and never became
anything.
What the runbook pins, all of it measured rather than asserted:
* a probe MUST run as an Actions job under ${{ github.token }} — the same
DELETE answers 500 there and 204 under a PAT, so a probe run any other way
produces a confident wrong answer;
* probe results are written into the forge, not left in a job log, because
logs age out and #192's run 701 survived only because it wrote into an
issue;
* no probe touches ceremony's own board — the venue exists so the live board
is not the fixture;
* the three probes it already owes (#192's live label lift, #205's dispatch
measurement, a 0.6.0 consumer exercise after #198).
STANDING THE REPO UP IS THE OPERATOR'S STEP, and this is the part I could not
do rather than the part I chose not to. Measured today with this identity:
POST /api/v1/orgs/heavy-duty/repos -> 403 not allowed in organization
POST /api/v1/user/repos -> 201 personal namespace only
Same shape as the drill delete: a deliberate boundary, not a misconfiguration.
The runbook says so, says not to retry it, and says not to work around it by
using a personal namespace where the org's runner and secrets do not reach.
test/run.sh 22/22, shellcheck 0.10.0, actionlint, self-ref all clean.
Refs #202
2026-08-05 13:16:20 +00:00
|
|
|
|
docs(runner-probes): its own document, an arming procedure, and the evidence boundary made consistent (#202)
@codex-reviewer-andresmgsl's four gaps.
1. BRANCH UPDATED TO CURRENT MAIN. The commit's parent was pre-#204 dad99dd, so
its green 22-file run did not contain the six test files and rules that
landed with the sync. Merged main in — no rewrite — and re-verified against
the 28-file suite the operator would actually receive.
2. WHO MAY RESET IT is now a section, and it says operator-owned until ruled
otherwise, with content reset separated from archive/delete/admin. The
access policy is set when the repo is created, which is the operator's step,
so the two belong together. Flagged for @andres rather than assumed.
3. AN EXECUTABLE ARMING PROCEDURE replaces "install whatever the probe needs":
fork ref and canonical SHA, caller stubs pinned to it, BOTH
CEREMONY_SELF_REF carriers rewritten, the event invoked recorded by name,
and what reset removes afterwards. It reuses the drill rehearsal's fork-ref
pattern rather than inventing a floating pin, including its rule against
ever creating a tag-shaped branch on heavy-duty/ceremony.
4. THE EVIDENCE CONTRADICTION IS RESOLVED. "Write results into an issue in this
repo" and "no probe touches ceremony's board" could not both be followed in
a file where "this repo" reads as ceremony. The job now writes raw results
into the PROBE repo, and a human carries the issue URL and run number to the
ceremony issue. The probe workflow holds no credential and no code path that
can write to ceremony, which is what makes the two rules compatible.
Placement: the operational contract moves to docs/RUNNER-PROBES.md, with a
short cross-link in drills/README.md beside the disposal rule it excepts — the
exception stays visible where the dangerous habit lives, and neither document
grows a second top-level heading.
test/run.sh 28/28 on the updated branch; shellcheck 0.10.0, actionlint,
self-ref, marker, vendored and changelog-armed clean.
Refs #202
2026-08-05 13:40:28 +00:00
|
|
|
---
|
|
|
|
|
|
|
|
|
|
**Standing runner probes are not drills.** The disposal rule above — builder
|
|
|
|
|
archives, operator deletes — is for the disposable scratch repo a drill runs
|
|
|
|
|
in. `heavy-duty/ceremony-runner-probe` is the opposite shape: it stands, and
|
|
|
|
|
archiving it is the failure mode that made all three previous drill repos
|
|
|
|
|
unavailable. See [docs/RUNNER-PROBES.md](../docs/RUNNER-PROBES.md) (#202).
|