From 08714530b3edb866c3e3504982a1054a405888ac Mon Sep 17 00:00:00 2001 From: cluade-reviewer-andresmgsl Date: Wed, 5 Aug 2026 13:16:20 +0000 Subject: [PATCH] docs(drills): the standing runner-probe venue, and why it is not a drill (#202) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit @andres ruled option A (#5631): one standing never-archived repo. This is the runbook half. The distinction the document exists to make: a drill is disposable by design and ends with the builder archiving it. This venue is the opposite — it exists so that runner-only facts can be measured on demand, and archiving it defeats the purpose. That is not hypothetical: all three drill repos were archived correctly, by the rule, and each then had to be un-archived or replaced. The request came three times in two days across #192 and #198 and never became anything. What the runbook pins, all of it measured rather than asserted: * a probe MUST run as an Actions job under ${{ github.token }} — the same DELETE answers 500 there and 204 under a PAT, so a probe run any other way produces a confident wrong answer; * probe results are written into the forge, not left in a job log, because logs age out and #192's run 701 survived only because it wrote into an issue; * no probe touches ceremony's own board — the venue exists so the live board is not the fixture; * the three probes it already owes (#192's live label lift, #205's dispatch measurement, a 0.6.0 consumer exercise after #198). STANDING THE REPO UP IS THE OPERATOR'S STEP, and this is the part I could not do rather than the part I chose not to. Measured today with this identity: POST /api/v1/orgs/heavy-duty/repos -> 403 not allowed in organization POST /api/v1/user/repos -> 201 personal namespace only Same shape as the drill delete: a deliberate boundary, not a misconfiguration. The runbook says so, says not to retry it, and says not to work around it by using a personal namespace where the org's runner and secrets do not reach. test/run.sh 22/22, shellcheck 0.10.0, actionlint, self-ref all clean. Refs #202 --- changelog.d/202.md | 17 ++++++++++ drills/README.md | 85 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 102 insertions(+) create mode 100644 changelog.d/202.md diff --git a/changelog.d/202.md b/changelog.d/202.md new file mode 100644 index 0000000..9c098a8 --- /dev/null +++ b/changelog.d/202.md @@ -0,0 +1,17 @@ +### Added + +- `drills/README.md` documents the standing runner-probe venue, + `heavy-duty/ceremony-runner-probe` — the place runner-only facts are measured + on demand, ruled as option A by the operator (#202). + +- The runbook states that the drill disposal rule does **not** apply to it. + Archiving it defeats its purpose, and that is exactly how the three existing + drill repos each became unavailable (#202). + +- It records that a probe must run as an Actions job under the workflow token: + the same call answers 500 there and 204 under a PAT, so a probe run any other + way produces a confident wrong answer (#202). + +- Creating the repo is recorded as the operator's step, measured rather than + assumed: a fleet identity gets 403 on org repo creation and 201 in its own + namespace (#202). diff --git a/drills/README.md b/drills/README.md index f59e961..b9bf710 100644 --- a/drills/README.md +++ b/drills/README.md @@ -70,3 +70,88 @@ evidence in the one file whose job is to be evidence. missing or blank. A waived drill is still a record: the file says WAIVED and why — a maintainer's call, visible and reviewable in the release PR's diff, never a silent skip. + +# The runner-probe venue + +**Not a drill.** A drill rehearses the release doors on a disposable repo and +ends. This is the opposite shape: one **standing** repo that exists so that +runner-only facts can be measured on demand, and it is **never archived**. + +`heavy-duty/ceremony-runner-probe` — private, standing, reset between probes. +Ruled by the operator as option (A) of ceremony#202 (#5631). + +## Why a standing repo, when drills are disposable + +Some facts are only true inside Actions, under the token Actions injects, and +no local harness or PAT can reproduce them. The worked example is ceremony#192: + +``` +DELETE /issues/{n}/labels/{id} -> 500 under ${{ github.token }} in a workflow +DELETE /issues/{n}/labels/{id} -> 204 under a maintainer PAT, same call +``` + +A probe that runs anywhere else passes and proves nothing. Before this venue +existed the answer was "un-archive a drill repo", which was requested three +times in two days across two issues and never became anything — the three +drill repos (`ceremony-drill-0.4.1`, `-0.4.1-final`, `-191`) are all archived, +and each was minted for one probe and then wanted again. + +## The disposal rule above does NOT apply here + +The rehearsal section says the builder archives the scratch repo and the +operator deletes it. **That rule is for drills.** Archiving this repo defeats +its entire purpose, and it is the failure mode the three archived drill repos +demonstrate — each was archived correctly, by the rule, and each then had to be +un-archived or replaced. + +So: never archive it, never delete it, and if you find it archived, un-archive +it rather than minting a fourth one. + +## Standing it up is the operator's step + +Bot identities cannot create repositories in `heavy-duty`. Measured +2026-08-05 with a fleet identity holding the `repo` scope: + +``` +POST /api/v1/orgs/heavy-duty/repos -> 403 "not allowed to create repository in organization" +POST /api/v1/user/repos -> 201 (personal namespace only) +``` + +This is the same shape as the drill delete: a deliberate permission boundary, +not a misconfiguration, and the same rule applies — do not retry it, and do not +work around it by putting the venue in a personal namespace, where the org's +runner and secrets do not reach. Ask the operator. + +## Running a probe + +1. Reset the repo to a clean state — the probe's own fixtures only, no + leftovers from the last one. A probe that inherits state is a probe whose + result you cannot attribute. +2. Install whatever the probe needs: a workflow that exercises the call, and + `docs/CONSUMERS.md`'s caller stubs if the probe is about the reconcilers. +3. **Run it as an Actions job under `${{ github.token }}`.** This is the whole + point of the venue and the one step that cannot be shortcut. A `curl` from a + laptop with a PAT answers a different question — see the 204/500 split + above — and a probe run that way is worse than no probe, because it produces + a confident wrong answer. +4. Write the results **into the forge** — an issue or a comment in this repo — + rather than leaving them in a job log. Logs age out; ceremony#192's run 701 + survived because the job wrote its findings into an issue it created. +5. Record the run number and the repo in whatever issue the probe serves. + +## What must never happen here + +No probe touches `heavy-duty/ceremony`'s board. No labels, no comments, no +runs attributable to a probe. The venue exists so that the live board does not +have to be the test fixture. + +## The probes this venue owes + +- **ceremony#192** — that the repaired sweep actually lifts a label under the + workflow token, which is the half its acceptance criteria cannot get from + the hermetic contract tests. +- **ceremony#205** — whether `POST /actions/workflows/{file}/dispatches` + works on this instance with a valid ref and inputs. Measured so far: + `GET /actions/workflows` 404s and the dispatch route answers 500 rather than + a 4xx, which is not enough to port against. +- A 0.6.0 consumer exercise once ceremony#198 has merged.