diff --git a/changelog.d/202.md b/changelog.d/202.md index 9c098a8..c824c63 100644 --- a/changelog.d/202.md +++ b/changelog.d/202.md @@ -1,9 +1,12 @@ ### Added -- `drills/README.md` documents the standing runner-probe venue, +- `docs/RUNNER-PROBES.md` documents the standing runner-probe venue, `heavy-duty/ceremony-runner-probe` — the place runner-only facts are measured on demand, ruled as option A by the operator (#202). +- `drills/README.md` cross-links it beside the disposal rule, so the exception + is visible where the dangerous habit lives (#202). + - The runbook states that the drill disposal rule does **not** apply to it. Archiving it defeats its purpose, and that is exactly how the three existing drill repos each became unavailable (#202). @@ -15,3 +18,11 @@ - Creating the repo is recorded as the operator's step, measured rather than assumed: a fleet identity gets 403 on org repo creation and 201 in its own namespace (#202). + +- It carries an executable arming procedure — fork ref, canonical SHA, both + `CEREMONY_SELF_REF` carriers rewritten, and what the result must record + (#202). + +- Probe results are written to an issue in the probe repo and carried to the + ceremony issue by a human, so the probe holds no path that can write to the + live board (#202). diff --git a/docs/RUNNER-PROBES.md b/docs/RUNNER-PROBES.md new file mode 100644 index 0000000..7797f6e --- /dev/null +++ b/docs/RUNNER-PROBES.md @@ -0,0 +1,136 @@ +# Runner probes + +**Not a drill.** A drill rehearses the release doors on a disposable repo and +ends. This is the opposite shape: one **standing** repo that exists so that +runner-only facts can be measured on demand, and it is **never archived**. + +`heavy-duty/ceremony-runner-probe` — private, standing, reset between probes. +Ruled by the operator as option (A) of ceremony#202 (#5631). + +## Why a standing repo, when drills are disposable + +Some facts are only true inside Actions, under the token Actions injects, and +no local harness or PAT can reproduce them. The worked example is ceremony#192: + +``` +DELETE /issues/{n}/labels/{id} -> 500 under ${{ github.token }} in a workflow +DELETE /issues/{n}/labels/{id} -> 204 under a maintainer PAT, same call +``` + +A probe that runs anywhere else passes and proves nothing. Before this venue +existed the answer was "un-archive a drill repo", which was requested three +times in two days across two issues and never became anything — the three +drill repos (`ceremony-drill-0.4.1`, `-0.4.1-final`, `-191`) are all archived, +and each was minted for one probe and then wanted again. + +## The disposal rule above does NOT apply here + +The rehearsal section says the builder archives the scratch repo and the +operator deletes it. **That rule is for drills.** Archiving this repo defeats +its entire purpose, and it is the failure mode the three archived drill repos +demonstrate — each was archived correctly, by the rule, and each then had to be +un-archived or replaced. + +So: never archive it, never delete it, and if you find it archived, un-archive +it rather than minting a fourth one. + +## Standing it up is the operator's step + +Bot identities cannot create repositories in `heavy-duty`. Measured +2026-08-05 with a fleet identity holding the `repo` scope: + +``` +POST /api/v1/orgs/heavy-duty/repos -> 403 "not allowed to create repository in organization" +POST /api/v1/user/repos -> 201 (personal namespace only) +``` + +This is the same shape as the drill delete: a deliberate permission boundary, +not a misconfiguration, and the same rule applies — do not retry it, and do not +work around it by putting the venue in a personal namespace, where the org's +runner and secrets do not reach. Ask the operator. + +## Running a probe + +1. Reset the repo to a clean state — the probe's own fixtures only, no + leftovers from the last one. A probe that inherits state is a probe whose + result you cannot attribute. +2. **Arm it against the candidate ref** (below), if the probe is about + ceremony's own code rather than about a bare API call. +3. **Run it as an Actions job under `${{ github.token }}`.** This is the whole + point of the venue and the one step that cannot be shortcut. A `curl` from a + laptop with a PAT answers a different question — see the 204/500 split + above — and a probe run that way is worse than no probe, because it produces + a confident wrong answer. +4. **The job writes its raw results into an issue in the PROBE repo** — + `heavy-duty/ceremony-runner-probe` — not into ceremony. Logs age out; + ceremony#192's run 701 survived only because the job wrote its findings + into an issue it created. +5. **A human then records the probe issue's URL and the Actions run number on + the ceremony issue the probe serves.** That hop is deliberate and is the + whole of the boundary: the probe workflow holds no credential and no code + path that can write to `heavy-duty/ceremony`, so "the probe reports its + findings" and "the probe cannot touch the live board" stay compatible + rather than contradicting each other (@codex-reviewer-andresmgsl, #202 + review). + +## Arming a candidate ref + +A probe that exercises ceremony's own machinery needs the candidate tree +reachable from a `uses:` line. The shape is the drill rehearsal's, reused +rather than reinvented (`drills/README.md` step 2): + +1. **The candidate is a commit SHA, on a fork ref.** Push the candidate tree to + a fork under the identity running the probe — + `/ceremony@probe-` — and take its canonical SHA. Never + create a branch on `heavy-duty/ceremony` named like a tag: it shadows that + tag for every consumer until somebody remembers to delete it. +2. **Install `docs/CONSUMERS.md`'s caller stubs** in the probe repo, pinned to + that SHA — every `uses:` reference, not some of them. +3. **Rewrite `CEREMONY_SELF_REF` in the candidate itself** to the same + canonical SHA. A candidate carries the tag it has not created yet, so the + self-consumption checkout cannot resolve until it is rewritten. Both + carriers — `labels.yml` and `release.yml` — take the same value. +4. **Invoke the probe by the event it is about**, and record which: a + `workflow_dispatch` of the caller, or the real board event the probe is + testing. A probe that fires a different event than the one under test + proves something else. +5. **Record the fork ref, the rewritten pin, the workflow invoked and the run + number** in the probe repo's result issue. Those four are what make the + result reproducible; without the pin especially, a later reader cannot tell + which tree answered. +6. **Reset removes the candidate-specific fixtures**: the caller stubs, the + probe workflow and the result branch, so the next probe does not inherit a + pin it did not choose. The result *issue* stays — it is the evidence. + +## Who may reset it + +**Operator-owned until ruled otherwise.** #202's task 4 asks who may reset the +venue, and creating the repo is the operator's step, so the access policy is +his to set at the same time (@codex-reviewer-andresmgsl, #202 review). + +Two levels, deliberately separated: + +- **content reset** — deleting probe branches, workflows, fixtures and stale + result issues; the ordinary between-probes operation; +- **archive / delete / admin** — which is where the drill rule's damage came + from, and which no bot identity should hold here. + +If fleet identities are given push access for content reset, this section +records that; until then, ask. + +## What must never happen here + +No probe touches `heavy-duty/ceremony`'s board. No labels, no comments, no +runs attributable to a probe. The venue exists so that the live board does not +have to be the test fixture. + +## The probes this venue owes + +- **ceremony#192** — that the repaired sweep actually lifts a label under the + workflow token, which is the half its acceptance criteria cannot get from + the hermetic contract tests. +- **ceremony#205** — whether `POST /actions/workflows/{file}/dispatches` + works on this instance with a valid ref and inputs. Measured so far: + `GET /actions/workflows` 404s and the dispatch route answers 500 rather than + a 4xx, which is not enough to port against. +- A 0.6.0 consumer exercise once ceremony#198 has merged. diff --git a/drills/README.md b/drills/README.md index 4f733fb..b75847f 100644 --- a/drills/README.md +++ b/drills/README.md @@ -111,87 +111,10 @@ missing or blank. A waived drill is still a record: the file says WAIVED and why — a maintainer's call, visible and reviewable in the release PR's diff, never a silent skip. -# The runner-probe venue +--- -**Not a drill.** A drill rehearses the release doors on a disposable repo and -ends. This is the opposite shape: one **standing** repo that exists so that -runner-only facts can be measured on demand, and it is **never archived**. - -`heavy-duty/ceremony-runner-probe` — private, standing, reset between probes. -Ruled by the operator as option (A) of ceremony#202 (#5631). - -## Why a standing repo, when drills are disposable - -Some facts are only true inside Actions, under the token Actions injects, and -no local harness or PAT can reproduce them. The worked example is ceremony#192: - -``` -DELETE /issues/{n}/labels/{id} -> 500 under ${{ github.token }} in a workflow -DELETE /issues/{n}/labels/{id} -> 204 under a maintainer PAT, same call -``` - -A probe that runs anywhere else passes and proves nothing. Before this venue -existed the answer was "un-archive a drill repo", which was requested three -times in two days across two issues and never became anything — the three -drill repos (`ceremony-drill-0.4.1`, `-0.4.1-final`, `-191`) are all archived, -and each was minted for one probe and then wanted again. - -## The disposal rule above does NOT apply here - -The rehearsal section says the builder archives the scratch repo and the -operator deletes it. **That rule is for drills.** Archiving this repo defeats -its entire purpose, and it is the failure mode the three archived drill repos -demonstrate — each was archived correctly, by the rule, and each then had to be -un-archived or replaced. - -So: never archive it, never delete it, and if you find it archived, un-archive -it rather than minting a fourth one. - -## Standing it up is the operator's step - -Bot identities cannot create repositories in `heavy-duty`. Measured -2026-08-05 with a fleet identity holding the `repo` scope: - -``` -POST /api/v1/orgs/heavy-duty/repos -> 403 "not allowed to create repository in organization" -POST /api/v1/user/repos -> 201 (personal namespace only) -``` - -This is the same shape as the drill delete: a deliberate permission boundary, -not a misconfiguration, and the same rule applies — do not retry it, and do not -work around it by putting the venue in a personal namespace, where the org's -runner and secrets do not reach. Ask the operator. - -## Running a probe - -1. Reset the repo to a clean state — the probe's own fixtures only, no - leftovers from the last one. A probe that inherits state is a probe whose - result you cannot attribute. -2. Install whatever the probe needs: a workflow that exercises the call, and - `docs/CONSUMERS.md`'s caller stubs if the probe is about the reconcilers. -3. **Run it as an Actions job under `${{ github.token }}`.** This is the whole - point of the venue and the one step that cannot be shortcut. A `curl` from a - laptop with a PAT answers a different question — see the 204/500 split - above — and a probe run that way is worse than no probe, because it produces - a confident wrong answer. -4. Write the results **into the forge** — an issue or a comment in this repo — - rather than leaving them in a job log. Logs age out; ceremony#192's run 701 - survived because the job wrote its findings into an issue it created. -5. Record the run number and the repo in whatever issue the probe serves. - -## What must never happen here - -No probe touches `heavy-duty/ceremony`'s board. No labels, no comments, no -runs attributable to a probe. The venue exists so that the live board does not -have to be the test fixture. - -## The probes this venue owes - -- **ceremony#192** — that the repaired sweep actually lifts a label under the - workflow token, which is the half its acceptance criteria cannot get from - the hermetic contract tests. -- **ceremony#205** — whether `POST /actions/workflows/{file}/dispatches` - works on this instance with a valid ref and inputs. Measured so far: - `GET /actions/workflows` 404s and the dispatch route answers 500 rather than - a 4xx, which is not enough to port against. -- A 0.6.0 consumer exercise once ceremony#198 has merged. +**Standing runner probes are not drills.** The disposal rule above — builder +archives, operator deletes — is for the disposable scratch repo a drill runs +in. `heavy-duty/ceremony-runner-probe` is the opposite shape: it stands, and +archiving it is the failure mode that made all three previous drill repos +unavailable. See [docs/RUNNER-PROBES.md](../docs/RUNNER-PROBES.md) (#202).