drills/README.md — the standing runner-probe venue, and why the drill disposal rule does not apply to it (#202) #207

Merged
andres merged 14 commits from build/202-runner-probe-venue into main 2026-08-05 17:18:59 +00:00
2 changed files with 102 additions and 0 deletions
Showing only changes of commit 08714530b3 - Show all commits

17
changelog.d/202.md Normal file
View file

@ -0,0 +1,17 @@
### Added
- `drills/README.md` documents the standing runner-probe venue,
`heavy-duty/ceremony-runner-probe` — the place runner-only facts are measured
on demand, ruled as option A by the operator (#202).
- The runbook states that the drill disposal rule does **not** apply to it.
Archiving it defeats its purpose, and that is exactly how the three existing
drill repos each became unavailable (#202).
- It records that a probe must run as an Actions job under the workflow token:
the same call answers 500 there and 204 under a PAT, so a probe run any other
way produces a confident wrong answer (#202).
- Creating the repo is recorded as the operator's step, measured rather than
assumed: a fleet identity gets 403 on org repo creation and 201 in its own
namespace (#202).

View file

@ -70,3 +70,88 @@ evidence in the one file whose job is to be evidence.
missing or blank. A waived drill is still a record: the file says WAIVED and missing or blank. A waived drill is still a record: the file says WAIVED and
why — a maintainer's call, visible and reviewable in the release PR's diff, why — a maintainer's call, visible and reviewable in the release PR's diff,
never a silent skip. never a silent skip.
# The runner-probe venue
**Not a drill.** A drill rehearses the release doors on a disposable repo and
ends. This is the opposite shape: one **standing** repo that exists so that
runner-only facts can be measured on demand, and it is **never archived**.
`heavy-duty/ceremony-runner-probe` — private, standing, reset between probes.
Ruled by the operator as option (A) of ceremony#202 (#5631).
## Why a standing repo, when drills are disposable
Some facts are only true inside Actions, under the token Actions injects, and
no local harness or PAT can reproduce them. The worked example is ceremony#192:
```
DELETE /issues/{n}/labels/{id} -> 500 under ${{ github.token }} in a workflow
DELETE /issues/{n}/labels/{id} -> 204 under a maintainer PAT, same call
```
A probe that runs anywhere else passes and proves nothing. Before this venue
existed the answer was "un-archive a drill repo", which was requested three
times in two days across two issues and never became anything — the three
drill repos (`ceremony-drill-0.4.1`, `-0.4.1-final`, `-191`) are all archived,
and each was minted for one probe and then wanted again.
## The disposal rule above does NOT apply here
The rehearsal section says the builder archives the scratch repo and the
operator deletes it. **That rule is for drills.** Archiving this repo defeats
its entire purpose, and it is the failure mode the three archived drill repos
demonstrate — each was archived correctly, by the rule, and each then had to be
un-archived or replaced.
So: never archive it, never delete it, and if you find it archived, un-archive
it rather than minting a fourth one.
## Standing it up is the operator's step
Bot identities cannot create repositories in `heavy-duty`. Measured
2026-08-05 with a fleet identity holding the `repo` scope:
```
POST /api/v1/orgs/heavy-duty/repos -> 403 "not allowed to create repository in organization"
POST /api/v1/user/repos -> 201 (personal namespace only)
```
This is the same shape as the drill delete: a deliberate permission boundary,
not a misconfiguration, and the same rule applies — do not retry it, and do not
work around it by putting the venue in a personal namespace, where the org's
runner and secrets do not reach. Ask the operator.
## Running a probe
1. Reset the repo to a clean state — the probe's own fixtures only, no
leftovers from the last one. A probe that inherits state is a probe whose
result you cannot attribute.
2. Install whatever the probe needs: a workflow that exercises the call, and
`docs/CONSUMERS.md`'s caller stubs if the probe is about the reconcilers.
3. **Run it as an Actions job under `${{ github.token }}`.** This is the whole
point of the venue and the one step that cannot be shortcut. A `curl` from a
laptop with a PAT answers a different question — see the 204/500 split
above — and a probe run that way is worse than no probe, because it produces
a confident wrong answer.
4. Write the results **into the forge** — an issue or a comment in this repo —
rather than leaving them in a job log. Logs age out; ceremony#192's run 701
survived because the job wrote its findings into an issue it created.
5. Record the run number and the repo in whatever issue the probe serves.
## What must never happen here
No probe touches `heavy-duty/ceremony`'s board. No labels, no comments, no
runs attributable to a probe. The venue exists so that the live board does not
have to be the test fixture.
## The probes this venue owes
- **ceremony#192** — that the repaired sweep actually lifts a label under the
workflow token, which is the half its acceptance criteria cannot get from
the hermetic contract tests.
- **ceremony#205** — whether `POST /actions/workflows/{file}/dispatches`
works on this instance with a valid ref and inputs. Measured so far:
`GET /actions/workflows` 404s and the dispatch route answers 500 rather than
a 4xx, which is not enough to port against.
- A 0.6.0 consumer exercise once ceremony#198 has merged.