forked from heavy-duty/ceremony
docs(drills): the standing runner-probe venue, and why it is not a drill (#202)
@andres ruled option A (#5631): one standing never-archived repo. This is the runbook half. The distinction the document exists to make: a drill is disposable by design and ends with the builder archiving it. This venue is the opposite — it exists so that runner-only facts can be measured on demand, and archiving it defeats the purpose. That is not hypothetical: all three drill repos were archived correctly, by the rule, and each then had to be un-archived or replaced. The request came three times in two days across #192 and #198 and never became anything. What the runbook pins, all of it measured rather than asserted: * a probe MUST run as an Actions job under ${{ github.token }} — the same DELETE answers 500 there and 204 under a PAT, so a probe run any other way produces a confident wrong answer; * probe results are written into the forge, not left in a job log, because logs age out and #192's run 701 survived only because it wrote into an issue; * no probe touches ceremony's own board — the venue exists so the live board is not the fixture; * the three probes it already owes (#192's live label lift, #205's dispatch measurement, a 0.6.0 consumer exercise after #198). STANDING THE REPO UP IS THE OPERATOR'S STEP, and this is the part I could not do rather than the part I chose not to. Measured today with this identity: POST /api/v1/orgs/heavy-duty/repos -> 403 not allowed in organization POST /api/v1/user/repos -> 201 personal namespace only Same shape as the drill delete: a deliberate boundary, not a misconfiguration. The runbook says so, says not to retry it, and says not to work around it by using a personal namespace where the org's runner and secrets do not reach. test/run.sh 22/22, shellcheck 0.10.0, actionlint, self-ref all clean. Refs #202
This commit is contained in:
parent
dad99ddfb9
commit
08714530b3
2 changed files with 102 additions and 0 deletions
17
changelog.d/202.md
Normal file
17
changelog.d/202.md
Normal file
|
|
@ -0,0 +1,17 @@
|
|||
### Added
|
||||
|
||||
- `drills/README.md` documents the standing runner-probe venue,
|
||||
`heavy-duty/ceremony-runner-probe` — the place runner-only facts are measured
|
||||
on demand, ruled as option A by the operator (#202).
|
||||
|
||||
- The runbook states that the drill disposal rule does **not** apply to it.
|
||||
Archiving it defeats its purpose, and that is exactly how the three existing
|
||||
drill repos each became unavailable (#202).
|
||||
|
||||
- It records that a probe must run as an Actions job under the workflow token:
|
||||
the same call answers 500 there and 204 under a PAT, so a probe run any other
|
||||
way produces a confident wrong answer (#202).
|
||||
|
||||
- Creating the repo is recorded as the operator's step, measured rather than
|
||||
assumed: a fleet identity gets 403 on org repo creation and 201 in its own
|
||||
namespace (#202).
|
||||
|
|
@ -70,3 +70,88 @@ evidence in the one file whose job is to be evidence.
|
|||
missing or blank. A waived drill is still a record: the file says WAIVED and
|
||||
why — a maintainer's call, visible and reviewable in the release PR's diff,
|
||||
never a silent skip.
|
||||
|
||||
# The runner-probe venue
|
||||
|
||||
**Not a drill.** A drill rehearses the release doors on a disposable repo and
|
||||
ends. This is the opposite shape: one **standing** repo that exists so that
|
||||
runner-only facts can be measured on demand, and it is **never archived**.
|
||||
|
||||
`heavy-duty/ceremony-runner-probe` — private, standing, reset between probes.
|
||||
Ruled by the operator as option (A) of ceremony#202 (#5631).
|
||||
|
||||
## Why a standing repo, when drills are disposable
|
||||
|
||||
Some facts are only true inside Actions, under the token Actions injects, and
|
||||
no local harness or PAT can reproduce them. The worked example is ceremony#192:
|
||||
|
||||
```
|
||||
DELETE /issues/{n}/labels/{id} -> 500 under ${{ github.token }} in a workflow
|
||||
DELETE /issues/{n}/labels/{id} -> 204 under a maintainer PAT, same call
|
||||
```
|
||||
|
||||
A probe that runs anywhere else passes and proves nothing. Before this venue
|
||||
existed the answer was "un-archive a drill repo", which was requested three
|
||||
times in two days across two issues and never became anything — the three
|
||||
drill repos (`ceremony-drill-0.4.1`, `-0.4.1-final`, `-191`) are all archived,
|
||||
and each was minted for one probe and then wanted again.
|
||||
|
||||
## The disposal rule above does NOT apply here
|
||||
|
||||
The rehearsal section says the builder archives the scratch repo and the
|
||||
operator deletes it. **That rule is for drills.** Archiving this repo defeats
|
||||
its entire purpose, and it is the failure mode the three archived drill repos
|
||||
demonstrate — each was archived correctly, by the rule, and each then had to be
|
||||
un-archived or replaced.
|
||||
|
||||
So: never archive it, never delete it, and if you find it archived, un-archive
|
||||
it rather than minting a fourth one.
|
||||
|
||||
## Standing it up is the operator's step
|
||||
|
||||
Bot identities cannot create repositories in `heavy-duty`. Measured
|
||||
2026-08-05 with a fleet identity holding the `repo` scope:
|
||||
|
||||
```
|
||||
POST /api/v1/orgs/heavy-duty/repos -> 403 "not allowed to create repository in organization"
|
||||
POST /api/v1/user/repos -> 201 (personal namespace only)
|
||||
```
|
||||
|
||||
This is the same shape as the drill delete: a deliberate permission boundary,
|
||||
not a misconfiguration, and the same rule applies — do not retry it, and do not
|
||||
work around it by putting the venue in a personal namespace, where the org's
|
||||
runner and secrets do not reach. Ask the operator.
|
||||
|
||||
## Running a probe
|
||||
|
||||
1. Reset the repo to a clean state — the probe's own fixtures only, no
|
||||
leftovers from the last one. A probe that inherits state is a probe whose
|
||||
result you cannot attribute.
|
||||
2. Install whatever the probe needs: a workflow that exercises the call, and
|
||||
`docs/CONSUMERS.md`'s caller stubs if the probe is about the reconcilers.
|
||||
3. **Run it as an Actions job under `${{ github.token }}`.** This is the whole
|
||||
point of the venue and the one step that cannot be shortcut. A `curl` from a
|
||||
laptop with a PAT answers a different question — see the 204/500 split
|
||||
above — and a probe run that way is worse than no probe, because it produces
|
||||
a confident wrong answer.
|
||||
4. Write the results **into the forge** — an issue or a comment in this repo —
|
||||
rather than leaving them in a job log. Logs age out; ceremony#192's run 701
|
||||
survived because the job wrote its findings into an issue it created.
|
||||
5. Record the run number and the repo in whatever issue the probe serves.
|
||||
|
||||
## What must never happen here
|
||||
|
||||
No probe touches `heavy-duty/ceremony`'s board. No labels, no comments, no
|
||||
runs attributable to a probe. The venue exists so that the live board does not
|
||||
have to be the test fixture.
|
||||
|
||||
## The probes this venue owes
|
||||
|
||||
- **ceremony#192** — that the repaired sweep actually lifts a label under the
|
||||
workflow token, which is the half its acceptance criteria cannot get from
|
||||
the hermetic contract tests.
|
||||
- **ceremony#205** — whether `POST /actions/workflows/{file}/dispatches`
|
||||
works on this instance with a valid ref and inputs. Measured so far:
|
||||
`GET /actions/workflows` 404s and the dispatch route answers 500 rather than
|
||||
a 4xx, which is not enough to port against.
|
||||
- A 0.6.0 consumer exercise once ceremony#198 has merged.
|
||||
|
|
|
|||
Loading…
Reference in a new issue