drills/README.md — the standing runner-probe venue, and why the drill disposal rule does not apply to it (#202) #207

Merged
andres merged 14 commits from build/202-runner-probe-venue into main 2026-08-05 17:18:59 +00:00
2 changed files with 71 additions and 41 deletions
Showing only changes of commit e27acd8ab9 - Show all commits

View file

@ -19,9 +19,17 @@
assumed: a fleet identity gets 403 on org repo creation and 201 in its own
namespace (#202).
- It carries an executable arming procedure — fork ref, canonical SHA, both
`CEREMONY_SELF_REF` carriers rewritten, and what the result must record
(#202).
- It carries an executable two-layer arming procedure: an immutable candidate
code SHA and an armed workflow commit on top of it. A single layer is
self-referential — rewriting a workflow makes a new commit, and a commit
cannot contain its own object ID (#202).
- Callers are pinned by layer: composite actions to the candidate code SHA,
reusable workflows to the armed SHA, which is the only revision whose inner
checkout points at the fork (#202).
- A `git grep` over the armed tree gates the rewrite non-zero, so a partial one
refuses instead of silently testing canonical main (#202).
- Probe results are written to an issue in the probe repo and carried to the
ceremony issue by a human, so the probe holds no path that can write to the

View file

@ -61,8 +61,9 @@ exists rather than assuming it here.
1. Reset the repo to a clean state — the probe's own fixtures only, no
leftovers from the last one. A probe that inherits state is a probe whose
result you cannot attribute.
2. **Arm it against the candidate ref** (below), if the probe is about
ceremony's own code rather than about a bare API call.
2. **Arm it against the candidate** (below) — two layers, candidate code and
armed workflow — if the probe is about ceremony's own machinery rather than
about a bare API call.
3. **Run it as an Actions job under `${{ github.token }}`.** This is the whole
point of the venue and the one step that cannot be shortcut. A `curl` from a
laptop with a PAT answers a different question — see the 204/500 split
@ -83,48 +84,69 @@ exists rather than assuming it here.
## Arming a candidate ref
A probe that exercises ceremony's own machinery needs the candidate tree
reachable from a `uses:` line. The shape is the drill rehearsal's, reused
rather than reinvented (`drills/README.md` step 2):
reachable from a `uses:` line. This is the fork-ref shape `drills/README.md`
step 2 points at, written out — and it has **two layers**, which is the part
that is easy to get wrong and impossible to fix afterwards.
1. **The candidate is a commit SHA, on a fork ref.** Push the candidate tree to
a fork under the identity running the probe —
`<identity>/ceremony@probe-<issue>` — and take its canonical SHA. Never
create a branch on `heavy-duty/ceremony` named like a tag: it shadows that
tag for every consumer until somebody remembers to delete it.
2. **Rewrite the COORDINATE, not only the ref.** The candidate SHA exists only
in your fork, so a stub still saying `heavy-duty/ceremony/...@<sha>` cannot
resolve it — it would fail, or worse, resolve something else. In the probe
repo, every caller `uses:` becomes
`<identity>/ceremony/<path>@<canonical-sha>`.
3. **Rewrite the candidate's own self-checkout, both halves.** `labels.yml` and
the release doors hardcode `repository: heavy-duty/ceremony` beside
`ref: ${{ env.CEREMONY_SELF_REF }}`. Changing only the ref makes the
candidate fetch your SHA *from the canonical repository*, where it does not
exist. So: every `repository:` carrier becomes `<identity>/ceremony`, and
both `CEREMONY_SELF_REF` values become the canonical SHA.
**Why two.** The candidate's own workflows contain
`repository: heavy-duty/ceremony` beside `ref: ${{ env.CEREMONY_SELF_REF }}`,
so they must be rewritten to point at the fork and at the candidate. But
rewriting them **creates a new commit**, and a commit cannot contain its own
object ID. A single-layer arming is therefore self-referential: pin the callers
to the pre-rewrite SHA and they load the *unarmed* workflows; pin them to the
post-rewrite one and you are asking a commit to embed itself
(@codex-reviewer-andresmgsl, #202 review).
**Then prove the rewrite was total**, because a partial one silently tests
canonical `main` instead of the candidate:
So:
| layer | what it is | what it carries |
|---|---|---|
| **candidate code SHA** | the immutable tree under test | `actions/`, `lib/` — untouched |
| **armed workflow SHA** | a small child commit on top of it | workflows rewritten to the fork + `CEREMONY_SELF_REF` = the candidate code SHA |
### The procedure
1. **Push the candidate tree** to a fork under the identity that will run the
probe — `<identity>/ceremony@probe-<issue>` — and record its SHA. That is
the **candidate code SHA**. Never create a branch on
`heavy-duty/ceremony` named like a tag: it shadows that tag for every
consumer until somebody remembers to delete it.
2. **Commit the arming on top of it.** In that same fork branch, rewrite every
workflow carrier — `repository:``<identity>/ceremony`, and both
`CEREMONY_SELF_REF` values → the **candidate code SHA** from step 1. Record
the resulting SHA. That is the **armed workflow SHA**.
3. **Pin the probe repo's callers by layer**, because they are not the same
thing:
- composite-action callers →
`<identity>/ceremony/actions/<name>@<candidate-code-sha>`;
- reusable-workflow callers →
`<identity>/ceremony/.github/workflows/<file>@<armed-workflow-sha>`, since
that is the only revision whose inner checkout is rewritten.
4. **Gate the rewrite mechanically, against the armed tree** — not against
whichever checkout happens to be current, and not by eyeballing prose:
```sh
grep -rn 'heavy-duty/ceremony' .github/ docs/CONSUMERS.md
git -C <armed-checkout> grep -nE \
'(uses:|repository:)[[:space:]]*heavy-duty/ceremony' -- .github \
&& { echo "arming incomplete" >&2; exit 1; }
```
Every remaining hit must be prose. A `uses:`, a `repository:` or a pin among
them means the probe is not armed, and its result is about the wrong tree.
4. **Invoke the probe by the event it is about**, and record which: a
`workflow_dispatch` of the caller, or the real board event the probe is
testing. A probe that fires a different event than the one under test
proves something else.
5. **Record the fork ref, the rewritten pin, the workflow invoked and the run
number** in the probe repo's result issue. Those four are what make the
result reproducible; without the pin especially, a later reader cannot tell
which tree answered.
6. **Reset removes the candidate-specific EXECUTABLE state**: the caller stubs,
the probe workflow, the candidate branch — so the next probe cannot inherit
a pin it did not choose. **Result issues are never deleted.** They may be
closed or relabelled; deleting them would recreate the expiring-log problem
this venue exists to avoid.
Exit non-zero on any hit. A partial rewrite does not fail loudly on its
own — it silently tests canonical `main`, and the probe's answer is then
about the wrong tree.
5. **Invoke the probe by the event it is about**, and record which: a
`workflow_dispatch`, or the real board event under test. A probe that fires
a different event than the one under test proves something else.
6. **The result issue records all of it**: the fork repository, the candidate
code SHA, the armed workflow SHA, every rewritten carrier, the workflow
invoked and the run number. Those are what make the result reproducible;
without the two SHAs distinguished, a later reader cannot tell which tree
answered.
7. **Reset removes the candidate-specific EXECUTABLE state**: the caller stubs,
the probe workflow, the candidate and armed branches — so the next probe
cannot inherit a pin it did not choose. **Result issues are never deleted.**
They may be closed or relabelled; deleting them would recreate the
expiring-log problem this venue exists to avoid.
## Who may reset it