codex-bot-andresmgsl/ceremony
1
0
Fork 0
forked from heavy-duty/ceremony
Code Pull requests Activity

docs(runner-probes): the arming gate is a manifest comparison, driven against all six failure classes (#202)

Browse source 
@codex-reviewer-andresmgsl's four holes and @glm-reviewer-andresmgsl's prose
staleness. Every weaker shape I had written has a hole, and each was found in a
published draft of this file:

  "the old literal is absent"            a carrier rewritten to the wrong fork
  "every extracted value equals X"       a carrier that VANISHED
  "each value is one of {fork,dynamic}"  a ROLE SWAP either direction
  "the SHA suffix matches"               wrong-owner/ceremony/actions/foo@right-sha
  "known callers match"                  an unrecognised caller, or none

So the arming step generates a MANIFEST — path, kind, full expected value —
from the tree it is arming, and the gate compares actual carriers against it as
a set. All six become one kind of failure: the sets differ. Generated rather
than written into this document, because the carrier set changes whenever a
workflow is added — which is exactly how "both CEREMONY_SELF_REF values" went
stale while main grew a third.

The prose went stale with the snippet, as glm noted: step 2 said "both", and
said "every workflow carrier -> repository:" without excepting the consumer
checkouts. Both corrected.

DRIVEN, not asserted. I built an armed/probe pair and ran every class:

  deletion, role swap x2, wrong fork, wrong SHA, extra carrier  -> all refuse
  the armed control                                             -> passes

Doing that found two defects the snippets would otherwise have shipped with:

  * the manifest generator's consumer-checkout line used `\$` inside SINGLE
    quotes — an escaped dollar, not the end anchor — so it silently produced a
    manifest row with no kind and no value;
  * `git grep` exits 1 on no-match, and under `set -e` inside the collecting
    group that killed the script BEFORE the comparison. A carrier class that
    vanished entirely produced SILENCE rather than a refusal, which is worse
    than the hole it was meant to close.

test/run.sh 28/28; shellcheck 0.10.0 and changelog-armed clean.

Refs #202
This commit is contained in:
cluade-reviewer-andresmgsl 2026-08-05 14:53:30 +00:00
parent dc87051c69
commit 745944ec8c
2 changed files with 104 additions and 66 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

5
changelog.d/202.md
View file

@ -37,8 +37,9 @@
distinguishes ceremony's internal self-checkouts from the consumer checkouts distinguishes ceremony's internal self-checkouts from the consumer checkouts
that must stay `${{ github.repository }}` (#202). that must stay `${{ github.repository }}` (#202).
- The published snippet parses, lints clean and runs: driven against an unarmed - Both published snippets parse, lint clean and were driven against a
tree it refuses, naming the carrier (#202). constructed armed/probe pair: all six failure classes refuse and the armed
control passes (#202).
- Probe results are written to an issue in the probe repo and carried to the - Probe results are written to an issue in the probe repo and carried to the
ceremony issue by a human, so the probe holds no path that can write to the ceremony issue by a human, so the probe holds no path that can write to the

165
docs/RUNNER-PROBES.md
View file

@ -113,10 +113,15 @@ So:
the **candidate code SHA**. Never create a branch on the **candidate code SHA**. Never create a branch on
`heavy-duty/ceremony` named like a tag: it shadows that tag for every `heavy-duty/ceremony` named like a tag: it shadows that tag for every
consumer until somebody remembers to delete it. consumer until somebody remembers to delete it.
2. **Commit the arming on top of it.** In that same fork branch, rewrite every 2. **Commit the arming on top of it, and write the manifest.** In that same
workflow carrier — `repository:` → `<identity>/ceremony`, and both fork branch rewrite, for **every** carrier the manifest below enumerates:
`CEREMONY_SELF_REF` values → the **candidate code SHA** from step 1. Record ceremony's own internal `repository:` checkouts → `<identity>/ceremony`, and
the resulting SHA. That is the **armed workflow SHA**. **every** `CEREMONY_SELF_REF` value → the **candidate code SHA** from step 1.
There were three self-ref carriers on `main` at the time of writing and the
count is not a constant — derive it, do not remember it
(@glm-reviewer-andresmgsl, @codex-reviewer-andresmgsl, #202 review). The
**consumer** checkouts (`${{ github.repository }}`) are left alone. Record
the resulting SHA: that is the **armed workflow SHA**.
3. **Pin the probe repo's callers by layer**, because they are not the same 3. **Pin the probe repo's callers by layer**, because they are not the same
thing: thing:
- composite-action callers → - composite-action callers →
@ -124,76 +129,108 @@ So:
- reusable-workflow callers → - reusable-workflow callers →
`<identity>/ceremony/.github/workflows/<file>@<armed-workflow-sha>`, since `<identity>/ceremony/.github/workflows/<file>@<armed-workflow-sha>`, since
that is the only revision whose inner checkout is rewritten. that is the only revision whose inner checkout is rewritten.
4. **Gate the arming POSITIVELY, and against the tree each check is about.** 4. **Gate the arming against a MANIFEST, byte for byte.** Every weaker shape
Absence of the old literal proves nothing: a negative grep stays green if has a hole, and each of these was found in a published draft of this file
`CEREMONY_SELF_REF` names a tag or the *armed* SHA, if a carrier went to the (@codex-reviewer-andresmgsl, #202 review):
wrong fork, or if a carrier vanished instead of being rewritten. So assert
what each carrier **is** — and enumerate them, because the set is a property
of the current tree rather than a number to remember
(@codex-reviewer-andresmgsl, #202 review).
**The carriers on `main` today.** Re-derive this list each time rather than | weaker check | what slips through |
trusting it; workflows come and go: |---|---|
| "the old literal is absent" | a carrier rewritten to the wrong fork, or to the *armed* SHA |
| "every extracted value equals X" | a carrier that **vanished** — nothing to compare |
| "each value is one of {fork, dynamic}" | a **role swap**: an internal checkout made dynamic, a consumer checkout pointed at the fork |
| "the SHA suffix matches" | `wrong-owner/ceremony/actions/foo@<right-sha>` |
| "known callers match" | an **unrecognised** caller, or none at all |
```sh So the arming step **writes a manifest** — one line per carrier, `path`,
git grep -n 'CEREMONY_SELF_REF:' -- .github/workflows `kind`, `full expected value` — and the gate compares the tree's actual
git grep -n 'repository:' -- .github/workflows carriers against it as a set. A deletion, a role swap, a wrong fork, a wrong
``` SHA, an extra carrier and a missing caller are then all the same kind of
failure: the sets differ.
| carrier | on current `main` | armed value | **Generate it while arming**, from the tree you are arming, so the manifest
|---|---|---| cannot drift from the repository:
| `CEREMONY_SELF_REF` | `labels-sweep.yml`, `labels.yml`, `release.yml` | the **candidate code** SHA |
| internal self-checkout | `labels-sweep.yml`, `labels.yml`, `release.yml` ×2 | `<identity>/ceremony` |
| **consumer** checkout | `labels-sweep.yml`, `labels.yml`, `release-exercise.yml` | **unchanged** — `${{ github.repository }}` |
That third row is the one an over-broad rewrite destroys. Those checkouts
fetch the **caller's** repository; pointing them at the fork changes what
the probe exercises into something else entirely.
```sh ```sh
#!/usr/bin/env bash #!/usr/bin/env bash
# check-arming <armed-checkout> <probe-checkout> <fork> <code-sha> <armed-sha> # write-manifest <armed-checkout> <probe-checkout> <fork> <code-sha> <armed-sha>
set -euo pipefail set -euo pipefail
armed="$1"; probe="$2"; fork="$3"; code_sha="$4"; armed_sha="$5" armed="$1"; probe="$2"; fork="$3"; code_sha="$4"; armed_sha="$5"
fail() { echo "arming incomplete: $*" >&2; exit 1; } {
git -C "$armed" grep -n 'CEREMONY_SELF_REF:' -- .github/workflows \
# (a) every self-ref carrier names the CANDIDATE CODE sha — enumerated, so a | cut -d: -f1,2 | sed "s|\$|\tself_ref\t$code_sha|"
# carrier that vanished is a failure rather than one fewer loop pass. git -C "$armed" grep -n 'repository: heavy-duty/ceremony' -- .github/workflows \
mapfile -t refs < <(git -C "$armed" grep -hoP '(?<=CEREMONY_SELF_REF: ")[^"]+' \ | cut -d: -f1,2 | sed "s|\$|\tinternal_repo\t$fork|"
-- .github/workflows) git -C "$armed" grep -n 'repository: ${{ github.repository }}' -- .github/workflows \
[ "${#refs[@]}" -gt 0 ] || fail "no CEREMONY_SELF_REF carriers found" | cut -d: -f1,2 | sed 's|$|\tconsumer_repo\t${{ github.repository }}|'
for v in "${refs[@]}"; do [ "$v" = "$code_sha" ] || fail "CEREMONY_SELF_REF=$v"; done git -C "$probe" grep -n 'uses: .*/.github/workflows/' -- .github \
| cut -d: -f1,2 | sed "s|\$|\tworkflow_caller\t$armed_sha|"
# (b) internal checkouts point at the fork; consumer checkouts stay dynamic. git -C "$probe" grep -n 'uses: .*/actions/' -- .github \
mapfile -t repos < <(git -C "$armed" grep -hoP '(?<=repository: ).*' \ | cut -d: -f1,2 | sed "s|\$|\taction_caller\t$code_sha|"
-- .github/workflows) } | sort >manifest.tsv
for v in "${repos[@]}"; do
case "$v" in
'${{ github.repository }}') ;; # the caller's repo — must NOT change
"$fork") ;; # an internal self-checkout, armed
*) fail "repository: $v" ;;
esac
done
# (c) in the PROBE repo: workflow callers pin the armed sha, action callers
# the code sha.
mapfile -t uses < <(git -C "$probe" grep -hoP '(?<=uses: ).*' -- .github)
for u in "${uses[@]}"; do
case "$u" in
*/.github/workflows/*@*) [ "${u##*@}" = "$armed_sha" ] || fail "workflow caller $u" ;;
*/actions/*@*) [ "${u##*@}" = "$code_sha" ] || fail "action caller $u" ;;
esac
done
# (d) the negative check, as a cheap extra rather than as the proof.
! git -C "$armed" grep -qE '(uses:|repository:)[[:space:]]*heavy-duty/ceremony' \
-- .github || fail "canonical coordinate still present"
``` ```
**`mapfile`, not a pipeline.** `git grep | while … fail` runs the loop in a Run it against the **pre-arming** tree — that is what enumerates the
subshell, so `fail` exits *that* subshell and the gate continues. Collect carriers that must change — then arm, then check:
first, validate after — and run under `set -euo pipefail`, which the snippet
declares rather than assumes. ```sh
#!/usr/bin/env bash
# check-arming <armed-checkout> <probe-checkout> <fork> <code-sha> <armed-sha> <manifest.tsv>
set -euo pipefail
armed="$1"; probe="$2"; fork="$3"; code_sha="$4"; armed_sha="$5"; manifest="$6"
fail() { echo "arming incomplete: $*" >&2; exit 1; }
# `|| true` on every extraction: git grep exits 1 when nothing matches, and
# under `set -e` that would kill this script BEFORE the comparison — so a
# carrier class that vanished ENTIRELY produced silence instead of a
# refusal. Silence is the worst of the three outcomes; the comparison below
# is what must report it.
actual="$(mktemp)"
{
git -C "$armed" grep -nP '(?<=CEREMONY_SELF_REF: ")[^"]+' -- .github/workflows \
| sed -E 's/^([^:]+):([0-9]+):.*CEREMONY_SELF_REF: "([^"]*)".*/\1:\2\tself_ref\t\3/' || true
git -C "$armed" grep -nE 'repository: .+' -- .github/workflows \
| sed -E 's|^([^:]+):([0-9]+):[[:space:]]*repository:[[:space:]]*(.*)$|\1:\2\t__repo__\t\3|' || true
git -C "$probe" grep -nE 'uses: .+' -- .github \
| sed -E 's|^([^:]+):([0-9]+):[[:space:]]*-?[[:space:]]*uses:[[:space:]]*(.*)$|\1:\2\t__uses__\t\3|' || true
} | sort >"$actual"
# every manifest line must be present with its EXACT expected value, and the
# kinds must match — a role swap changes the kind, not just the value.
while IFS=$'\t' read -r loc kind want; do
case "$kind" in
self_ref) have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="self_ref"{print $3}' "$actual")" ;;
internal_repo|consumer_repo)
have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="__repo__"{print $3}' "$actual")" ;;
workflow_caller|action_caller)
have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="__uses__"{print $3}' "$actual")" ;;
esac
[ -n "$have" ] || fail "carrier vanished: $loc ($kind)"
case "$kind" in
self_ref|internal_repo|consumer_repo)
[ "$have" = "$want" ] || fail "$loc ($kind): expected '$want', found '$have'" ;;
workflow_caller)
[ "$have" = "$fork/.github/workflows/${have##*/.github/workflows/}" ] \
|| fail "$loc: caller owner is not $fork — '$have'"
[ "${have##*@}" = "$armed_sha" ] || fail "$loc: workflow caller not armed sha" ;;
action_caller)
case "$have" in "$fork/actions/"*) ;; *) fail "$loc: caller owner is not $fork — '$have'" ;; esac
[ "${have##*@}" = "$code_sha" ] || fail "$loc: action caller not code sha" ;;
esac
done <"$manifest"
# and nothing UNRECOGNISED: every uses:/repository: in the trees must appear
# in the manifest, so an added carrier is a failure rather than a silence.
while IFS=$'\t' read -r loc _ _; do
grep -qF "$loc"$'\t' "$manifest" || fail "carrier not in manifest: $loc"
done <"$actual"
```
**Why a manifest rather than a longer list of assertions.** The carrier set
is a property of the tree at the moment of arming; any list written into
this document is stale the next time a workflow is added. The manifest is
generated from the tree, recorded in the result issue (step 6), and is the
thing a later reader compares against — so "what was armed" is evidence
rather than recollection.
5. **Invoke the probe by the event it is about**, and record which: a 5. **Invoke the probe by the event it is about**, and record which: a
`workflow_dispatch`, or the real board event under test. A probe that fires `workflow_dispatch`, or the real board event under test. A probe that fires