diff --git a/changelog.d/202.md b/changelog.d/202.md index a7e03c7..3768949 100644 --- a/changelog.d/202.md +++ b/changelog.d/202.md @@ -37,8 +37,9 @@ distinguishes ceremony's internal self-checkouts from the consumer checkouts that must stay `${{ github.repository }}` (#202). -- The published snippet parses, lints clean and runs: driven against an unarmed - tree it refuses, naming the carrier (#202). +- Both published snippets parse, lint clean and were driven against a + constructed armed/probe pair: all six failure classes refuse and the armed + control passes (#202). - Probe results are written to an issue in the probe repo and carried to the ceremony issue by a human, so the probe holds no path that can write to the diff --git a/docs/RUNNER-PROBES.md b/docs/RUNNER-PROBES.md index e03758a..476b271 100644 --- a/docs/RUNNER-PROBES.md +++ b/docs/RUNNER-PROBES.md @@ -113,10 +113,15 @@ So: the **candidate code SHA**. Never create a branch on `heavy-duty/ceremony` named like a tag: it shadows that tag for every consumer until somebody remembers to delete it. -2. **Commit the arming on top of it.** In that same fork branch, rewrite every - workflow carrier — `repository:` → `/ceremony`, and both - `CEREMONY_SELF_REF` values → the **candidate code SHA** from step 1. Record - the resulting SHA. That is the **armed workflow SHA**. +2. **Commit the arming on top of it, and write the manifest.** In that same + fork branch rewrite, for **every** carrier the manifest below enumerates: + ceremony's own internal `repository:` checkouts → `/ceremony`, and + **every** `CEREMONY_SELF_REF` value → the **candidate code SHA** from step 1. + There were three self-ref carriers on `main` at the time of writing and the + count is not a constant — derive it, do not remember it + (@glm-reviewer-andresmgsl, @codex-reviewer-andresmgsl, #202 review). The + **consumer** checkouts (`${{ github.repository }}`) are left alone. Record + the resulting SHA: that is the **armed workflow SHA**. 3. **Pin the probe repo's callers by layer**, because they are not the same thing: - composite-action callers → @@ -124,76 +129,108 @@ So: - reusable-workflow callers → `/ceremony/.github/workflows/@`, since that is the only revision whose inner checkout is rewritten. -4. **Gate the arming POSITIVELY, and against the tree each check is about.** - Absence of the old literal proves nothing: a negative grep stays green if - `CEREMONY_SELF_REF` names a tag or the *armed* SHA, if a carrier went to the - wrong fork, or if a carrier vanished instead of being rewritten. So assert - what each carrier **is** — and enumerate them, because the set is a property - of the current tree rather than a number to remember - (@codex-reviewer-andresmgsl, #202 review). +4. **Gate the arming against a MANIFEST, byte for byte.** Every weaker shape + has a hole, and each of these was found in a published draft of this file + (@codex-reviewer-andresmgsl, #202 review): - **The carriers on `main` today.** Re-derive this list each time rather than - trusting it; workflows come and go: + | weaker check | what slips through | + |---|---| + | "the old literal is absent" | a carrier rewritten to the wrong fork, or to the *armed* SHA | + | "every extracted value equals X" | a carrier that **vanished** — nothing to compare | + | "each value is one of {fork, dynamic}" | a **role swap**: an internal checkout made dynamic, a consumer checkout pointed at the fork | + | "the SHA suffix matches" | `wrong-owner/ceremony/actions/foo@` | + | "known callers match" | an **unrecognised** caller, or none at all | - ```sh - git grep -n 'CEREMONY_SELF_REF:' -- .github/workflows - git grep -n 'repository:' -- .github/workflows - ``` + So the arming step **writes a manifest** — one line per carrier, `path`, + `kind`, `full expected value` — and the gate compares the tree's actual + carriers against it as a set. A deletion, a role swap, a wrong fork, a wrong + SHA, an extra carrier and a missing caller are then all the same kind of + failure: the sets differ. - | carrier | on current `main` | armed value | - |---|---|---| - | `CEREMONY_SELF_REF` | `labels-sweep.yml`, `labels.yml`, `release.yml` | the **candidate code** SHA | - | internal self-checkout | `labels-sweep.yml`, `labels.yml`, `release.yml` ×2 | `/ceremony` | - | **consumer** checkout | `labels-sweep.yml`, `labels.yml`, `release-exercise.yml` | **unchanged** — `${{ github.repository }}` | - - That third row is the one an over-broad rewrite destroys. Those checkouts - fetch the **caller's** repository; pointing them at the fork changes what - the probe exercises into something else entirely. + **Generate it while arming**, from the tree you are arming, so the manifest + cannot drift from the repository: ```sh #!/usr/bin/env bash - # check-arming + # write-manifest set -euo pipefail armed="$1"; probe="$2"; fork="$3"; code_sha="$4"; armed_sha="$5" - fail() { echo "arming incomplete: $*" >&2; exit 1; } - - # (a) every self-ref carrier names the CANDIDATE CODE sha — enumerated, so a - # carrier that vanished is a failure rather than one fewer loop pass. - mapfile -t refs < <(git -C "$armed" grep -hoP '(?<=CEREMONY_SELF_REF: ")[^"]+' \ - -- .github/workflows) - [ "${#refs[@]}" -gt 0 ] || fail "no CEREMONY_SELF_REF carriers found" - for v in "${refs[@]}"; do [ "$v" = "$code_sha" ] || fail "CEREMONY_SELF_REF=$v"; done - - # (b) internal checkouts point at the fork; consumer checkouts stay dynamic. - mapfile -t repos < <(git -C "$armed" grep -hoP '(?<=repository: ).*' \ - -- .github/workflows) - for v in "${repos[@]}"; do - case "$v" in - '${{ github.repository }}') ;; # the caller's repo — must NOT change - "$fork") ;; # an internal self-checkout, armed - *) fail "repository: $v" ;; - esac - done - - # (c) in the PROBE repo: workflow callers pin the armed sha, action callers - # the code sha. - mapfile -t uses < <(git -C "$probe" grep -hoP '(?<=uses: ).*' -- .github) - for u in "${uses[@]}"; do - case "$u" in - */.github/workflows/*@*) [ "${u##*@}" = "$armed_sha" ] || fail "workflow caller $u" ;; - */actions/*@*) [ "${u##*@}" = "$code_sha" ] || fail "action caller $u" ;; - esac - done - - # (d) the negative check, as a cheap extra rather than as the proof. - ! git -C "$armed" grep -qE '(uses:|repository:)[[:space:]]*heavy-duty/ceremony' \ - -- .github || fail "canonical coordinate still present" + { + git -C "$armed" grep -n 'CEREMONY_SELF_REF:' -- .github/workflows \ + | cut -d: -f1,2 | sed "s|\$|\tself_ref\t$code_sha|" + git -C "$armed" grep -n 'repository: heavy-duty/ceremony' -- .github/workflows \ + | cut -d: -f1,2 | sed "s|\$|\tinternal_repo\t$fork|" + git -C "$armed" grep -n 'repository: ${{ github.repository }}' -- .github/workflows \ + | cut -d: -f1,2 | sed 's|$|\tconsumer_repo\t${{ github.repository }}|' + git -C "$probe" grep -n 'uses: .*/.github/workflows/' -- .github \ + | cut -d: -f1,2 | sed "s|\$|\tworkflow_caller\t$armed_sha|" + git -C "$probe" grep -n 'uses: .*/actions/' -- .github \ + | cut -d: -f1,2 | sed "s|\$|\taction_caller\t$code_sha|" + } | sort >manifest.tsv ``` - **`mapfile`, not a pipeline.** `git grep | while … fail` runs the loop in a - subshell, so `fail` exits *that* subshell and the gate continues. Collect - first, validate after — and run under `set -euo pipefail`, which the snippet - declares rather than assumes. + Run it against the **pre-arming** tree — that is what enumerates the + carriers that must change — then arm, then check: + + ```sh + #!/usr/bin/env bash + # check-arming + set -euo pipefail + armed="$1"; probe="$2"; fork="$3"; code_sha="$4"; armed_sha="$5"; manifest="$6" + fail() { echo "arming incomplete: $*" >&2; exit 1; } + + # `|| true` on every extraction: git grep exits 1 when nothing matches, and + # under `set -e` that would kill this script BEFORE the comparison — so a + # carrier class that vanished ENTIRELY produced silence instead of a + # refusal. Silence is the worst of the three outcomes; the comparison below + # is what must report it. + actual="$(mktemp)" + { + git -C "$armed" grep -nP '(?<=CEREMONY_SELF_REF: ")[^"]+' -- .github/workflows \ + | sed -E 's/^([^:]+):([0-9]+):.*CEREMONY_SELF_REF: "([^"]*)".*/\1:\2\tself_ref\t\3/' || true + git -C "$armed" grep -nE 'repository: .+' -- .github/workflows \ + | sed -E 's|^([^:]+):([0-9]+):[[:space:]]*repository:[[:space:]]*(.*)$|\1:\2\t__repo__\t\3|' || true + git -C "$probe" grep -nE 'uses: .+' -- .github \ + | sed -E 's|^([^:]+):([0-9]+):[[:space:]]*-?[[:space:]]*uses:[[:space:]]*(.*)$|\1:\2\t__uses__\t\3|' || true + } | sort >"$actual" + + # every manifest line must be present with its EXACT expected value, and the + # kinds must match — a role swap changes the kind, not just the value. + while IFS=$'\t' read -r loc kind want; do + case "$kind" in + self_ref) have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="self_ref"{print $3}' "$actual")" ;; + internal_repo|consumer_repo) + have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="__repo__"{print $3}' "$actual")" ;; + workflow_caller|action_caller) + have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="__uses__"{print $3}' "$actual")" ;; + esac + [ -n "$have" ] || fail "carrier vanished: $loc ($kind)" + case "$kind" in + self_ref|internal_repo|consumer_repo) + [ "$have" = "$want" ] || fail "$loc ($kind): expected '$want', found '$have'" ;; + workflow_caller) + [ "$have" = "$fork/.github/workflows/${have##*/.github/workflows/}" ] \ + || fail "$loc: caller owner is not $fork — '$have'" + [ "${have##*@}" = "$armed_sha" ] || fail "$loc: workflow caller not armed sha" ;; + action_caller) + case "$have" in "$fork/actions/"*) ;; *) fail "$loc: caller owner is not $fork — '$have'" ;; esac + [ "${have##*@}" = "$code_sha" ] || fail "$loc: action caller not code sha" ;; + esac + done <"$manifest" + + # and nothing UNRECOGNISED: every uses:/repository: in the trees must appear + # in the manifest, so an added carrier is a failure rather than a silence. + while IFS=$'\t' read -r loc _ _; do + grep -qF "$loc"$'\t' "$manifest" || fail "carrier not in manifest: $loc" + done <"$actual" + ``` + + **Why a manifest rather than a longer list of assertions.** The carrier set + is a property of the tree at the moment of arming; any list written into + this document is stale the next time a workflow is added. The manifest is + generated from the tree, recorded in the result issue (step 6), and is the + thing a later reader compares against — so "what was armed" is evidence + rather than recollection. 5. **Invoke the probe by the event it is about**, and record which: a `workflow_dispatch`, or the real board event under test. A probe that fires