forked from heavy-duty/ceremony
docs(runner-probes): the arming gate is a manifest comparison, driven against all six failure classes (#202)
@codex-reviewer-andresmgsl's four holes and @glm-reviewer-andresmgsl's prose
staleness. Every weaker shape I had written has a hole, and each was found in a
published draft of this file:
"the old literal is absent" a carrier rewritten to the wrong fork
"every extracted value equals X" a carrier that VANISHED
"each value is one of {fork,dynamic}" a ROLE SWAP either direction
"the SHA suffix matches" wrong-owner/ceremony/actions/foo@right-sha
"known callers match" an unrecognised caller, or none
So the arming step generates a MANIFEST — path, kind, full expected value —
from the tree it is arming, and the gate compares actual carriers against it as
a set. All six become one kind of failure: the sets differ. Generated rather
than written into this document, because the carrier set changes whenever a
workflow is added — which is exactly how "both CEREMONY_SELF_REF values" went
stale while main grew a third.
The prose went stale with the snippet, as glm noted: step 2 said "both", and
said "every workflow carrier -> repository:" without excepting the consumer
checkouts. Both corrected.
DRIVEN, not asserted. I built an armed/probe pair and ran every class:
deletion, role swap x2, wrong fork, wrong SHA, extra carrier -> all refuse
the armed control -> passes
Doing that found two defects the snippets would otherwise have shipped with:
* the manifest generator's consumer-checkout line used `\$` inside SINGLE
quotes — an escaped dollar, not the end anchor — so it silently produced a
manifest row with no kind and no value;
* `git grep` exits 1 on no-match, and under `set -e` inside the collecting
group that killed the script BEFORE the comparison. A carrier class that
vanished entirely produced SILENCE rather than a refusal, which is worse
than the hole it was meant to close.
test/run.sh 28/28; shellcheck 0.10.0 and changelog-armed clean.
Refs #202
This commit is contained in:
parent
dc87051c69
commit
745944ec8c
2 changed files with 104 additions and 66 deletions
|
|
@ -37,8 +37,9 @@
|
|||
distinguishes ceremony's internal self-checkouts from the consumer checkouts
|
||||
that must stay `${{ github.repository }}` (#202).
|
||||
|
||||
- The published snippet parses, lints clean and runs: driven against an unarmed
|
||||
tree it refuses, naming the carrier (#202).
|
||||
- Both published snippets parse, lint clean and were driven against a
|
||||
constructed armed/probe pair: all six failure classes refuse and the armed
|
||||
control passes (#202).
|
||||
|
||||
- Probe results are written to an issue in the probe repo and carried to the
|
||||
ceremony issue by a human, so the probe holds no path that can write to the
|
||||
|
|
|
|||
|
|
@ -113,10 +113,15 @@ So:
|
|||
the **candidate code SHA**. Never create a branch on
|
||||
`heavy-duty/ceremony` named like a tag: it shadows that tag for every
|
||||
consumer until somebody remembers to delete it.
|
||||
2. **Commit the arming on top of it.** In that same fork branch, rewrite every
|
||||
workflow carrier — `repository:` → `<identity>/ceremony`, and both
|
||||
`CEREMONY_SELF_REF` values → the **candidate code SHA** from step 1. Record
|
||||
the resulting SHA. That is the **armed workflow SHA**.
|
||||
2. **Commit the arming on top of it, and write the manifest.** In that same
|
||||
fork branch rewrite, for **every** carrier the manifest below enumerates:
|
||||
ceremony's own internal `repository:` checkouts → `<identity>/ceremony`, and
|
||||
**every** `CEREMONY_SELF_REF` value → the **candidate code SHA** from step 1.
|
||||
There were three self-ref carriers on `main` at the time of writing and the
|
||||
count is not a constant — derive it, do not remember it
|
||||
(@glm-reviewer-andresmgsl, @codex-reviewer-andresmgsl, #202 review). The
|
||||
**consumer** checkouts (`${{ github.repository }}`) are left alone. Record
|
||||
the resulting SHA: that is the **armed workflow SHA**.
|
||||
3. **Pin the probe repo's callers by layer**, because they are not the same
|
||||
thing:
|
||||
- composite-action callers →
|
||||
|
|
@ -124,76 +129,108 @@ So:
|
|||
- reusable-workflow callers →
|
||||
`<identity>/ceremony/.github/workflows/<file>@<armed-workflow-sha>`, since
|
||||
that is the only revision whose inner checkout is rewritten.
|
||||
4. **Gate the arming POSITIVELY, and against the tree each check is about.**
|
||||
Absence of the old literal proves nothing: a negative grep stays green if
|
||||
`CEREMONY_SELF_REF` names a tag or the *armed* SHA, if a carrier went to the
|
||||
wrong fork, or if a carrier vanished instead of being rewritten. So assert
|
||||
what each carrier **is** — and enumerate them, because the set is a property
|
||||
of the current tree rather than a number to remember
|
||||
(@codex-reviewer-andresmgsl, #202 review).
|
||||
4. **Gate the arming against a MANIFEST, byte for byte.** Every weaker shape
|
||||
has a hole, and each of these was found in a published draft of this file
|
||||
(@codex-reviewer-andresmgsl, #202 review):
|
||||
|
||||
**The carriers on `main` today.** Re-derive this list each time rather than
|
||||
trusting it; workflows come and go:
|
||||
| weaker check | what slips through |
|
||||
|---|---|
|
||||
| "the old literal is absent" | a carrier rewritten to the wrong fork, or to the *armed* SHA |
|
||||
| "every extracted value equals X" | a carrier that **vanished** — nothing to compare |
|
||||
| "each value is one of {fork, dynamic}" | a **role swap**: an internal checkout made dynamic, a consumer checkout pointed at the fork |
|
||||
| "the SHA suffix matches" | `wrong-owner/ceremony/actions/foo@<right-sha>` |
|
||||
| "known callers match" | an **unrecognised** caller, or none at all |
|
||||
|
||||
```sh
|
||||
git grep -n 'CEREMONY_SELF_REF:' -- .github/workflows
|
||||
git grep -n 'repository:' -- .github/workflows
|
||||
```
|
||||
So the arming step **writes a manifest** — one line per carrier, `path`,
|
||||
`kind`, `full expected value` — and the gate compares the tree's actual
|
||||
carriers against it as a set. A deletion, a role swap, a wrong fork, a wrong
|
||||
SHA, an extra carrier and a missing caller are then all the same kind of
|
||||
failure: the sets differ.
|
||||
|
||||
| carrier | on current `main` | armed value |
|
||||
|---|---|---|
|
||||
| `CEREMONY_SELF_REF` | `labels-sweep.yml`, `labels.yml`, `release.yml` | the **candidate code** SHA |
|
||||
| internal self-checkout | `labels-sweep.yml`, `labels.yml`, `release.yml` ×2 | `<identity>/ceremony` |
|
||||
| **consumer** checkout | `labels-sweep.yml`, `labels.yml`, `release-exercise.yml` | **unchanged** — `${{ github.repository }}` |
|
||||
|
||||
That third row is the one an over-broad rewrite destroys. Those checkouts
|
||||
fetch the **caller's** repository; pointing them at the fork changes what
|
||||
the probe exercises into something else entirely.
|
||||
**Generate it while arming**, from the tree you are arming, so the manifest
|
||||
cannot drift from the repository:
|
||||
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
# check-arming <armed-checkout> <probe-checkout> <fork> <code-sha> <armed-sha>
|
||||
# write-manifest <armed-checkout> <probe-checkout> <fork> <code-sha> <armed-sha>
|
||||
set -euo pipefail
|
||||
armed="$1"; probe="$2"; fork="$3"; code_sha="$4"; armed_sha="$5"
|
||||
fail() { echo "arming incomplete: $*" >&2; exit 1; }
|
||||
|
||||
# (a) every self-ref carrier names the CANDIDATE CODE sha — enumerated, so a
|
||||
# carrier that vanished is a failure rather than one fewer loop pass.
|
||||
mapfile -t refs < <(git -C "$armed" grep -hoP '(?<=CEREMONY_SELF_REF: ")[^"]+' \
|
||||
-- .github/workflows)
|
||||
[ "${#refs[@]}" -gt 0 ] || fail "no CEREMONY_SELF_REF carriers found"
|
||||
for v in "${refs[@]}"; do [ "$v" = "$code_sha" ] || fail "CEREMONY_SELF_REF=$v"; done
|
||||
|
||||
# (b) internal checkouts point at the fork; consumer checkouts stay dynamic.
|
||||
mapfile -t repos < <(git -C "$armed" grep -hoP '(?<=repository: ).*' \
|
||||
-- .github/workflows)
|
||||
for v in "${repos[@]}"; do
|
||||
case "$v" in
|
||||
'${{ github.repository }}') ;; # the caller's repo — must NOT change
|
||||
"$fork") ;; # an internal self-checkout, armed
|
||||
*) fail "repository: $v" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# (c) in the PROBE repo: workflow callers pin the armed sha, action callers
|
||||
# the code sha.
|
||||
mapfile -t uses < <(git -C "$probe" grep -hoP '(?<=uses: ).*' -- .github)
|
||||
for u in "${uses[@]}"; do
|
||||
case "$u" in
|
||||
*/.github/workflows/*@*) [ "${u##*@}" = "$armed_sha" ] || fail "workflow caller $u" ;;
|
||||
*/actions/*@*) [ "${u##*@}" = "$code_sha" ] || fail "action caller $u" ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# (d) the negative check, as a cheap extra rather than as the proof.
|
||||
! git -C "$armed" grep -qE '(uses:|repository:)[[:space:]]*heavy-duty/ceremony' \
|
||||
-- .github || fail "canonical coordinate still present"
|
||||
{
|
||||
git -C "$armed" grep -n 'CEREMONY_SELF_REF:' -- .github/workflows \
|
||||
| cut -d: -f1,2 | sed "s|\$|\tself_ref\t$code_sha|"
|
||||
git -C "$armed" grep -n 'repository: heavy-duty/ceremony' -- .github/workflows \
|
||||
| cut -d: -f1,2 | sed "s|\$|\tinternal_repo\t$fork|"
|
||||
git -C "$armed" grep -n 'repository: ${{ github.repository }}' -- .github/workflows \
|
||||
| cut -d: -f1,2 | sed 's|$|\tconsumer_repo\t${{ github.repository }}|'
|
||||
git -C "$probe" grep -n 'uses: .*/.github/workflows/' -- .github \
|
||||
| cut -d: -f1,2 | sed "s|\$|\tworkflow_caller\t$armed_sha|"
|
||||
git -C "$probe" grep -n 'uses: .*/actions/' -- .github \
|
||||
| cut -d: -f1,2 | sed "s|\$|\taction_caller\t$code_sha|"
|
||||
} | sort >manifest.tsv
|
||||
```
|
||||
|
||||
**`mapfile`, not a pipeline.** `git grep | while … fail` runs the loop in a
|
||||
subshell, so `fail` exits *that* subshell and the gate continues. Collect
|
||||
first, validate after — and run under `set -euo pipefail`, which the snippet
|
||||
declares rather than assumes.
|
||||
Run it against the **pre-arming** tree — that is what enumerates the
|
||||
carriers that must change — then arm, then check:
|
||||
|
||||
```sh
|
||||
#!/usr/bin/env bash
|
||||
# check-arming <armed-checkout> <probe-checkout> <fork> <code-sha> <armed-sha> <manifest.tsv>
|
||||
set -euo pipefail
|
||||
armed="$1"; probe="$2"; fork="$3"; code_sha="$4"; armed_sha="$5"; manifest="$6"
|
||||
fail() { echo "arming incomplete: $*" >&2; exit 1; }
|
||||
|
||||
# `|| true` on every extraction: git grep exits 1 when nothing matches, and
|
||||
# under `set -e` that would kill this script BEFORE the comparison — so a
|
||||
# carrier class that vanished ENTIRELY produced silence instead of a
|
||||
# refusal. Silence is the worst of the three outcomes; the comparison below
|
||||
# is what must report it.
|
||||
actual="$(mktemp)"
|
||||
{
|
||||
git -C "$armed" grep -nP '(?<=CEREMONY_SELF_REF: ")[^"]+' -- .github/workflows \
|
||||
| sed -E 's/^([^:]+):([0-9]+):.*CEREMONY_SELF_REF: "([^"]*)".*/\1:\2\tself_ref\t\3/' || true
|
||||
git -C "$armed" grep -nE 'repository: .+' -- .github/workflows \
|
||||
| sed -E 's|^([^:]+):([0-9]+):[[:space:]]*repository:[[:space:]]*(.*)$|\1:\2\t__repo__\t\3|' || true
|
||||
git -C "$probe" grep -nE 'uses: .+' -- .github \
|
||||
| sed -E 's|^([^:]+):([0-9]+):[[:space:]]*-?[[:space:]]*uses:[[:space:]]*(.*)$|\1:\2\t__uses__\t\3|' || true
|
||||
} | sort >"$actual"
|
||||
|
||||
# every manifest line must be present with its EXACT expected value, and the
|
||||
# kinds must match — a role swap changes the kind, not just the value.
|
||||
while IFS=$'\t' read -r loc kind want; do
|
||||
case "$kind" in
|
||||
self_ref) have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="self_ref"{print $3}' "$actual")" ;;
|
||||
internal_repo|consumer_repo)
|
||||
have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="__repo__"{print $3}' "$actual")" ;;
|
||||
workflow_caller|action_caller)
|
||||
have="$(awk -F'\t' -v l="$loc" '$1==l && $2=="__uses__"{print $3}' "$actual")" ;;
|
||||
esac
|
||||
[ -n "$have" ] || fail "carrier vanished: $loc ($kind)"
|
||||
case "$kind" in
|
||||
self_ref|internal_repo|consumer_repo)
|
||||
[ "$have" = "$want" ] || fail "$loc ($kind): expected '$want', found '$have'" ;;
|
||||
workflow_caller)
|
||||
[ "$have" = "$fork/.github/workflows/${have##*/.github/workflows/}" ] \
|
||||
|| fail "$loc: caller owner is not $fork — '$have'"
|
||||
[ "${have##*@}" = "$armed_sha" ] || fail "$loc: workflow caller not armed sha" ;;
|
||||
action_caller)
|
||||
case "$have" in "$fork/actions/"*) ;; *) fail "$loc: caller owner is not $fork — '$have'" ;; esac
|
||||
[ "${have##*@}" = "$code_sha" ] || fail "$loc: action caller not code sha" ;;
|
||||
esac
|
||||
done <"$manifest"
|
||||
|
||||
# and nothing UNRECOGNISED: every uses:/repository: in the trees must appear
|
||||
# in the manifest, so an added carrier is a failure rather than a silence.
|
||||
while IFS=$'\t' read -r loc _ _; do
|
||||
grep -qF "$loc"$'\t' "$manifest" || fail "carrier not in manifest: $loc"
|
||||
done <"$actual"
|
||||
```
|
||||
|
||||
**Why a manifest rather than a longer list of assertions.** The carrier set
|
||||
is a property of the tree at the moment of arming; any list written into
|
||||
this document is stale the next time a workflow is added. The manifest is
|
||||
generated from the tree, recorded in the result issue (step 6), and is the
|
||||
thing a later reader compares against — so "what was armed" is evidence
|
||||
rather than recollection.
|
||||
|
||||
5. **Invoke the probe by the event it is about**, and record which: a
|
||||
`workflow_dispatch`, or the real board event under test. A probe that fires
|
||||
|
|
|
|||
Loading…
Reference in a new issue