Found by combining all five open PRs and running the suite on the result —
which is the check this PR's own runbook adds, catching a real break the first
time it was applied at scale.
!203 (#201) makes actions/docs-sync/docs-sync.sh decide the forge from
GITHUB_SERVER_URL, because it was fetching the doctrine mirror from a
hard-coded github.com. This PR's guard requires every forge-deciding file to be
named in the inventory. Both are individually green; together the tree is red:
forge-specific but not in docs/UPSTREAM-SYNC.md:
actions/docs-sync/docs-sync.sh
The entry belongs here rather than in !203: the inventory is this PR's artifact,
and !203 is a bug fix that should not have to know about a guard absent from
its base. Adding it early is harmless — the guard checks that deciding files
ARE listed, not that listed files decide — and correct the moment both land.
Five-way combined tree after this: 28 test files 0 failed under the runner's
jq 1.6, shellcheck 0.10.0, actionlint, self-ref, marker, vendored and
changelog-armed all clean.
Refs #200
@codex-reviewer-andresmgsl reproduced it again, with one file: scanned_paths()
said "tracked" and used `find`, which walks the working directory and knows
nothing about the index.
Not pedantry — ci.yml extracts shellcheck.tar.xz, actionlint.tar.gz and their
binaries INTO the checkout before the suite runs, and any developer cache sits
there too. Today none happens to carry a matching marker; that is luck, not a
property, and a false red on a downloaded tarball would be indistinguishable
from a real finding.
`git ls-files -z` makes "tracked" executable rather than prose.
The fixtures become tiny git repositories, because a fixture that is only a
directory is invisible to ls-files and every must-fail below it would have
passed vacuously — the same trap as the earlier teeth that never invoked the
guard. Plus the negative case he asked for: an untracked marker-bearing cache
file is ignored, and the moment it is TRACKED the guard sees it.
Reverting discovery to find reds three.
Branch updated from merged main (e236318, now carrying !206) before verifying:
upstream-delta 28/28, test/run.sh 29/29, shellcheck 0.10.0 clean.
Refs #200
@codex-reviewer-andresmgsl did not argue this one, he reproduced it: an
`actions/*/action.yml` declaring CEREMONY_FORGE_CLIENT and a workflow written
`.yaml` rather than `.yml`, both invisible to the hand-picked globs, guard
still 21/21 green.
The first is not an edge case — `actions/*/action.yml` is this repository's
normal composite structure and a client declaration there IS a forge decision.
The second shows `*.yml` was never a complete workflow surface.
So discovery walks the tree and EXCLUDES by class rather than enumerating
directories, depths and extensions. Excluding is the safer default: a new file
type arrives scanned rather than invisible. Out of scope are .git/, test/
(whose harness asserts these tokens by design), changelog.d/ and *.md — prose,
including drills/, which stays in the inventory because its records are
forge-specific by CONTENT while a record mentioning a selector verb in prose is
not a decision.
Both of his reproductions are now fixtures driving the real no_unlisted, and
restricting discovery back to *.sh reds four cases.
The documentation claim is aligned with what the guard does rather than what
the table implies: it checks forge DECISIONS in executable and configuration
files; it is not a diff against upstream, so drills/ and labels.conf are listed
by judgement rather than found by scan. Saying otherwise made labels.conf and
drills/ look like evidence of completeness while action.yml was invisible.
upstream-delta 24/24; test/run.sh 29/29; shellcheck 0.10.0, actionlint and
changelog-armed clean.
Refs #200
@codex-reviewer-andresmgsl's five points. Three were correctness, and one of
them found that my must-fail cases could not fail.
1. THE OBJECT IS MANDATORY. UNVERIFIABLE-HERE is gone: a missing ref, an absent
object and a non-ancestor are three distinct refusals. The ref is now the
FULL 40-character SHA, and ci.yml fetches exactly that object before the
suite. "Runs offline" means the TEST reads local evidence; it never meant CI
may omit the evidence and pass.
2. THE SCAN COVERS WHAT THE INVENTORY CLAIMS. It walked shell under four globs
and never looked at workflows, .github/labels.conf or drills/ — three
categories the inventory governs. Widened, and it immediately found four
real blind spots on merged main: refs-not-closing's declaration,
labels.yml's inline forge decision, refs-guard.yml's GitHub-only scheduling
and release-exercise.yml's pinned CEREMONY_FORGE. All four are now inventory
entries with the issue that removes them, because a delta location with no
exit is indistinguishable from one nobody noticed. A file that DECLARES a
client is no longer exempt as a "consumer" — only files that merely CALL the
shim are.
3. THE TEETH NOW DRIVE THE GUARD. They asserted the predicates separately and
never invoked no_unlisted, so the guard could have been `return 0` and both
must-fail rows would still have passed. SCAN_ROOT is a parameter now and the
cases build a tree, add an unlisted decider — shell AND workflow, so
coverage cannot regress to the old glob — and assert the real top-level
check fails naming it. Replacing no_unlisted with `return 0` reds five.
4. PATH MATCHING, NOT PREFIX MATCHING. `drills/` accepted `drills-old/x` and
`lib/forge.sh` accepted `lib/forge.sh.backup`. Exact for files, `dir/` for
directories, with both negative boundaries covered.
5. THE IMMUTABLE SHA IS CAPTURED AT FETCH. The runbook now takes
upstream_sha=$(git rev-parse gh/main) once and merges and records that
value. This is not hypothetical: while this PR was in review upstream moved
from 8c3a4d1 to 08e2912, and re-reading gh/main at recording time would have
written a commit this tree does not contain. I caught that by walking into
it.
test/run.sh 29/29; upstream-delta 21/21; shellcheck 0.10.0, actionlint,
changelog-armed clean.
Refs #200
@codex-reviewer-andresmgsl's two additions (#5697), both measured in the #198
sync rather than anticipated.
Every branch open across a sync is stale afterwards: Forgejo never re-tests an
open PR when main moves under it, so #206 and #207's green 22-file suites were
about a tree that no longer existed once the 28-file one landed — and #206's
fragment was individually green while making the combined tree red under a rule
the sync itself introduces. The runbook now says to update each in-flight
branch from the newly synced main, or check them in a scratch merge, and that a
prior approval is evidence about the tree it was given on.
And post-merge runs are audited by executed steps, never by colour: inventory
what the sync changed about triggers and jobs, read which job actually ran, and
treat a green refusal path as evidence for that path only. Run 326 was green
and had reconciled nothing.
Both failures happened with the no-runtime-gh guard green and CI green, so the
runbook says that too.
Refs #200
The third child of #197, written immediately after performing the sync it
describes, while the findings are still first-hand.
docs/UPSTREAM-SYNC.md carries the procedure and the six standing resolutions,
each with the issue that decided it, so they are not re-argued every sync. The
parts that are not obvious from the outside, and that the 0.6.0 sync paid to
learn:
* THE AUDIT STEP. `git merge` takes upstream's side wherever only upstream
moved a region, so a function upstream ADDED to a file this tree owns
arrives with no conflict and no question. Reviewing the hunks cannot find
it — four reviewers read the same diff and each found a different subset.
That was eight runtime `gh` call sites in three files and two file types.
* THE SAME MECHANIC APPLIES TO STATE. A resolved region can remove a producer
whose consumers auto-merged, and those consumers degrade to empty rather
than erroring, so nothing goes red. Three such seams in one sync.
* VERIFY WHERE IT RUNS. "Green locally" was wrong three times, for three
different reasons: shellcheck-all lints TRACKED files so a new file's first
lint is meaningless; CI pins shellcheck 0.10.0; and the runner's jq 1.6
exits 0 where 1.7 exits 4 on `jq -e` with empty input — which was not a
test problem but a guard accepting an unreadable read.
* TEST THE MERGE RESULT. Forgejo tests heads, never what two branches produce
together, and two green PRs did produce a red tree in this sync.
* AFTER MERGING, CHECK THE SWEEP RECONCILED SOMETHING. The first post-merge
run was green and had done nothing.
.upstream-ref records the carried commit in machine-readable form beside the
CHANGELOG's prose. test/upstream-delta.test.sh asserts every forge-DECIDING
file is named in the inventory — offline, comment-aware, and refusing rather
than skipping when the ref is missing. Shim CONSUMERS are allowed by name, so
a seventh consumer is silent and a seventh decider is not.
docs/CONSUMERS.md now states that two ceremonies answer to the same version
number and how a consumer says which one it pinned.
Must-fail, both from the issue's test plan: scattering a forge_detect branch
into an unlisted file reds the guard; blanking .upstream-ref reds it too.
test/run.sh 29 files 0 failed under jq 1.7 and jq 1.6; shellcheck 0.10.0,
actionlint, self-ref, marker, vendored and changelog-armed all clean.
Refs #200
