docs: the recurring upstream sync, its standing resolutions, and a guard on where the delta lives (#200)

The third child of #197, written immediately after performing the sync it
describes, while the findings are still first-hand.

docs/UPSTREAM-SYNC.md carries the procedure and the six standing resolutions,
each with the issue that decided it, so they are not re-argued every sync. The
parts that are not obvious from the outside, and that the 0.6.0 sync paid to
learn:

  * THE AUDIT STEP. `git merge` takes upstream's side wherever only upstream
    moved a region, so a function upstream ADDED to a file this tree owns
    arrives with no conflict and no question. Reviewing the hunks cannot find
    it — four reviewers read the same diff and each found a different subset.
    That was eight runtime `gh` call sites in three files and two file types.

  * THE SAME MECHANIC APPLIES TO STATE. A resolved region can remove a producer
    whose consumers auto-merged, and those consumers degrade to empty rather
    than erroring, so nothing goes red. Three such seams in one sync.

  * VERIFY WHERE IT RUNS. "Green locally" was wrong three times, for three
    different reasons: shellcheck-all lints TRACKED files so a new file's first
    lint is meaningless; CI pins shellcheck 0.10.0; and the runner's jq 1.6
    exits 0 where 1.7 exits 4 on `jq -e` with empty input — which was not a
    test problem but a guard accepting an unreadable read.

  * TEST THE MERGE RESULT. Forgejo tests heads, never what two branches produce
    together, and two green PRs did produce a red tree in this sync.

  * AFTER MERGING, CHECK THE SWEEP RECONCILED SOMETHING. The first post-merge
    run was green and had done nothing.

.upstream-ref records the carried commit in machine-readable form beside the
CHANGELOG's prose. test/upstream-delta.test.sh asserts every forge-DECIDING
file is named in the inventory — offline, comment-aware, and refusing rather
than skipping when the ref is missing. Shim CONSUMERS are allowed by name, so
a seventh consumer is silent and a seventh decider is not.

docs/CONSUMERS.md now states that two ceremonies answer to the same version
number and how a consumer says which one it pinned.

Must-fail, both from the issue's test plan: scattering a forge_detect branch
into an unlisted file reds the guard; blanking .upstream-ref reds it too.

test/run.sh 29 files 0 failed under jq 1.7 and jq 1.6; shellcheck 0.10.0,
actionlint, self-ref, marker, vendored and changelog-armed all clean.

Refs #200

.upstream-ref

@@ -0,0 +1,4 @@
+# The upstream commit this tree carries (docs/UPSTREAM-SYNC.md).
+# Updated by each sync; read by test/upstream-delta.test.sh.
+# github.com/heavy-duty/ceremony
+8c3a4d1

changelog.d/200.md

@@ -0,0 +1,31 @@
+### Added
+
+- `docs/UPSTREAM-SYNC.md` — the recurring upstream sync as a runbook: the
+  standing resolutions, which side wins each and the issue that decided it
+  (#200).
+
+- It names the step the 0.6.0 sync nearly shipped without: auditing what the
+  merge brought in that did **not** conflict. `git merge` asks no question
+  about a function upstream added to a file this tree owns (#200).
+
+- It records that the same mechanic applies to state, not just to call sites: a
+  resolved region can silently remove a producer whose consumers auto-merged,
+  and every one of those consumers degrades to empty rather than erroring
+  (#200).
+
+- It says to verify with the runner's tooling, because "green locally" was
+  wrong three times in one sync — untracked files, a pinned linter, and a
+  pinned `jq` whose empty-input exit code differs (#200).
+
+- It says to test the merge RESULT when other PRs are open: Forgejo tests
+  heads, so two green branches can produce a red tree (#200).
+
+- `.upstream-ref` records the upstream commit this tree carries, in
+  machine-readable form beside the CHANGELOG's prose (#200).
+
+- `test/upstream-delta.test.sh` fails the PR that scatters forge branching into
+  a file the inventory does not name — offline, and refusing rather than
+  skipping when the recorded ref is missing (#200).
+
+- `docs/CONSUMERS.md` states that two ceremonies answer to the same version
+  number, and how a consumer says which one it pinned (#200).

docs/CONSUMERS.md

@@ -687,6 +687,26 @@ Bumping the pin re-syncs the mirror in the same PR —
 
 ## Version pinning
 
+**Two ceremonies answer to the same version number.** `heavy-duty/ceremony`
+exists on GitHub and on `forgejo.heavyduty.builders`, and the forge tree tracks
+upstream's version numbers deliberately (ceremony#197 D2) — so `0.6.0` names a
+different tree on each, differing by the forge-compatibility delta. They are
+not forks that drifted: the forge tree carries upstream's content and adds to
+it (`docs/UPSTREAM-SYNC.md`).
+
+What that means for a consumer:
+
+- **Name the forge you pinned, not just the tag.** `heavy-duty/ceremony@0.6.0`
+  is ambiguous on its own; the host in your `uses:` line is what disambiguates
+  it, so do not describe your pin anywhere without it.
+- **A tag that exists upstream may not exist here yet.** The forge tree's
+  `CEREMONY_SELF_REF` takes upstream's number as soon as the sync lands, which
+  is *before* the release ceremony cuts that tag here. Do not bump a pin to a
+  version whose tag you have not confirmed on the forge you consume from.
+- **The forge tree's `CHANGELOG.md` header names the upstream commit it
+  carries**, and `.upstream-ref` records the same SHA. That is how you tell
+  which `0.6.0` you are actually running.
+
 - **Pin an exact ceremony release tag** — `@0.1.0`, never a branch and
   never a moving major pointer: the family pins things and reviews
   updates ([#1 D2](https://github.com/heavy-duty/ceremony/issues/1)).

docs/UPSTREAM-SYNC.md

@@ -0,0 +1,192 @@
+# Syncing this tree with upstream ceremony
+
+`heavy-duty/ceremony` exists on two forges and they diverge in opposite
+directions on purpose:
+
+- **upstream** — `github.com/heavy-duty/ceremony`, where new ceremony features
+  are written. **Read-only from here.** No issue, PR, comment, review or
+  release is ever created there.
+- **this tree** — `forgejo.heavyduty.builders/heavy-duty/ceremony`, which
+  carries upstream's content plus the forge-compatibility delta and never
+  writes back.
+
+This document is the procedure for bringing upstream's work across. It is
+written to be followed without prior context; where it states a resolution, the
+resolution is standing and does not get re-decided each sync.
+
+Worked example throughout: the `0.6.0` sync (#197, #198), which merged upstream
+`8c3a4d1` onto `dad99dd` and took four heads to get green.
+
+## The standing resolutions
+
+These recur every sync. They are decided; re-deciding them is the cost this
+list exists to remove.
+
+| what | which side wins | decided by |
+|---|---|---|
+| `VERSION` | **upstream** — this tree tracks upstream's version numbers | #197 D2 |
+| `CEREMONY_SELF_REF` (both carriers) | **upstream** | #197 D2 |
+| `.github/labels.conf` | **this tree** — upstream's roster names identities that do not exist here | #195 |
+| `drills/*.md` | **this tree** — a drill record is a record of a run *here* | #198 |
+| `CHANGELOG.md` | **both**, upstream's new sections above this tree's | #198 |
+| a section for a version **both** trees released | **this tree's** — ours is the published body of the tag that exists here | #198 |
+
+Two consequences worth stating plainly:
+
+- **Two trees answer to the same version number**, differing by the forge
+  delta. That is accepted, not accidental (#197 D2). The mitigation is
+  provenance in prose: `CHANGELOG.md`'s header names the upstream commit this
+  tree carries, and `.upstream-ref` records it in machine-readable form.
+- **A tag that exists upstream may not exist here.** `CEREMONY_SELF_REF` takes
+  upstream's number, and both workflows carry the self-consumption bypass
+  (`if: github.repository != 'heavy-duty/ceremony'`), so ceremony's own CI is
+  unaffected. But **no consumer may bump its pin to that number until the
+  release ceremony cuts the tag here.**
+
+## The procedure
+
+### 1. Add the upstream remote, read-only, and confirm the merge base
+
+```sh
+git remote add gh https://github.com/heavy-duty/ceremony.git   # if absent
+git fetch gh
+git merge-base main gh/main
+```
+
+**Confirm the merge base against `.upstream-ref` before merging anything.** If
+it is not what the last sync recorded, something moved — stop and re-measure
+rather than proceeding. A sync that starts from an unexpected base is a sync
+whose conflict count means nothing.
+
+### 2. Merge, never rebase
+
+```sh
+git merge gh/main
+```
+
+One merge commit, conflicts resolved once (#197 D1). Rebasing the forge-only
+commits onto upstream would rewrite every SHA, re-resolve the same conflicts
+once per commit, and break any pin to them. A fresh re-import would discard the
+provenance in this repo's issue comments, which is where its documentation
+actually lives.
+
+### 3. Resolve the conflicts
+
+Apply the standing resolutions above. What is left is genuinely new and needs
+judgement — in the `0.6.0` sync that was 5 hunks of 18.
+
+### 4. Audit what the merge brought in that did NOT conflict
+
+**This is the step the `0.6.0` sync nearly shipped without, and the one this
+document exists for.**
+
+`git merge` takes upstream's side wherever only upstream moved a region. So a
+function upstream *added* to a file this tree already owns arrives with **no
+conflict and no question asked**. Reviewing the conflict hunks cannot find
+them: four reviewers read the same diff and each found a different subset.
+
+In the `0.6.0` sync that was **eight** runtime `gh` call sites, in three files
+and two file types, every one of which #188 had previously removed.
+
+So, after resolving:
+
+```sh
+bash test/no-runtime-gh.test.sh
+```
+
+That guard is the mechanical form of #197's acceptance bar — no runtime `gh`
+outside `lib/forge-github.sh` unless the file declares
+`CEREMONY_FORGE_CLIENT=gh` **and** refuses when it cannot run. Do not satisfy
+it by adding an exemption; a declaration without a refusal is a permission slip
+for `gh: command not found`.
+
+Then check the **variables** the same way, because the same mechanic applies to
+state: if a conflicted region assigns something that auto-merged code consumes,
+resolving it "to this tree's side" silently removes the producer. Every one of
+those consumers degrades to empty rather than erroring, so nothing goes red.
+The `0.6.0` sync had three such seams. Enumerate what each resolved region
+assigns, and confirm each still has a producer.
+
+### 5. Port or declare every new `gh` call site
+
+Where a `forge_*` verb exists, port it in the merge itself. Where none does,
+the file **declares** `CEREMONY_FORGE_CLIENT=gh` and refuses loudly, and the
+port gets its own issue (#199 for `refs-not-closing`, #205 for the sweep
+dispatch). "Never 'probably github'" applies to the sync as much as to a
+runtime probe.
+
+A workflow cannot call `forge_preflight`, so it declares in its `env:` block
+and refuses inline — deciding the **forge** first and the **binary** second. A
+guard that only asks whether `gh` is installed passes the moment a runner image
+ships it.
+
+### 6. Record the provenance
+
+- `CHANGELOG.md`'s header: which upstream commit this tree now carries.
+- `.upstream-ref`: the same SHA, machine-readable, checked by
+  `test/upstream-delta.test.sh`.
+- A `changelog.d/` fragment for the sync issue.
+
+### 7. Verify — and verify where it will actually run
+
+`test/run.sh` green on your machine is the weakest of the checks below. The
+`0.6.0` sync was "green locally" and red on the runner **three times, for three
+different reasons**:
+
+| what was green locally | why the runner disagreed |
+|---|---|
+| `shellcheck-all.sh` | it lints **tracked** files, and the new guard was untracked |
+| the whole suite | CI pins **shellcheck 0.10.0**; a different local version reports differently |
+| `issue_payload_valid` | `jq -e` on empty input exits **4** on jq 1.7 and **0** on jq 1.6 — and the runner image ships 1.6 |
+
+That last one was not a test problem: on jq 1.6 the guard that refuses an
+unreadable read was *accepting* one. **The distance between your environment
+and the runner's is part of the sync's risk surface, not an inconvenience.**
+
+So verify with the runner's own tooling:
+
+```sh
+git add -A                                        # or shellcheck sees nothing new
+CEREMONY_REQUIRE_NPM=1 CEREMONY_REQUIRE_YQ=1 bash test/run.sh
+bash .github/scripts/shellcheck-all.sh            # pinned 0.10.0, as ci.yml installs
+bash .github/scripts/actionlint-all.sh
+bash .github/scripts/self-ref-check.sh
+bash .github/scripts/marker-check.sh
+bash .github/scripts/vendored-check.sh
+bash actions/changelog-armed/changelog-armed.sh
+```
+
+and run the suite once under the runner's `jq` as well as your own.
+
+**And check the merge result, not just the head.** Forgejo tests branch heads;
+it never tests what two branches produce together. Two PRs can each be green
+and their merge red — that happened in this sync, because a rule the sync
+*introduces* (#262's terminal citation) was not met by a fragment written
+against the base that lacks it. If other PRs are open, merge them together
+locally and run the suite on the result.
+
+### 8. After it merges
+
+The sync issue uses `Refs`, not `Closes`, and stays open until a real sweep on
+the merged `main` is linked to it. A green workflow run is not that evidence by
+itself: verify the run actually **reconciled** something. In this sync the
+first post-merge run was green and had done nothing at all, because upstream's
+restructure moved reconcile behind a dispatch this forge cannot perform.
+
+## Where the forge delta lives
+
+Forge-specific behaviour is confined to the files below. Keeping it there is
+what makes each sync cost 18 hunks instead of hundreds, and
+`test/upstream-delta.test.sh` fails the PR that scatters it into a new file.
+
+| file | what is forge-specific about it |
+|---|---|
+| `lib/forge.sh` | the selector: `forge_detect`, `forge_client`, `forge_preflight` |
+| `lib/forge-github.sh` | the gh backend — the one file allowed to speak `gh` |
+| `lib/forge-forgejo.sh` | the Forgejo backend, `/api/v1` over curl + jq |
+| `lib/closes_references.sh` | the closing-keyword parser that replaced GraphQL |
+| `.github/labels.conf` | this instance's roster |
+| `drills/` | records of runs on this instance |
+
+If a sync needs forge branching somewhere else, that is a design decision, not
+a detail: add the file to the inventory in the same PR, with the reason.

test/upstream-delta.test.sh

@@ -0,0 +1,164 @@
+#!/usr/bin/env bash
+# The delta-inventory guard (#200, enforcing #197 D3's standing constraint).
+#
+#   Forge-specific behaviour lives in the files docs/UPSTREAM-SYNC.md names,
+#   and nowhere else.
+#
+# WHY. This tree carries upstream's content plus a forge delta, forever — the
+# sync is recurring and upstream is read-only (#197 D3). What makes that
+# tractable is not luck: the 0.6.0 sync cost 18 conflict hunks across 10 files
+# because the delta sits in six files upstream never touches. Scatter
+# `forge_detect` into a seventh and every future sync pays for it, in a place
+# nobody chose.
+#
+# So this fails on the PR that scatters it, rather than on the sync after next
+# — the same reason changelog-monotonic guards a property no single tree can
+# be asked about.
+#
+# IT RUNS OFFLINE, AND IT DOES NOT PASS BY ABSENCE. Tests must not need the
+# network, and a guard that goes green when it cannot see its input is the
+# blind-sweep shape this repo keeps writing issues about. So the inventory is
+# read from the document and the tree is scanned for forge markers — both local
+# — and a missing or empty `.upstream-ref` is a REFUSAL, not a skip.
+set -u
+
+ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
+# shellcheck source=test/harness.sh
+. "$ROOT/test/harness.sh"
+
+DOC="$ROOT/docs/UPSTREAM-SYNC.md"
+REF="$ROOT/.upstream-ref"
+
+# --- the recorded upstream ref ------------------------------------------------
+
+check "the sync document exists" 0 "" test -f "$DOC"
+check "the upstream ref is recorded" 0 "" test -f "$REF"
+
+recorded_ref() { grep -vE '^[[:space:]]*(#|$)' "$REF" | head -n1; }
+ref_is_recorded() { [ -n "$(recorded_ref)" ]; }
+check "...and is not blank — an unrecorded ref is a refusal, not a skip" 0 "" \
+  ref_is_recorded
+ref_looks_like_sha() { printf '%s' "$(recorded_ref)" | grep -qE '^[0-9a-f]{7,40}$'; }
+check "...and looks like a commit SHA" 0 "" ref_looks_like_sha
+
+# The ancestry half needs upstream's objects, which a CI clone of THIS repo
+# does not have. It is therefore conditional — but it says so out loud rather
+# than passing quietly, because "could not check" and "checked and fine" are
+# the two states this repo exists to keep apart.
+ancestry_state() {
+  if git -C "$ROOT" cat-file -e "$(recorded_ref)^{commit}" 2>/dev/null; then
+    git -C "$ROOT" merge-base --is-ancestor "$(recorded_ref)" HEAD 2>/dev/null \
+      && echo ANCESTOR || echo NOT-ANCESTOR
+  else
+    echo UNVERIFIABLE-HERE
+  fi
+}
+state="$(ancestry_state)"
+printf 'upstream-delta: recorded ref %s is %s\n' "$(recorded_ref)" "$state"
+ancestry_acceptable() { [ "$state" = ANCESTOR ] || [ "$state" = UNVERIFIABLE-HERE ]; }
+check "the recorded ref is an ancestor of HEAD, or is honestly reported as unverifiable" 0 "" \
+  ancestry_acceptable
+
+# --- the inventory ------------------------------------------------------------
+
+# The document's inventory table: the first `code` span of each table row in
+# the "Where the forge delta lives" section.
+inventory() {
+  awk '/^## Where the forge delta lives/ { on = 1; next }
+       on && /^## / { on = 0 }
+       on && /^\| `/ { gsub(/^\| `/, ""); sub(/`.*$/, ""); print }' "$DOC"
+}
+
+inventory_size() { inventory | wc -l; }
+inventory_is_populated() { [ "$(inventory_size)" -ge 6 ]; }
+check "the inventory names at least the six known delta files" 0 "" \
+  inventory_is_populated
+
+in_inventory() { # $1 = repo-relative path
+  local entry
+  while IFS= read -r entry; do
+    [ -n "$entry" ] || continue
+    case "$1" in "$entry" | "$entry"*) return 0 ;; esac
+  done < <(inventory)
+  return 1
+}
+check "a listed file is recognised" 0 "" in_inventory lib/forge.sh
+check "...including one listed as a directory" 0 "" in_inventory drills/0.4.1.md
+check "an unlisted file is not" 1 "" in_inventory lib/version.sh
+
+# --- the scan -----------------------------------------------------------------
+
+# What "forge-specific" means, mechanically: the selector's own verbs and the
+# environment override that drives them. Comments are stripped — this tree
+# documents the forge split at length, and a guard that read prose as evidence
+# would flag every file that merely explains the design.
+FORGE_MARKERS='forge_detect|forge_select|forge_preflight|forge_client|CEREMONY_FORGE\b|CEREMONY_FORGE_CLIENT'
+
+forge_specific_files() {
+  local f rel
+  for f in "$ROOT"/lib/*.sh "$ROOT"/actions/*/*.sh "$ROOT"/bin/* \
+           "$ROOT"/.github/scripts/*.sh; do
+    [ -f "$f" ] || continue
+    rel="${f#"$ROOT"/}"
+    sed 's/[[:space:]]#.*$//; s/^[[:space:]]*#.*$//' "$f" \
+      | grep -qE "$FORGE_MARKERS" && printf '%s\n' "$rel"
+  done
+  return 0
+}
+
+# A call site that merely USES the shim is not forge-specific — every
+# reconciler calls forge_preflight and that is the point of the shim. What the
+# inventory is about is where the forge is DECIDED or BRANCHED ON. So the scan
+# reports files carrying the markers, and the assertion below allows the shim's
+# own consumers explicitly, by name, so adding a seventh consumer is silent but
+# adding a seventh DECIDER is not.
+SHIM_CONSUMERS='actions/issueflow-reconcile/issueflow-reconcile.sh
+actions/labels-reconcile/labels-reconcile.sh
+actions/labels-scope/labels-scope.sh
+actions/refs-not-closing/run.sh
+lib/facts.sh
+lib/ruling.sh
+lib/attention.sh'
+
+unlisted_deciders() {
+  local rel
+  while IFS= read -r rel; do
+    [ -n "$rel" ] || continue
+    grep -qxF "$rel" <<<"$SHIM_CONSUMERS" && continue
+    in_inventory "$rel" && continue
+    printf '%s\n' "$rel"
+  done < <(forge_specific_files)
+}
+
+no_unlisted() {
+  local found
+  found="$(unlisted_deciders)"
+  [ -z "$found" ] || {
+    printf 'forge-specific but not in docs/UPSTREAM-SYNC.md:\n' >&2
+    printf '  %s\n' "$found" >&2
+    return 1
+  }
+}
+check "every forge-deciding file is named in the inventory" 0 "" no_unlisted
+
+# --- teeth --------------------------------------------------------------------
+# The sweep above is a property of the whole tree and cannot be made to fail
+# without editing it, so the predicates are driven directly.
+
+TMP="$(mktemp -d)"
+trap 'rm -rf "$TMP"' EXIT
+
+# shellcheck disable=SC2016 # fixture CONTENT: the literal text a scanned file would hold
+printf '%s\n' '#!/usr/bin/env bash' 'case "$(forge_detect)" in forgejo) : ;; esac' \
+  >"$TMP/scattered.sh"
+# Byte-identical to the strip forge_specific_files uses: a helper that
+# normalised differently would be testing a predicate the scan does not have.
+marker_seen() {
+  sed 's/[[:space:]]#.*$//; s/^[[:space:]]*#.*$//' "$1" | grep -qE "$FORGE_MARKERS"
+}
+check "a new file branching on forge_detect is seen" 0 "" marker_seen "$TMP/scattered.sh"
+printf '%s\n' '#!/usr/bin/env bash' '# forge_detect used to live here' >"$TMP/prose.sh"
+check "prose about forge_detect is not" 1 "" marker_seen "$TMP/prose.sh"
+check "...and such a file would not be in the inventory" 1 "" in_inventory scattered.sh
+
+summary