diff --git a/changelog.d/129.md b/changelog.d/129.md new file mode 100644 index 0000000..c6ff707 --- /dev/null +++ b/changelog.d/129.md @@ -0,0 +1,3 @@ +### Added + +- The release drill exercises `rig forgejo-runner` beside `rig runner`, so both shipped runner families carry evidence (#129) diff --git a/drill/README.md b/drill/README.md index efbbc62..fd54cf7 100644 --- a/drill/README.md +++ b/drill/README.md @@ -11,7 +11,7 @@ release (#105, and #107's debt). - **A throwaway Debian 13 machine** you can format, reached as root. The drill hardens its sshd, renames it, joins it to a tailnet, and installs - box/Incus, Coolify and a GitHub runner on it. It is not coming back. + box/Incus, Coolify and Actions runners on it. It is not coming back. The machine is its own reset — there is no teardown script and no need for one. - **The pinned candidate refs, both of them.** `--rig-ref` and @@ -34,6 +34,18 @@ release (#105, and #107's debt). and does something trivial (`echo drilled`). Tokens come from an authenticated `gh`, or from `RUNNER_TOKEN` / `RUNNER_REMOVE_TOKEN`. Without a fork the leg **skips, loudly, into the record**. +- **For leg 3's Forgejo half** (#129): `--forgejo-instance ` and + `--forgejo-runner-repo /`, where that repo carries the same + `workflow_dispatch` workflow — but with `runs-on: drill`, because a + Forgejo runner matches the bare label it registered with. Tokens come from + `FORGEJO_RUNNER_TOKEN` (a registration token) or `FORGEJO_API_TOKEN`, which + mints one and is also what dispatches the job. `--forgejo-ref` names the + branch to dispatch (default `main`): Forgejo's dispatch endpoint requires a + ref in the body, where GitHub's defaults to the repo's default branch. + Without an instance and a repo this half **skips, loudly and separately**. + Note there is no removal token — Forgejo has no deregistration endpoint, so + the leg removes locally and the record tells you to delete the stale runner + row by hand. - **For leg 4** (coolify): a version pin, `--coolify-version 4.1.2`. No pin, no leg — rig's own `coolify install` refuses to default a version and so does its drill. The skip is recorded. @@ -80,7 +92,11 @@ passes, failures and skips separately. skip, exit 0) survives into the record as a SKIP, never a pass. 3. **Runner lifecycle** — register against the fork, dispatch the drill workflow and watch the runner take it, deregister, and assert the - box's registration is actually gone. + box's registration is actually gone. Runs **once per forge**: `rig runner` + against GitHub, then `rig forgejo-runner` against a Forgejo instance + (#129). Both families ship, so a release that evidences only one + evidences half of what it ships; each half skips separately, so a record + can honestly show one forge drilled and the other not. 4. **Coolify** — installed at the pin, `AUTOUPDATE=false` landed in the effective `.env`, container running. diff --git a/drill/drill.sh b/drill/drill.sh index a6f161d..a5ba7b8 100644 --- a/drill/drill.sh +++ b/drill/drill.sh @@ -4,13 +4,16 @@ # ⚠ DESTRUCTIVE, AND MEANT TO BE. Run it on a THROWAWAY Debian machine you # can format. It wipes any installed rig and reinstalls from the pinned # ref, hardens sshd, sets the hostname, joins the tailnet, installs box -# and its Incus stack, installs Coolify and a GitHub Actions runner. +# and its Incus stack, installs Coolify and Actions runners (GitHub, and +# Forgejo when --forgejo-instance is given). # Never run it on a machine you care about. # # TS_AUTHKEY=tskey-... bash drill/drill.sh \ # --rig-ref release/0.4.0 --box-ref 0.9.0 \ # --users ./drill-users --run-id drill-2026-07-24-a \ -# --coolify-version 4.1.2 --runner-repo you/rig --yes +# --coolify-version 4.1.2 --runner-repo you/rig \ +# --forgejo-instance https://forgejo.example.com \ +# --forgejo-runner-repo you/drill-probe --yes # (--box-ref is a tag: since #103 the box that ships is the BOX_RELEASE tag.) # rig's drill asserts CONVERGENCE — a machine reaches its role, idempotently. # The legs (drills/README.md, issue #105): @@ -22,6 +25,10 @@ # the isolation boundary is box's drill's assertion, not this one's). # 2. db — the real dump/restore round-trip, test/db-integration.sh. # 3. runner lifecycle — register, take a job, deregister, against a fork. +# Runs once per forge: `rig runner` against GitHub (--runner-repo), and +# `rig forgejo-runner` against a Forgejo instance (--forgejo-instance + +# --forgejo-runner-repo). Both forges ship, so both need evidence; each +# skips loudly and separately when its inputs are absent (#129). # 4. coolify install — at a pinned version, AUTOUPDATE=false. # # Execution order is 1, 4, 2, 3 — coolify's installer is what puts Docker on @@ -67,6 +74,11 @@ RECORD="${DRILL_RECORD:-}" COOLIFY_VERSION="${DRILL_COOLIFY_VERSION:-}" RUNNER_REPO="${DRILL_RUNNER_REPO:-}" RUNNER_WORKFLOW="${DRILL_RUNNER_WORKFLOW:-drill.yml}" +FJ_INSTANCE="${DRILL_FORGEJO_INSTANCE:-}" +FJ_RUNNER_REPO="${DRILL_FORGEJO_RUNNER_REPO:-}" +# The branch the dispatch names. Forgejo's dispatch endpoint requires a ref in +# the body — unlike GitHub's, which defaults to the repo's default branch. +FJ_REF="${DRILL_FORGEJO_REF:-main}" YES=0 while [ $# -gt 0 ]; do @@ -83,7 +95,10 @@ while [ $# -gt 0 ]; do --coolify-version) COOLIFY_VERSION="$2"; shift 2 ;; --runner-repo) RUNNER_REPO="$2"; shift 2 ;; --runner-workflow) RUNNER_WORKFLOW="$2"; shift 2 ;; - -h|--help) sed -n '2,33p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; + --forgejo-instance) FJ_INSTANCE="$2"; shift 2 ;; + --forgejo-runner-repo) FJ_RUNNER_REPO="$2"; shift 2 ;; + --forgejo-ref) FJ_REF="$2"; shift 2 ;; + -h|--help) sed -n '2,40p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;; *) echo "drill: unknown option: $1 (see --help)" >&2; exit 2 ;; esac done @@ -108,6 +123,133 @@ phase(){ printf '\n\033[1m══ %s\033[0m\n' "$*"; } LEG_NAMES=(); LEG_RESULTS=() leg() { LEG_NAMES+=("$1"); LEG_RESULTS+=("$2"); } +# forgejo_run_verdict — the verdict for OUR dispatch: +# success | failed | pending. Reads GET /repos/{o}/{r}/actions/tasks, whose +# shape is NOT GitHub's and was measured against forgejo.heavyduty.builders +# (8.0.3+gitea-1.22.0) on 2026-07-30 rather than read from the docs: +# +# * There is no `conclusion` field. `status` carries the terminal outcome +# directly ("success"), where GitHub splits status:completed + +# conclusion:success. Reading `conclusion` here gets an empty string on +# every run, which would grade a green job as failed. +# * `id` is a GLOBAL task id; the run's own URL ends in `run_number`. The +# pre-dispatch guard therefore compares `id`, exactly as the GitHub leg +# compares databaseId — an old run must never be read as this one. +# * The payload lists ASSIGNED tasks only. A run sitting queued is simply +# absent (measured: 200s of total_count 0 while the web UI showed the run +# as "job is not started"). So "no new id" is the ONLY signal that the +# runner never took the job — which is the verdict this leg exists for. +# * A task appears when ASSIGNED, so it can be seen mid-flight. A +# non-terminal status is pending, not failed; grading a running job as a +# failure would make the leg flaky inside its own watch window. +# +# EVERY entry is inspected, and the NEWEST id above pre_id decides — never +# entry[0]. actions/tasks accumulates, so the moment a repo is drilled twice +# our run shares the payload with older ones, and nothing documents the sort +# order. Reading the first entry made a green job report as a timeout, a false +# FAILURE on the gate this leg exists to provide (grok/kimi on !130). +# +# grep-and-sed, not jq: a throwaway drill machine has neither jq nor an +# authenticated forge CLI, the same constraint json_field() carries in +# commands/lib/runner-config.sh. `id` is a bare number, which json_field's +# quoted-value shape cannot read, so this reads both forms itself. Newlines are +# stripped first so a pretty-printed payload parses identically to a compact +# one — the instance documents neither. +forgejo_run_verdict() { + local pre="$1" file="$2" obj id best_id="" best_st="" + [ -r "$file" ] || { echo pending; return 0; } + while IFS= read -r obj; do + [ -n "$obj" ] || continue + id="$(printf '%s' "$obj" | grep -o '"id"[[:space:]]*:[[:space:]]*[0-9][0-9]*' \ + | head -n1 | sed 's/.*:[[:space:]]*//')" + [ -n "$id" ] || continue + # Strictly newer than the pre-dispatch id. Equal is the run that was + # already there; lower is older still. + if [ -n "$pre" ]; then + [ "$id" -gt "$pre" ] 2>/dev/null || continue + fi + if [ -z "$best_id" ] || [ "$id" -gt "$best_id" ] 2>/dev/null; then + best_id="$id" + best_st="$(printf '%s' "$obj" | grep -o '"status"[[:space:]]*:[[:space:]]*"[^"]*"' \ + | head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//')" + fi + done </dev/null | grep -o '{[^{}]*}') +EOF + [ -n "$best_id" ] || { echo pending; return 0; } + case "$best_st" in + success) echo success ;; + failure | cancelled | skipped | timedout) echo failed ;; + *) echo pending ;; + esac +} + +# forgejo_leg_row +# The record row for the Forgejo runner leg. PASS requires the WHOLE lifecycle, +# not just the take-a-job outcome. +# +# Keying the row on alone let it read "PASS — registered, took a job, +# removed" when install had failed, because the dispatched job only needs +# SOMETHING answering runs-on: drill — and this leg removes locally, telling the +# operator to delete the stale runner by hand, so a leftover drill-labeled +# runner from the previous drill is the designed-for aftermath rather than a +# contrived case (codex/grok/kimi on !130). drills/.md is the release's +# durable evidence; a row claiming a lifecycle that did not happen is exactly +# what the gate exists to refuse. +# +# The drill's exit code was never wrong here — every one of those failures also +# called `no`. What was wrong is the row, and the row is what outlives the run. +forgejo_leg_row() { + local install_ok="$1" status_ok="$2" took="$3" remove_ok="$4" absent_ok="$5" + if [ "$install_ok" != 1 ] || [ "$status_ok" != 1 ] \ + || [ "$remove_ok" != 1 ] || [ "$absent_ok" != 1 ]; then + echo "FAIL — see Failed below" + return 0 + fi + case "$took" in + success) echo "PASS — registered, took a job, removed (stale row needs deleting by hand)" ;; + none) echo "PARTIAL — registered and removed; took a job: not attempted (no FORGEJO_API_TOKEN)" ;; + *) echo "FAIL — see Failed below" ;; + esac +} + +# forgejo_max_task_id — the highest numeric task id in the payload, +# empty when there is none. This is the PRE-DISPATCH baseline, and it must fold +# max exactly as forgejo_run_verdict does: taking the first id instead names an +# OLD run as the baseline whenever the payload is not newest-first (the order is +# undocumented). A later poll that finds the same body then reads the PREVIOUS +# drill's run as this dispatch's result — a false PASS on the take-a-job +# assertion, which is worse than the false FAIL the same mistake caused inside +# the verdict (grok/kimi, !130). The GitHub leg is safe from this only because +# `gh run list --limit 1` contracts newest-first; this API contracts nothing. +forgejo_max_task_id() { + local file="$1" id best="" + [ -r "$file" ] || return 0 + while IFS= read -r id; do + [ -n "$id" ] || continue + if [ -z "$best" ] || [ "$id" -gt "$best" ] 2>/dev/null; then best="$id"; fi + done </dev/null \ + | grep -o '"id"[[:space:]]*:[[:space:]]*[0-9][0-9]*' | sed 's/.*:[[:space:]]*//') +EOF + printf '%s\n' "$best" +} + +# forgejo_token_verdict — ok | mint-failed | no-source. +# #129's acceptance: "Token source present but the instance is unreachable -> +# the leg FAILS; it must not skip and must not pass". A mint that yields +# nothing — unreachable instance, under-scoped token, wrong repo — is a +# CONFIGURED leg failing, and reporting it as "no token source" both writes +# SKIPPED where the record owes a FAIL and sends the operator to check an env +# var they already set. Absent inputs are the only honest skip. +forgejo_token_verdict() { + if [ -n "$1" ]; then echo ok + elif [ -n "$2" ]; then echo mint-failed + else echo no-source + fi +} + + # run_logged — run a long command with its narration in a file # and a dot every 5s on the terminal: a silent multi-minute apt/install run is # indistinguishable from a wedge, and that ambiguity has cost box whole @@ -352,7 +494,7 @@ This will, ON THIS HOST ($(hostname)): · run 'rig bootstrap $ROLE --users $USERS_FILE' — sshd hardening, hostname change, tailnet join, box ($BOXREPO@$BOXREF) + its Incus stack — TWICE (the second run is the idempotence assertion) - · install Coolify${COOLIFY_VERSION:+ $COOLIFY_VERSION} and a GitHub runner${RUNNER_REPO:+ against $RUNNER_REPO} + · install Coolify${COOLIFY_VERSION:+ $COOLIFY_VERSION}, a GitHub runner${RUNNER_REPO:+ against $RUNNER_REPO} and a Forgejo runner${FJ_INSTANCE:+ against $FJ_INSTANCE}${FJ_RUNNER_REPO:+ ($FJ_RUNNER_REPO)} Only do this on a THROWAWAY machine you can format. EOF [ -t 0 ] || { echo "drill: no TTY to confirm on — pass --yes if you mean it." >&2; exit 2; } @@ -690,6 +832,139 @@ else fi fi +# ============================================================================= +phase "Leg 3 — forgejo runner lifecycle against an instance" +# ============================================================================= +# The same leg as above, for the other forge. Both forges ship a runner family +# (#109 added `rig forgejo-runner` beside `rig runner`), so a release that +# evidences only GitHub evidences half of what it ships (#129). +# +# Three things differ from the GitHub leg, all measured against +# forgejo.heavyduty.builders (8.0.3+gitea-1.22.0) on 2026-07-30, not read: +# +# * Scope is the TOKEN's, never a flag — `rig forgejo-runner install` refuses +# --repo on purpose (commands/forgejo-runner-install.sh:160). The repo here +# is only where the registration token is minted from, and where the +# dispatched workflow lives. +# * The mint path is /repos///actions/runners/registration-token. The +# instance's own swagger documents /repos///runners/registration-token +# — WITHOUT /actions/ — and that path 404s. Do not "fix" this to match the +# published API reference. +# * There is no deregistration endpoint, so there is no removal token and no +# remote deregistration: `rig forgejo-runner remove` is local-only by +# design (commands/forgejo-runner-remove.sh:7-11) and the runner row +# survives in the UI until a human deletes it. The record says so rather +# than implying a clean remote teardown the way the GitHub leg can. +# +# Tokens: FORGEJO_RUNNER_TOKEN (a registration token) is used directly; else +# FORGEJO_API_TOKEN mints one from FJ_RUNNER_REPO. Without an instance, a repo, +# or a token source the leg SKIPS loudly and the record says it did not run. +if [ -z "$FJ_INSTANCE" ] || [ -z "$FJ_RUNNER_REPO" ]; then + skip "forgejo runner lifecycle: no --forgejo-instance/--forgejo-runner-repo given — the leg did not run" + leg "forgejo runner lifecycle" "SKIPPED — no instance/repo provided" +else + fj_reg="${FORGEJO_RUNNER_TOKEN:-}" + if [ -z "$fj_reg" ] && [ -n "${FORGEJO_API_TOKEN:-}" ]; then + fj_reg="$(curl -fsSL -H "Authorization: token ${FORGEJO_API_TOKEN}" \ + "${FJ_INSTANCE%/}/api/v1/repos/${FJ_RUNNER_REPO}/actions/runners/registration-token" 2>/dev/null \ + | grep -o '"token"[[:space:]]*:[[:space:]]*"[^"]*"' | head -n1 \ + | sed 's/.*:[[:space:]]*"//; s/"$//')" + fi + fj_tok_verdict="$(forgejo_token_verdict "$fj_reg" "${FORGEJO_API_TOKEN:-}")" + if [ "$fj_tok_verdict" = mint-failed ]; then + # Configured, and it did not work. Never a skip: see forgejo_token_verdict. + no "registration-token mint FAILED against ${FJ_INSTANCE} — is it reachable, and does FORGEJO_API_TOKEN own ${FJ_RUNNER_REPO}? (the token is never printed)" + leg "forgejo runner lifecycle ($FJ_RUNNER_REPO)" "FAIL — registration-token mint failed" + elif [ "$fj_tok_verdict" = no-source ]; then + skip "forgejo runner lifecycle: no FORGEJO_RUNNER_TOKEN and no FORGEJO_API_TOKEN to mint one — the leg did not run" + leg "forgejo runner lifecycle ($FJ_RUNNER_REPO)" "SKIPPED — no registration token source" + else + FJ_NAME="drill-$(hostname)-$$" + fj_install_ok=0 fj_status_ok=0 fj_remove_ok=0 fj_absent_ok=0 + # The label MUST carry a docker:// image: forgejo-runner runs jobs in + # containers, and a bare label leaves runs-on matched but unrunnable. + if FORGEJO_RUNNER_TOKEN="$fj_reg" run_logged /tmp/drill-forgejo-runner-install.log \ + rig forgejo-runner install --instance "$FJ_INSTANCE" --name "$FJ_NAME" \ + --labels 'drill:docker://node:22-bookworm'; then + ok "rig forgejo-runner install --instance $FJ_INSTANCE exited 0 (registered as $FJ_NAME)" + fj_install_ok=1 + else + no "forgejo-runner install FAILED — tail: $(tail -3 /tmp/drill-forgejo-runner-install.log | tr '\n' ' ')" + fi + if rig forgejo-runner status 2>/dev/null | grep -qF "${FJ_INSTANCE%/}"; then + ok "forgejo-runner status names the instance: $FJ_INSTANCE"; fj_status_ok=1 + else + no "forgejo-runner status does not name ${FJ_INSTANCE}" + fi + + fj_took=none + # Do NOT dispatch once install or status has failed. The job would be taken + # by whatever else answers runs-on: drill — a stale runner this leg's own + # hand-delete caveat leaves behind — and its success would be evidence about + # someone else's runner (codex/grok/kimi, !130). + if [ "$fj_install_ok" != 1 ] || [ "$fj_status_ok" != 1 ]; then + skip "took a job: not attempted — install or status failed, and a foreign runner answering 'drill' could only manufacture a false pass" + elif [ -n "${FORGEJO_API_TOKEN:-}" ]; then + fj_api="${FJ_INSTANCE%/}/api/v1/repos/${FJ_RUNNER_REPO}" + # Read the newest ASSIGNED task id BEFORE dispatching, same guard as the + # GitHub leg: an already-completed run must never be read as ours. + fj_pre_body="$(curl -fsSL -H "Authorization: token ${FORGEJO_API_TOKEN}" \ + "$fj_api/actions/tasks" 2>/dev/null || echo '{}')" + printf '%s' "$fj_pre_body" > /tmp/drill-forgejo-pre.json + fj_pre="$(forgejo_max_task_id /tmp/drill-forgejo-pre.json)" + if curl -fsSL -o /dev/null -X POST -H "Authorization: token ${FORGEJO_API_TOKEN}" \ + -H "Content-Type: application/json" -d "{\"ref\":\"${FJ_REF}\"}" \ + "$fj_api/actions/workflows/${RUNNER_WORKFLOW}/dispatches" 2>/dev/null; then + inf "dispatched $RUNNER_WORKFLOW on $FJ_RUNNER_REPO — waiting for the runner to take it (≤5 min)…" + fj_took=timeout + for _i in $(seq 1 30); do + sleep 10 + curl -fsSL -H "Authorization: token ${FORGEJO_API_TOKEN}" \ + "$fj_api/actions/tasks" -o /tmp/drill-forgejo-tasks.json 2>/dev/null || continue + case "$(forgejo_run_verdict "${fj_pre:-}" /tmp/drill-forgejo-tasks.json)" in + success) fj_took=success; break ;; + failed) fj_took=failed; break ;; + *) : ;; # pending — queued, or assigned and still running + esac + done + else + fj_took=nodispatch + fi + case "$fj_took" in + success) ok "the forgejo runner took a job and it succeeded ($RUNNER_WORKFLOW)" ;; + failed) no "the dispatched job completed UNSUCCESSFULLY — the runner ran it, the workflow failed; read the run on $FJ_RUNNER_REPO" ;; + # A queued task is INVISIBLE in this API until a runner claims it, so a + # timeout means "nothing ever took it". Two causes, and the second one + # is not rig's: the runs-on label may not match, or the daemon's poller + # can go quiet — a restarted daemon claims a minutes-old backlog in + # about a second. Check 'systemctl restart forgejo-runner' before + # reading this as a rig defect. + timeout) no "the dispatched job was never taken within 5 min — check the workflow's runs-on is 'drill', then restart forgejo-runner and re-read (a quiet poller looks exactly like this)" ;; + nodispatch) no "could not dispatch $RUNNER_WORKFLOW on $FJ_RUNNER_REPO — does it carry that workflow, with workflow_dispatch, on its default branch?" ;; + esac + else + skip "took a job: not attempted — no FORGEJO_API_TOKEN to dispatch $RUNNER_WORKFLOW with" + fi + + # No removal token exists on this forge — remove is local by design. + if rig forgejo-runner remove >/dev/null 2>&1; then + note "forgejo-runner removed locally — Forgejo has no deregistration endpoint, so DELETE the stale '$FJ_NAME' row under $FJ_RUNNER_REPO > Settings > Actions > Runners by hand" + fj_remove_ok=1 + else + no "forgejo-runner remove FAILED" + fi + if rig forgejo-runner status >/dev/null 2>&1; then + no "forgejo-runner status still answers after remove — the removal did not take" + else + ok "forgejo-runner status confirms: nothing registered"; fj_absent_ok=1 + fi + + leg "forgejo runner lifecycle ($FJ_RUNNER_REPO)" \ + "$(forgejo_leg_row "$fj_install_ok" "$fj_status_ok" "$fj_took" \ + "$fj_remove_ok" "$fj_absent_ok")" + fi +fi + # ============================================================================= phase "Summary" # ============================================================================= diff --git a/drills/README.md b/drills/README.md index 5b2927a..581e0db 100644 --- a/drills/README.md +++ b/drills/README.md @@ -113,6 +113,7 @@ Candidate refs: box@1a2b3c4 (BOX_REF=release/0.4.0), rig@5d6e7f8, cast@9a0b1c2. | --host yes: pinned box installed, host stack up | PASS — box doctor clean | | `test/db-integration.sh` | PASS — 14 passed, 0 failed | | runner lifecycle against a fork | PASS — registered, took a job, deregistered clean | +| forgejo runner lifecycle (you/drill-probe) | PASS — registered, took a job, removed (stale row needs deleting by hand) | | coolify install (4.1.2) | PASS (6 min) | Failed: `rig users apply` left one revoked key in `authorized_keys` diff --git a/test/drill.sh b/test/drill.sh index 8c01a08..7b94eb7 100644 --- a/test/drill.sh +++ b/test/drill.sh @@ -50,10 +50,10 @@ trap 'rm -rf "$WORK"' EXIT # --- the functions under test, extracted ------------------------------------- FNS="$WORK/drill-fns.sh" -for fn in tree_of assert_installed_from classify_leg capture_state emit_record; do +for fn in tree_of assert_installed_from classify_leg capture_state emit_record forgejo_run_verdict forgejo_token_verdict forgejo_max_task_id forgejo_leg_row; do awk "/^${fn}\(\) \{/,/^\}/" "$ROOT/drill/drill.sh" >> "$FNS" done -for fn in tree_of assert_installed_from classify_leg capture_state emit_record; do +for fn in tree_of assert_installed_from classify_leg capture_state emit_record forgejo_run_verdict forgejo_token_verdict forgejo_max_task_id forgejo_leg_row; do check "extraction guards the awk: ${fn}() landed" 0 "${fn}() {" grep -F "${fn}() {" "$FNS" done # shellcheck source=/dev/null @@ -201,6 +201,177 @@ check "an all-green record says every leg ran and passed" 0 "Every leg ran and e # ============================================================================= # the shipped script itself # ============================================================================= +# ============================================================================= +# forgejo_run_verdict — did OUR dispatched run land, and how (#129) +# ============================================================================= +# The Forgejo half of the runner leg cannot reuse the GitHub reader. Measured +# against forgejo.heavyduty.builders (8.0.3+gitea-1.22.0) on 2026-07-30, a +# completed run in GET /repos/{o}/{r}/actions/tasks carries NO `conclusion` +# field at all — `status` holds the terminal outcome directly, where GitHub +# splits status:completed + conclusion:success. And `id` is a global task id +# (25) while the run's own URL ends in run_number (1), so the pre-dispatch +# guard has to compare `id`. +# +# The payload is also an ASSIGNED-task view: it reads total_count 0 for as +# long as a run sits queued (measured: 200s), so "no new id" is the ONLY +# signal that the runner never took the job. That is the verdict this leg +# exists to produce, which is why it gets its own function and its own tests. +FJ="$WORK/fj"; mkdir -p "$FJ" +printf '%s' '{"workflow_runs":[],"total_count":0}' > "$FJ/empty.json" +printf '%s' '{"workflow_runs":[{"id":25,"status":"success","run_number":1,"url":"https://f/o/r/actions/runs/1"}],"total_count":1}' > "$FJ/new-success.json" +printf '%s' '{"workflow_runs":[{"id":25,"status":"failure","run_number":1,"url":"https://f/o/r/actions/runs/1"}],"total_count":1}' > "$FJ/new-failure.json" +printf '%s' '{"workflow_runs":[{"id":25,"status":"cancelled","run_number":1,"url":"https://f/o/r/actions/runs/1"}],"total_count":1}' > "$FJ/new-cancelled.json" +printf '%s' '{"workflow_runs":[{"id":24,"status":"success","run_number":1,"url":"https://f/o/r/actions/runs/1"}],"total_count":1}' > "$FJ/stale-only.json" + +check "verdict: an empty task list is PENDING, never a pass" 0 "pending" \ + forgejo_run_verdict "" "$FJ/empty.json" +check "verdict: a queued run the runner never took stays PENDING" 0 "pending" \ + forgejo_run_verdict "24" "$FJ/stale-only.json" +check "verdict: OUR new run, status success, is SUCCESS" 0 "success" \ + forgejo_run_verdict "24" "$FJ/new-success.json" +check "verdict: status carries the outcome — failure is FAILED, not success" 0 "failed" \ + forgejo_run_verdict "24" "$FJ/new-failure.json" +check "verdict: a cancelled run is FAILED, not silently passed" 0 "failed" \ + forgejo_run_verdict "24" "$FJ/new-cancelled.json" +check "verdict: the first run ever (no pre-id) still resolves" 0 "success" \ + forgejo_run_verdict "" "$FJ/new-success.json" +# A task appears in this payload the moment it is ASSIGNED, which can be before +# it finishes — so a non-terminal status must read as pending, not as a failure. +# Calling a still-running job "failed" would make the leg flaky in exactly the +# window the leg is watching. +printf '%s' '{"workflow_runs":[{"id":25,"status":"running","run_number":1}],"total_count":1}' > "$FJ/new-running.json" +check "verdict: an assigned-but-running task is PENDING, not FAILED" 0 "pending" \ + forgejo_run_verdict "24" "$FJ/new-running.json" + +# grok/kimi on !130: the reader must not stop at the FIRST run. actions/tasks +# accumulates — the moment a repo is drilled twice, our run shares the payload +# with older ones, and nothing documents the sort order. Reading entry[0] makes +# a green job read as a timeout, which is a FALSE FAILURE on the very gate this +# leg exists to provide. +printf '%s' '{"workflow_runs":[{"id":24,"status":"success"},{"id":25,"status":"success"}],"total_count":2}' > "$FJ/oldest-first.json" +printf '%s' '{"workflow_runs":[{"id":25,"status":"success"},{"id":24,"status":"success"}],"total_count":2}' > "$FJ/newest-first.json" +printf '%s' '{"workflow_runs":[{"id":25,"status":"running"},{"id":26,"status":"success"}],"total_count":2}' > "$FJ/ours-not-first.json" +printf '%s' '{"workflow_runs":[{"id":23,"status":"success"},{"id":24,"status":"failure"}],"total_count":2}' > "$FJ/all-stale.json" +# Pretty-printed: the instance may or may not compact its JSON, and a parser +# that silently depends on one-line objects is a latent failure (kimi, !130). +# Written HERE, like every other fixture: a suite that copies from a scratch +# path passes only on the box that built it (grok/kimi, !130 round 2). +printf '%s\n' '{ + "workflow_runs": [ + {"id": 24, "status": "success"}, + {"id": 25, "name": "drill", "status": "success"} + ], + "total_count": 2 +}' > "$FJ/pretty.json" +check "verdict: ours is LAST in the payload — order must not decide" 0 "success" \ + forgejo_run_verdict "24" "$FJ/oldest-first.json" +check "verdict: ours is FIRST in the payload — same answer" 0 "success" \ + forgejo_run_verdict "24" "$FJ/newest-first.json" +check "verdict: a stale RUNNING entry ahead of ours does not mask it" 0 "success" \ + forgejo_run_verdict "24" "$FJ/ours-not-first.json" +check "verdict: every entry at or below pre is stale — PENDING" 0 "pending" \ + forgejo_run_verdict "24" "$FJ/all-stale.json" +check "verdict: a pretty-printed payload parses too" 0 "success" \ + forgejo_run_verdict "24" "$FJ/pretty.json" + +# The PRE-DISPATCH snapshot has the same multi-entry hazard as the verdict, and +# getting it wrong is worse: a `head -n1` pre-id on an oldest-first payload +# names an OLD run as the baseline, so a later poll that finds the same body +# reports the PREVIOUS drill's run as ours — a false PASS on the take-a-job +# assertion, where the entry[0] bug only produced a false failure (grok, !130). +# Both sides must fold max over every id, which is why they share one function. +check "max id: oldest-first payload yields the NEWEST id, not the first" 0 "25" \ + forgejo_max_task_id "$FJ/oldest-first.json" +check "max id: newest-first payload yields the same answer" 0 "25" \ + forgejo_max_task_id "$FJ/newest-first.json" +check "max id: an empty payload has no id at all" 0 "" \ + forgejo_max_task_id "$FJ/empty.json" +check "max id: a pretty-printed payload folds too" 0 "25" \ + forgejo_max_task_id "$FJ/pretty.json" +# The false PASS, pinned end to end: snapshot the oldest-first body, dispatch, +# the runner never takes it so the body is unchanged — the verdict must stay +# pending. With head -n1 this returned success. +# The two halves composed exactly as the leg composes them. +verdict_after_no_new_run() { forgejo_run_verdict "$(forgejo_max_task_id "$1")" "$1"; } +check "no new run after dispatch: max-id baseline keeps it PENDING (false-PASS guard)" 0 "pending" \ + verdict_after_no_new_run "$FJ/oldest-first.json" +check "…and the same composition on a pretty payload" 0 "pending" \ + verdict_after_no_new_run "$FJ/pretty.json" + +# ============================================================================= +# forgejo_leg_row — the row is the WHOLE lifecycle, not just the job +# ============================================================================= +# codex/grok/kimi on !130: keying the record row on the take-a-job outcome alone +# lets it read "PASS — registered, took a job, removed" when install failed, so +# long as SOMETHING answered runs-on: drill. That is not contrived — this leg +# removes locally and tells the operator to delete the stale runner by hand, so +# a leftover drill-labeled runner from the previous drill is the DESIGNED-FOR +# aftermath, and it answers the fixture exactly. +# +# drills/.md is the release's durable evidence. A row claiming a lifecycle +# that did not happen is precisely what the gate exists to refuse, so PASS +# requires every assertion, not just the interesting one. +check "leg row: everything succeeded is the only PASS" 0 "PASS" \ + forgejo_leg_row 1 1 success 1 1 +check "leg row: install failed cannot PASS, even when a foreign runner took the job" 0 "FAIL" \ + forgejo_leg_row 0 1 success 1 1 +check "leg row: status failed cannot PASS either" 0 "FAIL" \ + forgejo_leg_row 1 0 success 1 1 +check "leg row: remove failed cannot PASS" 0 "FAIL" \ + forgejo_leg_row 1 1 success 0 1 +check "leg row: a runner still registered after remove cannot PASS" 0 "FAIL" \ + forgejo_leg_row 1 1 success 1 0 +check "leg row: no dispatch attempted, everything else clean, is PARTIAL" 0 "PARTIAL" \ + forgejo_leg_row 1 1 none 1 1 +check "leg row: a job that was never taken is a FAIL" 0 "FAIL" \ + forgejo_leg_row 1 1 timeout 1 1 +check "leg row: PARTIAL requires a clean lifecycle too" 0 "FAIL" \ + forgejo_leg_row 0 1 none 1 1 + +# The end-to-end shape codex/grok/kimi asked for, composed the way the leg +# composes it: a tasks payload carrying a NEWER successful run (as a foreign +# drill-labeled runner would produce) must still not yield a PASS row when the +# drill's own install failed. This is the exact false-evidence case. +row_after_failed_install() { + forgejo_leg_row 0 1 "$(forgejo_run_verdict "$(forgejo_max_task_id "$1")" "$1")" 1 1 +} +printf '%s' '{"workflow_runs":[{"id":24,"status":"success"},{"id":99,"status":"success"}],"total_count":2}' \ + > "$FJ/foreign-runner-took-it.json" +check "install failed + a newer successful run in the payload is still FAIL, never PASS" 0 "FAIL" \ + row_after_failed_install "$FJ/foreign-runner-took-it.json" +# …and the same payload with a clean lifecycle is the PASS, so the check above +# is discriminating rather than always-FAIL. +row_after_clean_install() { + forgejo_leg_row 1 1 "$(forgejo_run_verdict "" "$1")" 1 1 +} +check "…while the same payload with a clean lifecycle does PASS" 0 "PASS" \ + row_after_clean_install "$FJ/foreign-runner-took-it.json" + +# ============================================================================= +# forgejo_token_verdict — a configured leg that cannot mint must FAIL, not SKIP +# ============================================================================= +# #129's own acceptance: "Token source present but the instance is unreachable +# -> the leg FAILS; it must not skip and must not pass". A mint that returns +# nothing because the instance is unreachable, the token is under-scoped or the +# repo name is wrong is a CONFIGURED leg failing — reporting "no token source" +# sends the operator to check an env var they already set, and writes SKIPPED +# where the record owes a FAIL. That is the UNREADABLE-vs-NONE shape +# drills/README.md names. +check "token: a resolved registration token is ok" 0 "ok" \ + forgejo_token_verdict "reg-tok" "" +check "token: an explicit token wins even with no API token" 0 "ok" \ + forgejo_token_verdict "reg-tok" "" +check "token: no token at all and no API token is a genuine SKIP" 0 "no-source" \ + forgejo_token_verdict "" "" +check "token: API token offered but mint produced nothing is a FAILURE" 0 "mint-failed" \ + forgejo_token_verdict "" "api-tok" + +# The anti-false-positive guard, stated as its own case: an OLD completed run +# with the SAME id as pre_id must never be read as this dispatch's result. +check "verdict: a pre-existing success with the pre-id is NOT our run" 0 "pending" \ + forgejo_run_verdict "25" "$FJ/new-success.json" + + # Arg refusals fire before the root check (repo doctrine, bootstrap.sh:114), # which is what makes them provable here without a throwaway machine. check "drill.sh refuses to run without BOTH refs pinned (#103)" 2 "--box-ref" \ @@ -217,6 +388,14 @@ check "an unknown flag dies loudly, exit 2" 2 "unknown option" \ bash "$ROOT/drill/drill.sh" --frobnicate check "--help prints the header and exits 0" 0 "THROWAWAY" \ bash "$ROOT/drill/drill.sh" --help +check "--forgejo-instance is a known flag (the leg's opt-in)" 2 "--users is required" \ + bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --forgejo-instance https://f.example.com --yes +check "--forgejo-runner-repo is a known flag" 2 "--users is required" \ + bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --forgejo-runner-repo o/r --yes +check "--help names the forgejo runner leg's flags" 0 "--forgejo-instance" \ + bash "$ROOT/drill/drill.sh" --help +check "--forgejo-ref is a known flag (Forgejo's dispatch needs a ref)" 2 "--users is required" \ + bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --forgejo-ref dev --yes echo "---" echo "$PASS passed, $FAIL failed"