cluade-reviewer-andresmgsl
7d52b2cd4a
The preflight half of #188, landed first so it stands alone: the forge is
decided once, before any sweep, and a client that cannot speak it exits
non-zero with a named reason.
Measured against forgejo.heavyduty.builders at 84bb1a4 — two of the three
actions reported SUCCESS having read nothing:
labels-scope exit 0 "no .github/labeler.yml" (the file is HTTP 200)
labels-reconcile exit 0 "reconciled." (zero PRs enumerated)
issueflow-reconcile exit 1 "unexpected end of JSON input"
labels-reconcile's blind-sweep warning (#96) could not fire: it counts
unreadable PRs against a list `gh pr list` never produced, and a process
substitution's failure does not trip set -e, so total stayed 0. Installing
gh makes it worse, silencing the one loud failure.
Detection is measured, not inferred from docs: a real forgejo-runner v6.3.1
job (probe task 278) shows Forgejo populating the whole GITHUB_* namespace,
so GITHUB_ACTIONS proves nothing. GITHUB_API_URL's shape, GITEA_ACTIONS and
GITHUB_SERVER_URL do. The same probe shows the runner image carries neither
gh nor stoke, which is what makes the forgejo backend REST.
Tests declare CEREMONY_FORGE at the forge boundary rather than stubbing gh
and staying silent about the forge — the boundary move term 5 asks for.
Refs #188
138 lines
6.7 KiB
Bash
138 lines
6.7 KiB
Bash
#!/usr/bin/env bash
|
|
# Contract tests for lib/forge.sh (issue #188). set -u, not -e: failing
|
|
# commands are behavior for the harness to inspect.
|
|
set -u
|
|
|
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
# shellcheck source=test/harness.sh
|
|
. "$ROOT/test/harness.sh"
|
|
# shellcheck source=lib/forge.sh
|
|
. "$ROOT/lib/forge.sh"
|
|
|
|
# eq <want> <cmd...> — succeeds AND prints exactly <want>. check()'s
|
|
# substring match cannot prove "forgejo" was not printed as "forgejox".
|
|
eq() {
|
|
local want="$1" got
|
|
shift
|
|
got="$("$@")" || return 1
|
|
[ "$got" = "$want" ]
|
|
}
|
|
|
|
# detect_in <env-assignments…> — run forge_detect in a clean environment
|
|
# carrying only the named vars, so a leaked GITHUB_* from the CI running
|
|
# THIS suite cannot decide the answer. Every case below is hermetic.
|
|
detect_in() {
|
|
env -i PATH="$PATH" "$@" bash -c '. '"$ROOT"'/lib/forge.sh; forge_detect'
|
|
}
|
|
|
|
preflight_in() {
|
|
env -i PATH="$PATH" "$@" bash -c '. '"$ROOT"'/lib/forge.sh; forge_preflight'
|
|
}
|
|
|
|
# --- forge_detect: the explicit override --------------------------------
|
|
# CEREMONY_FORGE outranks every probe. It is the escape hatch for a forge
|
|
# whose env this file has not met yet, and the handle the tests below use
|
|
# to drive the backends without a live instance.
|
|
|
|
check "override: github" 0 "" eq github detect_in CEREMONY_FORGE=github
|
|
check "override: forgejo" 0 "" eq forgejo detect_in CEREMONY_FORGE=forgejo
|
|
check "override refuses an unknown forge" 1 "unknown forge" \
|
|
detect_in CEREMONY_FORGE=gitlab
|
|
# A typo must not silently fall through to a probe that guesses right by
|
|
# accident: the operator said something, and it was wrong.
|
|
check "override outranks the env" 1 "unknown forge" \
|
|
detect_in CEREMONY_FORGE=gitlab GITHUB_API_URL=https://api.github.com
|
|
|
|
# --- forge_detect: GITHUB_API_URL, the load-bearing signal ---------------
|
|
# Measured on forgejo.heavyduty.builders 2026-08-02 with a real
|
|
# forgejo-runner v6.3.1 job (probe run, task 278). The Forgejo runner
|
|
# populates the GITHUB_* namespace — GITHUB_ACTIONS=true and all — so
|
|
# "GITHUB_ACTIONS is set" proves nothing at all. What differs is where
|
|
# those URLs point:
|
|
#
|
|
# GitHub GITHUB_API_URL=https://api.github.com
|
|
# Forgejo GITHUB_API_URL=https://forgejo.heavyduty.builders/api/v1
|
|
#
|
|
# That is the whole bug this issue exists for, in one variable: gh speaks
|
|
# /api/v3 against api.github.com, and neither half is true here.
|
|
|
|
check "api url: api.github.com is github" 0 "" \
|
|
eq github detect_in GITHUB_API_URL=https://api.github.com
|
|
check "api url: /api/v1 is forgejo" 0 "" \
|
|
eq forgejo detect_in GITHUB_API_URL=https://forgejo.heavyduty.builders/api/v1
|
|
# GitHub Enterprise Server: a self-hosted GitHub still speaks /api/v3, and
|
|
# it is a github backend on a non-github.com host. Getting this wrong would
|
|
# route a GHES consumer to the forgejo backend and break term 5.
|
|
check "api url: GHES /api/v3 is github" 0 "" \
|
|
eq github detect_in GITHUB_API_URL=https://ghe.example.com/api/v3
|
|
|
|
# --- forge_detect: GITEA_ACTIONS, the positive marker --------------------
|
|
# The Forgejo runner also exports GITEA_ACTIONS=true (measured, task 278),
|
|
# which GitHub never sets. It is checked BEFORE the URL shape because it is
|
|
# unambiguous where a hand-set GITHUB_API_URL might not be.
|
|
|
|
check "gitea marker alone is enough" 0 "" eq forgejo detect_in GITEA_ACTIONS=true
|
|
check "gitea marker outranks a github-shaped api url" 0 "" \
|
|
eq forgejo detect_in GITEA_ACTIONS=true GITHUB_API_URL=https://api.github.com
|
|
|
|
# --- forge_detect: refusing to guess ------------------------------------
|
|
# Nothing to read is NOT "probably github". A wrong guess here is exactly
|
|
# the silent blind sweep #188 measured; the whole point of this file is
|
|
# that an unknown forge is loud.
|
|
|
|
check "bare environment refuses" 1 "cannot determine which forge" detect_in
|
|
check "refusal names what it looked at" 1 "GITHUB_API_URL" detect_in
|
|
check "refusal names the escape hatch" 1 "CEREMONY_FORGE" detect_in
|
|
|
|
# --- forge_preflight: the must-fail case --------------------------------
|
|
# The Test plan's named must-fail: "point it at a Forgejo instance with a
|
|
# GitHub-shaped client and assert it refuses loudly rather than sweeping
|
|
# blind."
|
|
#
|
|
# Measured before this guard existed, against this instance:
|
|
# labels-scope exit 0 "no .github/labeler.yml — nothing to derive" (it exists)
|
|
# labels-reconcile exit 0 "reconciled." (zero PRs read)
|
|
# issueflow-reconcile exit 1 "unexpected end of JSON input"
|
|
# Two of three swept blind and reported success. gh present made it WORSE:
|
|
# it silenced the one loud failure. Hence: refuse before the sweep, not
|
|
# after — and say which forge and which client, so the log answers "why"
|
|
# without a second run (#101 D5's report-do-not-diagnose, one layer up).
|
|
|
|
check "forgejo + gh-only client refuses" 1 "cannot speak" \
|
|
preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh
|
|
check "the refusal names the forge" 1 "forgejo" \
|
|
preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh
|
|
check "the refusal names the client" 1 "gh" \
|
|
preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh
|
|
# The refusal must be actionable, not merely loud: #188's whole cost was a
|
|
# red check that told nobody what to do.
|
|
check "the refusal names the issue" 1 "#188" \
|
|
preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh
|
|
|
|
# --- forge_preflight: the passing pairs ---------------------------------
|
|
|
|
check "github + gh passes" 0 "" preflight_in CEREMONY_FORGE=github CEREMONY_FORGE_CLIENT=gh
|
|
check "forgejo + rest passes" 0 "" preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=rest
|
|
# The mirror of the must-fail: a Forgejo client against GitHub is just as
|
|
# wrong, and symmetric refusal is cheaper than explaining why only one
|
|
# direction is checked.
|
|
check "github + rest refuses" 1 "cannot speak" \
|
|
preflight_in CEREMONY_FORGE=github CEREMONY_FORGE_CLIENT=rest
|
|
|
|
# --- forge_preflight: it refuses when the forge itself is unknown --------
|
|
# Detection failure must not be swallowed into a pass — that would restore
|
|
# the blind sweep through the back door.
|
|
check "unknown forge fails the preflight" 1 "cannot determine which forge" preflight_in
|
|
|
|
# --- forge_client: what each backend actually needs ----------------------
|
|
# Measured in the runner image the Forgejo instance actually uses
|
|
# (ghcr.io/catthehacker/ubuntu:act-22.04, task 278): gh ABSENT, stoke
|
|
# ABSENT, curl and jq present. So the forgejo backend is REST-over-curl by
|
|
# necessity, not preference — this is the measurement that retired option
|
|
# A (port to stoke) as well: the CLI is not on the runner either.
|
|
|
|
check "github backend wants gh" 0 "" eq gh forge_client github
|
|
check "forgejo backend wants rest" 0 "" eq rest forge_client forgejo
|
|
check "forge_client refuses an unknown backend" 1 "unknown forge" forge_client gitlab
|
|
|
|
summary
|