forked from heavy-duty/rig
275 lines
12 KiB
Markdown
275 lines
12 KiB
Markdown
|
|
# Forgejo-native CI: a `ci-box` tenant and a `forgejo-runner` command family
|
||
|
|
|
||
|
|
Status: proposed (#109)
|
||
|
|
Date: 2026-07-27
|
||
|
|
|
||
|
|
## The shape of the thing
|
||
|
|
|
||
|
|
A fleet machine hosts boxes. One of those boxes is the CI box. The Forgejo
|
||
|
|
runner lives inside it, and CI jobs run in containers on that box's own
|
||
|
|
dockerd.
|
||
|
|
|
||
|
|
```
|
||
|
|
+-----------------------------+ outbound HTTPS (long-poll)
|
||
|
|
| Forgejo (Coolify) | <---------------------------------+
|
||
|
|
| forgejo.heavyduty.builders | |
|
||
|
|
+-----------------------------+ |
|
||
|
|
+------------------------------+---+
|
||
|
|
No inbound path to the box. | staging-server (host=yes) |
|
||
|
|
The runner polls out. | +---------------------------+ |
|
||
|
|
| | ci-box (tenant) | |
|
||
|
|
| | forgejo-runner (systemd)| |
|
||
|
|
| | dockerd -> job containers| |
|
||
|
|
| +---------------------------+ |
|
||
|
|
+----------------------------------+
|
||
|
|
```
|
||
|
|
|
||
|
|
Three pieces, in two repos.
|
||
|
|
|
||
|
|
## 1. A forge-agnostic registry fetch (`commands/lib/templates.sh`)
|
||
|
|
|
||
|
|
`templates_resolve` builds three candidate URLs, all on `github.com`. A
|
||
|
|
registry hosted anywhere else cannot be fetched.
|
||
|
|
|
||
|
|
The fix is a fourth knob beside `_DIR` / `_REF` / `_REPO`:
|
||
|
|
|
||
|
|
```sh
|
||
|
|
RIG_TEMPLATES_HOST the forge origin (default https://github.com)
|
||
|
|
```
|
||
|
|
|
||
|
|
The candidate list becomes the host's fact rather than a constant, because the
|
||
|
|
two forges genuinely differ:
|
||
|
|
|
||
|
|
| | GitHub | Forgejo |
|
||
|
|
|---|---|---|
|
||
|
|
| Candidates | `archive/refs/tags/<ref>.tar.gz`, `archive/refs/heads/<ref>.tar.gz`, `archive/<ref>.tar.gz` | `archive/<ref>.tar.gz` — one form, which resolves tags, branches and SHAs alike |
|
||
|
|
| Archive top-level dir | `<repo>-<ref>` | `<repo>` |
|
||
|
|
|
||
|
|
Both were measured against `forgejo.heavyduty.builders`, not assumed.
|
||
|
|
|
||
|
|
The existing "exactly one top-level directory" assert survives untouched — it
|
||
|
|
globs `*/` rather than reconstructing the name, so the differing directory name
|
||
|
|
costs nothing. Its **comment** was wrong for Forgejo and is corrected.
|
||
|
|
|
||
|
|
`templates_source_desc` grows the host, so a misconfigured origin shows up in
|
||
|
|
the unknown-role refusal instead of reading like a typo. The GitHub default
|
||
|
|
means every existing caller behaves exactly as before.
|
||
|
|
|
||
|
|
`install.sh`'s `snapshot_templates` duplicates the same candidate list for its
|
||
|
|
install-time cache. It gets the same knob, from the same environment variable,
|
||
|
|
so the snapshot and the live fetch cannot disagree about where the registry is.
|
||
|
|
|
||
|
|
### The blocker this exposes
|
||
|
|
|
||
|
|
`templates_resolve` documents a hard contract:
|
||
|
|
|
||
|
|
> the fetch is unauthenticated by contract (box auto-runs bootstrap at mint,
|
||
|
|
> holding nothing)
|
||
|
|
|
||
|
|
Measured: on `forgejo.heavyduty.builders`, anonymous requests for
|
||
|
|
`heavy-duty/rig` — reported by the API as `private: false` — return **404** for
|
||
|
|
the API, the web page, the git remote and the archive endpoint. Only an
|
||
|
|
authenticated request succeeds. The instance requires sign-in to view.
|
||
|
|
|
||
|
|
A mint holds no credentials, so **a Forgejo-hosted registry is unreachable at
|
||
|
|
mint time** until the instance serves public repos anonymously:
|
||
|
|
|
||
|
|
```
|
||
|
|
FORGEJO__service__REQUIRE_SIGNIN_VIEW=false
|
||
|
|
```
|
||
|
|
|
||
|
|
This is a Coolify env-var change on the Forgejo service, next to the
|
||
|
|
`FORGEJO__actions__ENABLED=true` that Actions already needs. It is recorded
|
||
|
|
here and in the README as a prerequisite. Nothing in this change silently
|
||
|
|
assumes it: with the gate up, the fetch fails the way any unreachable ref
|
||
|
|
fails, listing every URL tried.
|
||
|
|
|
||
|
|
## 2. The `ci-box` tenant definition
|
||
|
|
|
||
|
|
Data, not mechanism — it belongs in the registry repo. It is staged in this PR
|
||
|
|
under `docs/templates/ci-box/` so it can be reviewed and linted here, and moves
|
||
|
|
to the registry verbatim once that repo exists on Forgejo.
|
||
|
|
|
||
|
|
```
|
||
|
|
USER="ci"
|
||
|
|
CONTEXT_PATH=".ci/CONTEXT.md"
|
||
|
|
CLI_NAME="forgejo-runner"
|
||
|
|
CLI_SRC="/usr/local/bin/forgejo-runner"
|
||
|
|
PATH_LINE="export PATH="$HOME/.local/bin:$PATH""
|
||
|
|
NEEDS_NODE="no"
|
||
|
|
```
|
||
|
|
|
||
|
|
`CLI_SRC` is absolute rather than `~/`-relative, which is the one place this
|
||
|
|
definition departs from the agent tenants. Their CLI lives in the tenant's
|
||
|
|
home; this binary is executed by a systemd unit, and a tenant-writable binary
|
||
|
|
that a root-installed unit runs is a trivial path to root inside the box. So it
|
||
|
|
lands root-owned under `/usr/local/bin`. `test/cli.sh` pins `CLI_SRC` against
|
||
|
|
the path `install.sh` actually writes — a drifted pair converges to a CLI that
|
||
|
|
exists and cannot run, which is the scar `grok-box` left.
|
||
|
|
|
||
|
|
`NEEDS_NODE="no"` because the runner is a static Go binary. Jobs get their Node
|
||
|
|
from the container image, which is the whole point of the label mapping below —
|
||
|
|
installing a second Node on the host would be a toolchain nobody reads.
|
||
|
|
|
||
|
|
`install.sh` fetches the release binary for the architecture and **verifies the
|
||
|
|
published `.sha256` before installing it**. Forgejo ships bare binaries with
|
||
|
|
`.sha256` and `.asc` beside them rather than a tarball, so a checksum is
|
||
|
|
available for free; taking it makes the install auditable in the way
|
||
|
|
`coolify install`'s version pin is.
|
||
|
|
|
||
|
|
This satisfies the tenant schema honestly rather than by paperwork:
|
||
|
|
`bootstrap-tenant.sh` asserts `<CLI> --version` answers **as the tenant user**,
|
||
|
|
and `forgejo-runner --version` does.
|
||
|
|
|
||
|
|
`creds.md` states the box holds no Forgejo credential and that registration is
|
||
|
|
a separate, operator-run act — which is true, and is what the rendered context
|
||
|
|
file needs to say.
|
||
|
|
|
||
|
|
### Why the box replaces docker-in-docker
|
||
|
|
|
||
|
|
The setup guide this design came from builds a `docker:dind` sidecar with
|
||
|
|
`privileged: true` and a plaintext `tcp://…:2375` daemon socket. Inside a
|
||
|
|
tenant that is redundant: `bootstrap-tenant.sh` already installs Docker on
|
||
|
|
every tenant and adds the tenant user to the `docker` group. The runner talks
|
||
|
|
to that daemon over its own socket.
|
||
|
|
|
||
|
|
The isolation argument that justifies dind on a shared CI server is already
|
||
|
|
paid for here by the box: it is network-isolated, disposable, and has no
|
||
|
|
inbound path. Stacking dind inside it would add a privileged container to buy
|
||
|
|
a boundary that already exists.
|
||
|
|
|
||
|
|
Note the trade this makes explicit: `docker` group membership is
|
||
|
|
root-equivalent *within the box*. `rig runner install` refuses Docker for
|
||
|
|
exactly that reason — but it converges a fleet **machine**, where the blast
|
||
|
|
radius is the machine. Here the blast radius is a guest that is thrown away.
|
||
|
|
That is the difference that makes the same trade correct in one place and wrong
|
||
|
|
in the other.
|
||
|
|
|
||
|
|
## 3. `rig forgejo-runner install|status|remove`
|
||
|
|
|
||
|
|
A new family beside `rig runner`, which is left untouched.
|
||
|
|
|
||
|
|
```sh
|
||
|
|
box shell ci-box
|
||
|
|
sudo rig forgejo-runner install \
|
||
|
|
--instance https://forgejo.heavyduty.builders \
|
||
|
|
--name ci-runner-1
|
||
|
|
```
|
||
|
|
|
||
|
|
Default labels:
|
||
|
|
|
||
|
|
```
|
||
|
|
ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-22.04,docker:docker://node:22-bookworm
|
||
|
|
```
|
||
|
|
|
||
|
|
so `runs-on: ubuntu-latest` works in a workflow written for GitHub.
|
||
|
|
|
||
|
|
### Why not `rig runner --forge forgejo`
|
||
|
|
|
||
|
|
GitHub's runner registers against a repository URL, and
|
||
|
|
`assert_runner_repo` converges toward `--repo`: re-running against the same
|
||
|
|
repo is a no-op, a different repo is refused because silently restarting on the
|
||
|
|
old one would leave the requested repo with jobs queued forever.
|
||
|
|
|
||
|
|
Forgejo has no such argument. The runner registers against an **instance**, and
|
||
|
|
whether the registration is instance-wide, org, or single-repo is a property of
|
||
|
|
the *token*, decided in Forgejo's UI before rig ever sees it. There is no repo
|
||
|
|
to converge toward and nothing to compare.
|
||
|
|
|
||
|
|
Folding that into one command would make every guard bimodal to share a flag
|
||
|
|
name, while the contract underneath differs. Two families is the smaller lie.
|
||
|
|
|
||
|
|
### `register` is deprecated upstream, and is still the right call
|
||
|
|
|
||
|
|
Measured on v12.13.2: both `forgejo-runner register` and `create-runner-file`
|
||
|
|
carry `(deprecated)` in their help. The successor is
|
||
|
|
`daemon --url <instance> --uuid <uuid> --token-url <url>`, which requires the
|
||
|
|
runner to **already exist on the instance** — the operator creates it via the
|
||
|
|
API or UI and carries a UUID back.
|
||
|
|
|
||
|
|
`register` is chosen anyway:
|
||
|
|
|
||
|
|
- It still works, and this was verified rather than assumed. A planted
|
||
|
|
`.runner` made `daemon` resolve the instance from the file, connect to a live
|
||
|
|
Forgejo, and fail with `Unauthenticated: unregistered runner` — the transport
|
||
|
|
and the file format are intact; only the credential was fake.
|
||
|
|
- The successor asks the operator for a second, differently-shaped credential
|
||
|
|
dance for no gain today.
|
||
|
|
- `register` writes a file `status` can read back. Under the successor, "what
|
||
|
|
is this box registered to" has no on-disk answer that is not merely rig's own
|
||
|
|
copy of what it was told.
|
||
|
|
|
||
|
|
When upstream removes it: the unit line gains `--url`/`--uuid`, and the readers
|
||
|
|
in `forgejo-runner-config.sh` move to whatever holds the UUID.
|
||
|
|
`assert_runner_instance`'s contract survives either spelling, because it asks
|
||
|
|
about the instance — which both record.
|
||
|
|
|
||
|
|
Not verified here, and left to the drill: that `register` writes `.runner` on a
|
||
|
|
live instance. That needs a real registration token, which this design cannot
|
||
|
|
mint.
|
||
|
|
|
||
|
|
### The convergence guard that does apply
|
||
|
|
|
||
|
|
`assert_runner_instance`: re-running against the instance this box is already
|
||
|
|
registered to re-uses the binary and skips registration; a **different**
|
||
|
|
instance refuses and names both. Same trust-boundary reasoning as
|
||
|
|
`assert_runner_repo`, asked about the axis Forgejo actually has.
|
||
|
|
|
||
|
|
### `.runner` holds a credential here
|
||
|
|
|
||
|
|
GitHub's `.runner` names a repo. Forgejo's holds `address`, `name`, `labels`
|
||
|
|
**and the runner's own long-lived token** — the credential it authenticates
|
||
|
|
with on every poll.
|
||
|
|
|
||
|
|
So this family does something its GitHub sibling never had to: it installs
|
||
|
|
`.runner` as `0600` owned by the runner user, and **re-asserts that mode on
|
||
|
|
every converge**. A registration secret readable by every account on the box
|
||
|
|
would be a quiet, permanent credential leak, and convergence is the only moment
|
||
|
|
rig can notice a mode that drifted.
|
||
|
|
|
||
|
|
`status` therefore reads `address`, `name` and `labels` and never prints the
|
||
|
|
token.
|
||
|
|
|
||
|
|
### Removal
|
||
|
|
|
||
|
|
Forgejo's runner has no deregistration handshake — no removal-token endpoint,
|
||
|
|
no `config.sh remove`. `remove` stops and disables the unit and wipes the local
|
||
|
|
registration, then says plainly that the entry must be deleted in Forgejo's
|
||
|
|
admin UI. `rig runner remove`'s `--local` escape hatch is the *only* mode here,
|
||
|
|
so it is not offered as a flag that suggests a server-side alternative exists.
|
||
|
|
|
||
|
|
## Guard: `bootstrap --undo`
|
||
|
|
|
||
|
|
`bootstrap-undo.sh` refuses to leave the tailnet while a GitHub runner is
|
||
|
|
installed, so undo cannot strand a ghost runner in a repository. The same
|
||
|
|
hazard exists for a Forgejo runner on a machine, so the guard learns the
|
||
|
|
`forgejo-runner.service` unit and the `.runner` under the runner user's home.
|
||
|
|
|
||
|
|
## Testing
|
||
|
|
|
||
|
|
`test/cli.sh` is dependency-free, non-root, offline. What it can prove:
|
||
|
|
|
||
|
|
- Every new command's arg validation, `--help`, and unknown-flag refusals.
|
||
|
|
- `bin/rig` dispatch, including bare `rig forgejo-runner` showing usage.
|
||
|
|
- The forge-aware URL builder as a **pure function**, lifted and driven
|
||
|
|
directly: GitHub host yields the three-candidate list in tag-first order,
|
||
|
|
a Forgejo host yields the single `/archive/<ref>.tar.gz`.
|
||
|
|
- `templates_source_desc` names a non-default host.
|
||
|
|
- The staged `ci-box` definition passes `rig template-lint` — the same parser
|
||
|
|
a mint runs, so the definition cannot ship malformed.
|
||
|
|
- Grep-pins that the `.runner` 0600 assert and the `bootstrap --undo`
|
||
|
|
forgejo-runner guard are present, so a deleted guard cannot ship green
|
||
|
|
(the repo's existing precedent for guards that need a real machine).
|
||
|
|
|
||
|
|
What it cannot prove, and is left to the drill: a real registration against a
|
||
|
|
live Forgejo, and a job actually executing in a container.
|
||
|
|
|
||
|
|
## Out of scope
|
||
|
|
|
||
|
|
`install.sh` hardcodes `github.com` in `resolve_latest_tag` and
|
||
|
|
`ref_candidate_urls` (rig's own source), and `commands/bootstrap.sh` fetches
|
||
|
|
box from `raw.githubusercontent.com`. Hosting rig itself on Forgejo needs those
|
||
|
|
too. They are follow-ups, not this change — `snapshot_templates` is included
|
||
|
|
here only because it fetches *the registry*, and leaving it behind would let
|
||
|
|
the snapshot and the live fetch disagree about where the registry lives.
|