diff --git a/README.md b/README.md index 36e0fd1..ddd707f 100644 --- a/README.md +++ b/README.md @@ -20,7 +20,7 @@ PATH (`/usr/local/bin` when root). Re-run any time to upgrade. ## Commands -### `rig bootstrap ` +### `rig bootstrap ` Run as root on the fresh box (over SSH). Convergent — safe to re-run; a second run changes nothing. @@ -29,12 +29,13 @@ second run changes nothing. rig bootstrap control-plane --hostname my-coolify-box rig bootstrap workload --hostname my-prod-box rig bootstrap runner --hostname my-ci-box +rig bootstrap dev --hostname dev-server ``` - `--hostname ` — tailnet hostname (default: the role name) - `--ts-tag ` — tailnet tag to advertise (default: `tag:server`; - the `runner` role defaults to `tag:ci` instead, and **refuses** - `tag:server` outright — see below) + the `runner` role defaults to `tag:ci` and the `dev` role to `tag:local`, + and both **refuse** `tag:server` outright — see below) What it does: installs `curl ca-certificates unattended-upgrades` (and enables periodic unattended upgrades); writes an sshd hardening drop-in @@ -76,6 +77,50 @@ code, and advertising your server tag would extend every grant your servers hold (SSH between them, say) to that code. The refusal turns the worst misconfiguration from a documentation warning into a hard error. +### `rig bootstrap dev` + +The **Incus claudebox host** — the one machine class rig didn't make. Everything +else (control planes, workloads, runners) came up rig-made and reproducible; the +box that runs the claudeboxes was hand-built, so "every box is rig-made" had a +hole exactly where an agent runs. `dev` closes it. + +```sh +rig bootstrap dev --hostname dev-server +``` + +On top of the shared machinery (the `00-rig.conf` sshd drop-in **and** its +`sshd -T` effective-config assert, hostname convergence, tailscale join), `dev` +installs and initialises **Incus**: `incus admin init --auto` gives it a default +storage pool, the `default` profile, and a managed bridge (`incusbr0`). Init runs +**once** — a second `bootstrap dev` detects the existing pool + profile root disk +and skips it, so the run is a true no-op — and rig asserts the *effective* Incus +state (`incus profile device show default`, `incus network list`) rather than +trusting `init`'s exit code, the same discipline that caught the sshd first-wins +bug. + +Three hard constraints, each enforced rather than documented: + +- **`tag:local`, never `tag:server`.** The ACL grants `tag:server → :22`, so a + dev host wearing the server tag hands the control plane free SSH. `dev` defaults + `--ts-tag` to `tag:local` and **refuses `tag:server`** (exit 2) — the correct + tag is the *only* reachable outcome, not a flag the operator remembers. This + already bit us: both M900s came up `tag:server` and had to be retagged by hand. +- **The guest claudeboxes never join the tailnet.** The **host** joins; the + **guests** do not. An agent-inhabited box with its own tailnet node is a + foothold into the control plane, so operator SSH into a claudebox goes *through* + the host (ProxyJump), never a tunnel of its own. rig joins the host and stops — + there is deliberately no "enrol the guests" step, and if one is ever added, + that convenience is the bug. +- **No credentials on the host.** Claudeboxes are creds-free by design; the + operator adds their own interactively. rig installs, templates, and holds no + credential — here as everywhere. + +**The rehearsal must assert *effective* state, not files rig wrote.** The existing +Incus rehearsal runs in a pristine Debian container with no cloud-init drop-in, so +it is structurally blind to the sshd first-wins bug. A dev-role rehearsal asserts +what actually resolved: `sshd -T`, `incus info`, and `tailscale status --json` +showing `tag:local` — then a second `bootstrap dev` proving a clean no-op. + ### `rig coolify install --version ` Control-plane box only. Installs Coolify at exactly the pinned version with diff --git a/bin/rig b/bin/rig index 8301af6..7f9073d 100755 --- a/bin/rig +++ b/bin/rig @@ -8,11 +8,13 @@ usage() { usage: rig [args] commands: - bootstrap [--hostname ] [--ts-tag ] + bootstrap [--hostname ] [--ts-tag ] OS plumbing on a pristine Debian box: hardening, unattended-upgrades, tailscale join. Prompts for a single-use tailnet pre-auth key (TS_AUTHKEY env overrides the prompt). Run as root. Role runner - defaults to tag:ci and refuses tag:server. + defaults to tag:ci and refuses tag:server. Role dev (the Incus + claudebox host) defaults to tag:local, refuses tag:server, and also + installs + initialises Incus. coolify install --version Pinned Coolify install (AUTOUPDATE=false). Control-plane box only. coolify backup install [options] diff --git a/commands/bootstrap.sh b/commands/bootstrap.sh index 22e171d..9f6d891 100755 --- a/commands/bootstrap.sh +++ b/commands/bootstrap.sh @@ -9,13 +9,20 @@ die() { printf 'rig-bootstrap: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; } usage() { cat <<'EOF' -usage: rig bootstrap [--hostname ] [--ts-tag ] +usage: rig bootstrap [--hostname ] [--ts-tag ] --hostname system + tailnet hostname (default: the role name) --ts-tag tailnet tag to advertise (default: tag:server; role runner defaults to tag:ci and refuses tag:server — a CI box executes repo-controlled code, and your server - tag's grants must never extend to it) + tag's grants must never extend to it; role dev defaults to + tag:local and likewise refuses tag:server — the server tag's + ACL grants :22, so a mis-tagged Incus host would hand the + control plane free SSH) + +Role dev also installs and initialises Incus (the claudebox host). The HOST +joins the tailnet; the guest claudeboxes deliberately do NOT — an +agent-inhabited box on the tailnet is a foothold into the control plane. Provide the single-use tailscale pre-auth key via the TS_AUTHKEY env var, or enter it at the interactive prompt. It is used once and never written to disk. @@ -25,15 +32,21 @@ EOF # --- args (validated before the root check, so errors are testable) --------- ROLE="${1:-}" case "$ROLE" in - control-plane|workload|runner) shift ;; + control-plane|workload|runner|dev) shift ;; -h|--help) usage; exit 0 ;; - "") usage >&2; die "role required (control-plane|workload|runner)" 2 ;; - *) die "unknown role: $ROLE (want control-plane|workload|runner)" 2 ;; + "") usage >&2; die "role required (control-plane|workload|runner|dev)" 2 ;; + *) die "unknown role: $ROLE (want control-plane|workload|runner|dev)" 2 ;; esac TS_HOSTNAME="$ROLE" if [ "$ROLE" = "runner" ]; then TS_TAG="tag:ci" +elif [ "$ROLE" = "dev" ]; then + # The Incus claudebox host. tag:server's ACL grants it :22, so a dev box + # carrying it hands the control plane free SSH — so dev advertises tag:local, + # never tag:server (refused below, not merely defaulted). This already bit us: + # both M900s came up tag:server and had to be retagged by hand. + TS_TAG="tag:local" else TS_TAG="tag:server" fi @@ -54,6 +67,12 @@ done if [ "$ROLE" = "runner" ] && [ "$TS_TAG" = "tag:server" ]; then die "role runner must not advertise tag:server" 2 fi +# A dev box is the Incus claudebox host. tag:server's ACL grants it :22, so a +# dev box wearing it hands the control plane free SSH — the exact bug that made +# the M900s retag-by-hand jobs. Correct-tag-only is enforced, not documented. +if [ "$ROLE" = "dev" ] && [ "$TS_TAG" = "tag:server" ]; then + die "role dev must not advertise tag:server" 2 +fi # --- guards ------------------------------------------------------------------ [ "$(id -u)" -eq 0 ] || die "must run as root" @@ -201,9 +220,59 @@ else tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME" --advertise-tags="$TS_TAG" fi +# --- incus (dev role only) ---------------------------------------------------- +# The Incus claudebox host is the one machine class rig didn't make — it was +# hand-built, so "every box is rig-made, reproducibly" had a hole exactly where +# an agent runs. This block closes it: install Incus, initialise it once. +# +# NOTE — the guest claudeboxes deliberately do NOT join the tailnet. Only the +# HOST joined above; an agent-inhabited box with its own tailnet node is a +# foothold into the control plane, so operator SSH into a claudebox goes through +# the host (ProxyJump), never a tunnel of its own. rig joins the host and stops. +# There is intentionally no code here to enrol the guests: if bootstrap dev ever +# grows a "join the guests too" convenience, that convenience is the bug. +# +# No credentials, either: claudeboxes are creds-free by design and the operator +# adds their own interactively. rig installs, templates and holds nothing secret. +if [ "$ROLE" = "dev" ]; then + if ! command -v incus >/dev/null 2>&1; then + log "installing incus" + # Debian 13 packages incus directly; keep the noninteractive frontend the + # base package block set, so a prompt never wedges an unattended bootstrap. + apt-get install -y -qq incus + else + log "incus already installed" + fi + + # Initialise ONCE. `incus admin init --auto` is NOT idempotent — a second run + # errors out ("storage pool already exists"), which would break convergence. + # Detect a prior init by the artefacts --auto leaves behind — a storage pool + # AND a root disk on the default profile — and skip re-init when both exist, + # so a second `bootstrap dev` is a true no-op. + if incus storage list -f csv 2>/dev/null | grep -q . \ + && incus profile device show default 2>/dev/null | grep -q 'type: disk'; then + log "incus already initialised; skipping incus admin init" + else + log "initialising incus (default storage pool, default profile, managed bridge)" + incus admin init --auto + fi + + # Assert the EFFECTIVE state, not `init`'s exit code — the repo's "assert what + # resolved, not the action" rule (the same discipline that caught the sshd + # first-wins bug). A green `init` that left no root disk or no managed network + # is a host that cannot launch a claudebox; die here rather than at first use. + incus profile device show default 2>/dev/null | grep -q 'type: disk' \ + || die "incus init did not leave a root disk on the default profile — check 'incus profile show default'" + incus network list -f csv 2>/dev/null | grep -q '^incusbr0,' \ + || die "incus init did not create the managed bridge incusbr0 — check 'incus network list'" + log "incus initialised and verified (default profile has a root disk; incusbr0 present)" +fi + log "done — role ${ROLE}, hostname ${TS_HOSTNAME}" if [ "$ROLE" = "control-plane" ]; then log "next: rig coolify install --version " elif [ "$ROLE" = "runner" ]; then log "next: rig runner install --repo --version " +elif [ "$ROLE" = "dev" ]; then + log "next: launch claudeboxes on this host (guests stay off the tailnet; reach them via ProxyJump through this host)" fi diff --git a/test/cli.sh b/test/cli.sh index 0332275..ee7ac37 100644 --- a/test/cli.sh +++ b/test/cli.sh @@ -37,9 +37,15 @@ check "bootstrap: unknown role exits 2" 2 "unknown role" "$ROOT/commands/bo check "bootstrap: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/bootstrap.sh" workload --nope check "bootstrap: hostname needs value" 2 "needs a value" "$ROOT/commands/bootstrap.sh" workload --hostname check "bootstrap: runner refuses tag:server" 2 "must not advertise tag:server" "$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server +# dev is the Incus claudebox host: tag:server would grant it :22 via the ACL, so +# it must refuse the server tag exactly as runner does (correct-tag-only, not a +# flag to remember). The incus init + effective tag:local assertion need a real +# host, so they live in the rehearsal, not here. +check "bootstrap: dev refuses tag:server" 2 "must not advertise tag:server" "$ROOT/commands/bootstrap.sh" dev --ts-tag tag:server if [ "$(id -u)" -ne 0 ]; then check "bootstrap: refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" workload check "bootstrap: runner role parses, refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" runner + check "bootstrap: dev role parses, refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" dev else echo "skip: bootstrap non-root refusals (running as root)" fi