bootstrap: add a dev role — the Incus claudebox host #19

Closed
dan-claude-bot wants to merge 1 commit from feat/bootstrap-dev-role into main
4 changed files with 132 additions and 10 deletions

View file

@ -20,7 +20,7 @@ PATH (`/usr/local/bin` when root). Re-run any time to upgrade.
## Commands ## Commands
### `rig bootstrap <control-plane|workload|runner>` ### `rig bootstrap <control-plane|workload|runner|dev>`
Run as root on the fresh box (over SSH). Convergent — safe to re-run; a Run as root on the fresh box (over SSH). Convergent — safe to re-run; a
second run changes nothing. second run changes nothing.
@ -29,12 +29,13 @@ second run changes nothing.
rig bootstrap control-plane --hostname my-coolify-box rig bootstrap control-plane --hostname my-coolify-box
rig bootstrap workload --hostname my-prod-box rig bootstrap workload --hostname my-prod-box
rig bootstrap runner --hostname my-ci-box rig bootstrap runner --hostname my-ci-box
rig bootstrap dev --hostname dev-server
``` ```
- `--hostname <name>` — tailnet hostname (default: the role name) - `--hostname <name>` — tailnet hostname (default: the role name)
- `--ts-tag <tag>` — tailnet tag to advertise (default: `tag:server`; - `--ts-tag <tag>` — tailnet tag to advertise (default: `tag:server`;
the `runner` role defaults to `tag:ci` instead, and **refuses** the `runner` role defaults to `tag:ci` and the `dev` role to `tag:local`,
`tag:server` outright — see below) and both **refuse** `tag:server` outright — see below)
What it does: installs `curl ca-certificates unattended-upgrades` (and What it does: installs `curl ca-certificates unattended-upgrades` (and
enables periodic unattended upgrades); writes an sshd hardening drop-in enables periodic unattended upgrades); writes an sshd hardening drop-in
@ -76,6 +77,50 @@ code, and advertising your server tag would extend every grant your servers
hold (SSH between them, say) to that code. The refusal turns the worst hold (SSH between them, say) to that code. The refusal turns the worst
misconfiguration from a documentation warning into a hard error. misconfiguration from a documentation warning into a hard error.
### `rig bootstrap dev`
The **Incus claudebox host** — the one machine class rig didn't make. Everything
else (control planes, workloads, runners) came up rig-made and reproducible; the
box that runs the claudeboxes was hand-built, so "every box is rig-made" had a
hole exactly where an agent runs. `dev` closes it.
```sh
rig bootstrap dev --hostname dev-server
```
On top of the shared machinery (the `00-rig.conf` sshd drop-in **and** its
`sshd -T` effective-config assert, hostname convergence, tailscale join), `dev`
installs and initialises **Incus**: `incus admin init --auto` gives it a default
storage pool, the `default` profile, and a managed bridge (`incusbr0`). Init runs
**once** — a second `bootstrap dev` detects the existing pool + profile root disk
and skips it, so the run is a true no-op — and rig asserts the *effective* Incus
state (`incus profile device show default`, `incus network list`) rather than
trusting `init`'s exit code, the same discipline that caught the sshd first-wins
bug.
Three hard constraints, each enforced rather than documented:
- **`tag:local`, never `tag:server`.** The ACL grants `tag:server → :22`, so a
dev host wearing the server tag hands the control plane free SSH. `dev` defaults
`--ts-tag` to `tag:local` and **refuses `tag:server`** (exit 2) — the correct
tag is the *only* reachable outcome, not a flag the operator remembers. This
already bit us: both M900s came up `tag:server` and had to be retagged by hand.
- **The guest claudeboxes never join the tailnet.** The **host** joins; the
**guests** do not. An agent-inhabited box with its own tailnet node is a
foothold into the control plane, so operator SSH into a claudebox goes *through*
the host (ProxyJump), never a tunnel of its own. rig joins the host and stops —
there is deliberately no "enrol the guests" step, and if one is ever added,
that convenience is the bug.
- **No credentials on the host.** Claudeboxes are creds-free by design; the
operator adds their own interactively. rig installs, templates, and holds no
credential — here as everywhere.
**The rehearsal must assert *effective* state, not files rig wrote.** The existing
Incus rehearsal runs in a pristine Debian container with no cloud-init drop-in, so
it is structurally blind to the sshd first-wins bug. A dev-role rehearsal asserts
what actually resolved: `sshd -T`, `incus info`, and `tailscale status --json`
showing `tag:local` — then a second `bootstrap dev` proving a clean no-op.
### `rig coolify install --version <pin>` ### `rig coolify install --version <pin>`
Control-plane box only. Installs Coolify at exactly the pinned version with Control-plane box only. Installs Coolify at exactly the pinned version with

View file

@ -8,11 +8,13 @@ usage() {
usage: rig <command> [args] usage: rig <command> [args]
commands: commands:
bootstrap <control-plane|workload|runner> [--hostname <name>] [--ts-tag <tag>] bootstrap <control-plane|workload|runner|dev> [--hostname <name>] [--ts-tag <tag>]
OS plumbing on a pristine Debian box: hardening, unattended-upgrades, OS plumbing on a pristine Debian box: hardening, unattended-upgrades,
tailscale join. Prompts for a single-use tailnet pre-auth key tailscale join. Prompts for a single-use tailnet pre-auth key
(TS_AUTHKEY env overrides the prompt). Run as root. Role runner (TS_AUTHKEY env overrides the prompt). Run as root. Role runner
defaults to tag:ci and refuses tag:server. defaults to tag:ci and refuses tag:server. Role dev (the Incus
claudebox host) defaults to tag:local, refuses tag:server, and also
installs + initialises Incus.
coolify install --version <pin> coolify install --version <pin>
Pinned Coolify install (AUTOUPDATE=false). Control-plane box only. Pinned Coolify install (AUTOUPDATE=false). Control-plane box only.
coolify backup install [options] coolify backup install [options]

View file

@ -9,13 +9,20 @@ die() { printf 'rig-bootstrap: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
usage() { usage() {
cat <<'EOF' cat <<'EOF'
usage: rig bootstrap <control-plane|workload|runner> [--hostname <name>] [--ts-tag <tag>] usage: rig bootstrap <control-plane|workload|runner|dev> [--hostname <name>] [--ts-tag <tag>]
--hostname system + tailnet hostname (default: the role name) --hostname system + tailnet hostname (default: the role name)
--ts-tag tailnet tag to advertise (default: tag:server; --ts-tag tailnet tag to advertise (default: tag:server;
role runner defaults to tag:ci and refuses tag:server — role runner defaults to tag:ci and refuses tag:server —
a CI box executes repo-controlled code, and your server a CI box executes repo-controlled code, and your server
tag's grants must never extend to it) tag's grants must never extend to it; role dev defaults to
tag:local and likewise refuses tag:server — the server tag's
ACL grants :22, so a mis-tagged Incus host would hand the
control plane free SSH)
Role dev also installs and initialises Incus (the claudebox host). The HOST
joins the tailnet; the guest claudeboxes deliberately do NOT — an
agent-inhabited box on the tailnet is a foothold into the control plane.
Provide the single-use tailscale pre-auth key via the TS_AUTHKEY env var, or Provide the single-use tailscale pre-auth key via the TS_AUTHKEY env var, or
enter it at the interactive prompt. It is used once and never written to disk. enter it at the interactive prompt. It is used once and never written to disk.
@ -25,15 +32,21 @@ EOF
# --- args (validated before the root check, so errors are testable) --------- # --- args (validated before the root check, so errors are testable) ---------
ROLE="${1:-}" ROLE="${1:-}"
case "$ROLE" in case "$ROLE" in
control-plane|workload|runner) shift ;; control-plane|workload|runner|dev) shift ;;
-h|--help) usage; exit 0 ;; -h|--help) usage; exit 0 ;;
"") usage >&2; die "role required (control-plane|workload|runner)" 2 ;; "") usage >&2; die "role required (control-plane|workload|runner|dev)" 2 ;;
*) die "unknown role: $ROLE (want control-plane|workload|runner)" 2 ;; *) die "unknown role: $ROLE (want control-plane|workload|runner|dev)" 2 ;;
esac esac
TS_HOSTNAME="$ROLE" TS_HOSTNAME="$ROLE"
if [ "$ROLE" = "runner" ]; then if [ "$ROLE" = "runner" ]; then
TS_TAG="tag:ci" TS_TAG="tag:ci"
elif [ "$ROLE" = "dev" ]; then
# The Incus claudebox host. tag:server's ACL grants it :22, so a dev box
# carrying it hands the control plane free SSH — so dev advertises tag:local,
# never tag:server (refused below, not merely defaulted). This already bit us:
# both M900s came up tag:server and had to be retagged by hand.
TS_TAG="tag:local"
else else
TS_TAG="tag:server" TS_TAG="tag:server"
fi fi
@ -54,6 +67,12 @@ done
if [ "$ROLE" = "runner" ] && [ "$TS_TAG" = "tag:server" ]; then if [ "$ROLE" = "runner" ] && [ "$TS_TAG" = "tag:server" ]; then
die "role runner must not advertise tag:server" 2 die "role runner must not advertise tag:server" 2
fi fi
# A dev box is the Incus claudebox host. tag:server's ACL grants it :22, so a
# dev box wearing it hands the control plane free SSH — the exact bug that made
# the M900s retag-by-hand jobs. Correct-tag-only is enforced, not documented.
if [ "$ROLE" = "dev" ] && [ "$TS_TAG" = "tag:server" ]; then
die "role dev must not advertise tag:server" 2
fi
# --- guards ------------------------------------------------------------------ # --- guards ------------------------------------------------------------------
[ "$(id -u)" -eq 0 ] || die "must run as root" [ "$(id -u)" -eq 0 ] || die "must run as root"
@ -201,9 +220,59 @@ else
tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME" --advertise-tags="$TS_TAG" tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME" --advertise-tags="$TS_TAG"
fi fi
# --- incus (dev role only) ----------------------------------------------------
# The Incus claudebox host is the one machine class rig didn't make — it was
# hand-built, so "every box is rig-made, reproducibly" had a hole exactly where
# an agent runs. This block closes it: install Incus, initialise it once.
#
# NOTE — the guest claudeboxes deliberately do NOT join the tailnet. Only the
# HOST joined above; an agent-inhabited box with its own tailnet node is a
# foothold into the control plane, so operator SSH into a claudebox goes through
# the host (ProxyJump), never a tunnel of its own. rig joins the host and stops.
# There is intentionally no code here to enrol the guests: if bootstrap dev ever
# grows a "join the guests too" convenience, that convenience is the bug.
#
# No credentials, either: claudeboxes are creds-free by design and the operator
# adds their own interactively. rig installs, templates and holds nothing secret.
if [ "$ROLE" = "dev" ]; then
if ! command -v incus >/dev/null 2>&1; then
log "installing incus"
# Debian 13 packages incus directly; keep the noninteractive frontend the
# base package block set, so a prompt never wedges an unattended bootstrap.
apt-get install -y -qq incus
else
log "incus already installed"
fi
# Initialise ONCE. `incus admin init --auto` is NOT idempotent — a second run
# errors out ("storage pool already exists"), which would break convergence.
# Detect a prior init by the artefacts --auto leaves behind — a storage pool
# AND a root disk on the default profile — and skip re-init when both exist,
# so a second `bootstrap dev` is a true no-op.
if incus storage list -f csv 2>/dev/null | grep -q . \
&& incus profile device show default 2>/dev/null | grep -q 'type: disk'; then
log "incus already initialised; skipping incus admin init"
else
log "initialising incus (default storage pool, default profile, managed bridge)"
incus admin init --auto
fi
# Assert the EFFECTIVE state, not `init`'s exit code — the repo's "assert what
# resolved, not the action" rule (the same discipline that caught the sshd
# first-wins bug). A green `init` that left no root disk or no managed network
# is a host that cannot launch a claudebox; die here rather than at first use.
incus profile device show default 2>/dev/null | grep -q 'type: disk' \
|| die "incus init did not leave a root disk on the default profile — check 'incus profile show default'"
incus network list -f csv 2>/dev/null | grep -q '^incusbr0,' \
|| die "incus init did not create the managed bridge incusbr0 — check 'incus network list'"
log "incus initialised and verified (default profile has a root disk; incusbr0 present)"
fi
log "done — role ${ROLE}, hostname ${TS_HOSTNAME}" log "done — role ${ROLE}, hostname ${TS_HOSTNAME}"
if [ "$ROLE" = "control-plane" ]; then if [ "$ROLE" = "control-plane" ]; then
log "next: rig coolify install --version <pin>" log "next: rig coolify install --version <pin>"
elif [ "$ROLE" = "runner" ]; then elif [ "$ROLE" = "runner" ]; then
log "next: rig runner install --repo <owner/repo> --version <pin>" log "next: rig runner install --repo <owner/repo> --version <pin>"
elif [ "$ROLE" = "dev" ]; then
log "next: launch claudeboxes on this host (guests stay off the tailnet; reach them via ProxyJump through this host)"
fi fi

View file

@ -37,9 +37,15 @@ check "bootstrap: unknown role exits 2" 2 "unknown role" "$ROOT/commands/bo
check "bootstrap: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/bootstrap.sh" workload --nope check "bootstrap: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/bootstrap.sh" workload --nope
check "bootstrap: hostname needs value" 2 "needs a value" "$ROOT/commands/bootstrap.sh" workload --hostname check "bootstrap: hostname needs value" 2 "needs a value" "$ROOT/commands/bootstrap.sh" workload --hostname
check "bootstrap: runner refuses tag:server" 2 "must not advertise tag:server" "$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server check "bootstrap: runner refuses tag:server" 2 "must not advertise tag:server" "$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server
# dev is the Incus claudebox host: tag:server would grant it :22 via the ACL, so
# it must refuse the server tag exactly as runner does (correct-tag-only, not a
# flag to remember). The incus init + effective tag:local assertion need a real
# host, so they live in the rehearsal, not here.
check "bootstrap: dev refuses tag:server" 2 "must not advertise tag:server" "$ROOT/commands/bootstrap.sh" dev --ts-tag tag:server
if [ "$(id -u)" -ne 0 ]; then if [ "$(id -u)" -ne 0 ]; then
check "bootstrap: refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" workload check "bootstrap: refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" workload
check "bootstrap: runner role parses, refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" runner check "bootstrap: runner role parses, refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" runner
check "bootstrap: dev role parses, refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" dev
else else
echo "skip: bootstrap non-root refusals (running as root)" echo "skip: bootstrap non-root refusals (running as root)"
fi fi