# rig A CLI that turns a **pristine Debian server into a hardened, tailnet-joined node** — one curl, one command. A second command installs a version-pinned Coolify on a control-plane box. And inside a [box](https://github.com/heavy-duty/box)-minted guest, the same verb converges the **box tenants** — claude, codex, grok, staging — from thin, creds-free seeds (see *the box tenants* below). Philosophy (shared with [box](https://github.com/heavy-duty/box)): **public tool, private state**. rig carries plumbing logic only — no hostnames, no bindings, no secrets, nothing about *your* infrastructure. It takes arguments, does its work, and stores no credential, ever. ## Install ```sh curl -fsSL https://raw.githubusercontent.com/heavy-duty/rig/main/install.sh | bash ``` That installs the **latest release**: the installer resolves the newest tag by following GitHub's `releases/latest` redirect (no API, no token) and downloads that tag's source tarball — which, for a pure-bash tree, *is* the package. Three channels from the same script; `RIG_REF` picks: ```sh curl -fsSL .../install.sh | bash # the latest release curl -fsSL .../install.sh | RIG_REF=0.1.0 bash # pinned to a release curl -fsSL .../install.sh | RIG_REF=main bash # the development tree ``` A tag outranks a branch of the same name (the pin must win); anything that is not a tag falls back to `refs/heads/`. > **Transitional, until 0.1.0 is cut** (right after rig#32 lands): rig has > no GitHub release yet, so the default channel has nothing to resolve — > it **fails loudly** naming `RIG_REF=main` as the way to install today, > and never silently falls back to main. Once 0.1.0 exists, the plain > `curl | bash` above is the normal path. The layout, under the install root (`~/.local/share/rig`): ``` versions// one full tree per installed version current -> versions/ the tracked default $BINDIR/rig -> current/bin/rig the PATH entry, riding the chain ``` `rig` lands on your PATH via `~/.local/bin` (`/usr/local/bin` when root). **Re-running is a safe converge.** Installing a version you already have changes nothing and says so (`RIG_REINSTALL=1` replaces that version's tree); a **new** version installs side by side and becomes the default — so "re-run any time to upgrade" stays true, and now means *upgrade to the latest release*; every version you had stays installed as the way back: ```sh rig versions # what is installed, which is current, which is running rig use # flip the default (atomic; asserts the flip took) ``` On a **bootstrapped host** (one where `/etc/rig/role` exists) switching the default — by upgrade or by `rig use` — prints a WARNING, because a different rig under a converged host changes what a re-converge (`rig bootstrap`, `rig users apply`) would do. It warns rather than refuses: unlike box (which protects live boxes), rig holds no user state a flip can strand, and upgrading a bootstrapped host is the normal case. A pre-versioning flat install is migrated into `versions/` automatically on the next installer run — the tree is moved, not re-downloaded, and preserved bit for bit. For scripting: `RIG_HOME`/`RIG_BIN` override the install root and bin dir, `RIG_INSTALL_SOURCE=` installs from a local tree instead of downloading (how the test suite proves the installer under review), and `RIG_YES=1` answers `rig uninstall`'s prompt in automation. ### Uninstall `rig uninstall` is the real uninstall — no more "rm -rf two paths" prose — and it **ends with an absence assert**: every path it removed is re-checked, and any survivor makes it exit 1 naming the leftovers instead of reporting a clean uninstall that wasn't. ```sh rig uninstall # one non-current version (side-by-side cleanup) rig uninstall --all # everything: every version, current, the PATH symlinks ``` Asks before removing; `--force` or `RIG_YES=1` skips the prompt. The host itself is untouched — what bootstrap converged stays converged. ## Commands ### `rig bootstrap ` Run as root on the fresh box (over SSH). Convergent — safe to re-run; a second run changes nothing. (The box TENANT roles — `claude`, `codex`, `grok`, `staging` — share the verb but are their own family; see *the box tenants* below.) ```sh rig bootstrap control-plane --hostname my-coolify-box rig bootstrap workload --hostname my-prod-box rig bootstrap runner --hostname my-ci-box rig bootstrap dev --hostname my-dev-box rig bootstrap workstation --hostname my-laptop rig bootstrap custom --hostname my-vm-host --class server --host yes --join authkey ``` - `--hostname ` — system + tailnet hostname (default: the role name; `custom` has no default and requires it) - `--class ` — who lives here; decides root SSH's fate after `rig users apply` (see *The identity model* below) - `--host ` — does this box host VMs (box/Incus) - `--join ` — how it enters the tailnet **Roles are presets over three orthogonal traits**, nothing more — every per-role behavior keys off a trait, so any flag overrides its trait without needing a new role (`rig bootstrap workstation --host no` for a laptop that will never run VMs), and `custom` exists for the shape nobody foresaw: it presets nothing and requires `--hostname` plus all three traits. | trait | values | what it drives | |---------|--------------------|----------------| | `class` | `human`, `server` | root SSH's fate once operators exist — human closes it, server keeps it as the control plane's automation door | | `host` | `yes`, `no` | whether the box exists to run VMs — the `/dev/kvm` advisory and, on `yes`, installing the `box` CLI + running box's `setup-host` | | `join` | `authkey`, `login` | tagged pre-auth key (fleet identity) vs interactive browser login (user-owned device) | | role | class | host | join | tailnet tag | |-----------------|--------|------|---------|-------------| | `control-plane` | server | no | authkey | `tag:server` | | `workload` | server | no | authkey | `tag:server` | | `runner` | server | no | authkey | `tag:ci` — refuses `tag:server` | | `dev` | human | yes | authkey | `tag:local` — refuses `tag:server` | | `workstation` | human | yes | login | untagged — any tag refused | > **Where the `staging` preset went.** Before #31, `staging` was the VM-host > preset (`class=server host=yes join=authkey`). The name now belongs to the > box TENANT family — the *guest*, not the host under it — because that is > what "staging" mostly names in practice (the box the control plane will > manage), and two meanings on one word was the worse bug. The host shape > lost nothing: it is one flag away — `rig bootstrap dev --class server > --hostname my-vm-host` — or fully spelled as `custom --class server --host > yes --join authkey`. Tag policy is unchanged: mint the host's key with > `tag:local`; an effective `tag:server` is refused on every role but > `control-plane` and `workload`. The tag column is **derived policy, not a fourth trait**: `tag:server` means "the control plane manages this box", and `control-plane` and `workload` are the only shapes it manages — every other role refuses an effective `tag:server` after join, one rule instead of per-role exceptions. After the tag verification passes, bootstrap writes `/etc/rig/role` — one line, `role=… class=… host=… join=…` — recording the **effective** traits, overrides and all, so an overridden role never lies to the commands that read the marker later (`rig users` keys root policy off `class=`). Written post-join and cmp-guarded, so a marker never describes a box that failed to become what it claims. **`join=login` inverts the tag assertion.** A workstation joins as a user-owned device: there is no pre-auth key — a set `TS_AUTHKEY` is a loud usage error (exit 2; unset it, or pass `--join authkey`) — `tailscale up` prints a login URL, and the human at the keyboard is the credential. After join the assertion flips: **untagged** is what rig asserts, and any effective tag is the refusal — a tag here means control granted this device fleet identity, and on a first join the half-joined node is backed out with `tailscale logout` (a box that was already joined is refused without backout; rig never unwinds state it did not create). Same principle as the authkey path, mirrored: verify what control **granted**, never what was requested. There is **no `--ts-tag` flag**. A pre-auth key is minted *with* its tags, so the key is the single source of truth for the tailnet tag — rig no longer states a second one it might disagree with. It **verifies** the tag control actually granted after join instead (see below). Passing `--ts-tag` now exits 2 with a message pointing you at the key. What it does: installs `curl ca-certificates unattended-upgrades` (and enables periodic unattended upgrades); writes an sshd hardening drop-in (`PermitRootLogin prohibit-password`, `PasswordAuthentication no`) and **verifies it took effect** via `sshd -T`; sets the system hostname; installs tailscale and joins your tailnet — then **verifies the tag the key granted** (see *The tag comes from the key* below). **`--hostname` converges both names.** On a box that has already joined, `bootstrap` skips `tailscale up` (so a re-run needs no pre-auth key) — but it still reconciles the **tailnet** hostname via `tailscale set --hostname`. Without that, a box which joined under the wrong name — say `--hostname` was omitted, so it defaulted to the *role* — stayed misnamed forever, and re-running rig, the documented repair, could not fix it. A machine you deliberately renamed in the admin console keeps that name; rig will not fight it. > **Why the drop-in is `00-rig.conf` and not `99-`.** `sshd_config` is > **first-wins** — *"for each keyword, the first obtained value will be used"* > (`sshd_config(5)`) — and `Include` expands its glob in lexical order. Cloud > images ship `/etc/ssh/sshd_config.d/50-cloud-init.conf` carrying > `PasswordAuthentication yes`, so a `99-` drop-in is read **second** and every > keyword in it is silently discarded. This is the opposite of the > last-wins convention most config systems use, and it shipped green here for > a month: rig asserted the *file existed* rather than what `sshd` actually > resolved, and the Incus rehearsal container has no cloud-init drop-in to > lose to. Every Hetzner box rig had bootstrapped was still serving > `passwordauthentication yes`. `bootstrap` now sweeps a stale `99-rig.conf` > on re-run, and refuses to claim success unless `sshd -T` agrees. **The pre-auth key** (`join=authkey` roles — everything but `workstation`): provide it via the `TS_AUTHKEY` env var or type it at the interactive prompt. Use a **single-use, tagged, short-expiry** key — the **tagged** part is now load-bearing, not advice (see below). It lives in process memory only — rig never writes a credential to disk. **The tag comes from the key, and rig verifies the one control granted.** rig used to pass `--ts-tag` to `tailscale up --advertise-tags`, stating the tag a *second* time — with no way to know whether its request and the key's own tags agreed. It asserted the tag it **requested**, never the tag control **granted**; this is the same shape as the sshd first-wins bug above, and it left the same scar (both M900s joined carrying `tag:server` and had to be retagged by hand, because nothing in rig ever read the effective tag back). So rig stops overriding the key: `tailscale up` carries no `--advertise-tags`, the key's tags apply, and after join rig polls `tailscale status --json` for `.Self.Tags` — the netmap's ground truth, not `tailscale debug prefs`, which prints what was *requested* — and asserts on that, on **first join and on every re-run** (which catches a box bootstrapped before this change, or retagged behind rig's back). > **An untagged key is a hard refusal.** Drop `--advertise-tags` and you also > drop the accidental net that used to tag an untagged key's node anyway. An > untagged node joins owned by the *key creator's user identity* — it inherits > that human's ACL grants, expires with the key, and vanishes if the account is > deleted. That is a fleet-shaped mistake, not a warning: rig runs `tailscale > logout` to back the half-joined node out and dies telling you to mint a tagged > key. A wrong tag **cannot** be fixed in place either — `tailscale set` has no > tag flag, re-tagging needs a fresh key via `up --force-reauth` — so rig detects > and refuses, and never claims a convergence it cannot perform. `control-plane` and `workload` are identical today except the default hostname; they exist because the boxes diverge over time, and because each follow-up command applies to exactly one role. `runner` is the box a CI agent will live on, and it differs behaviorally: it **refuses `tag:server`**. That refusal moved onto the *effective* tag and is strictly stronger for it — it is no longer "don't advertise `tag:server`" but "the key you actually used must not grant `tag:server` to repo-controlled code." A runner executes that code, and `tag:server`'s grants (SSH between your servers, say) must never extend to it; the check turns the worst misconfiguration from a documentation warning into a hard, post-join error. The VM-host shape — the box that *hosts* staging boxes: Incus VMs minted by the [`box`](https://github.com/heavy-duty/box) CLI, each converged from inside with the tenant roles and (for staging guests) `rig bootstrap workload` — rides the traits since #31 (`--class server --host yes --join authkey`; see the note above). It is `class=server`: an unattended VM appliance — operators converge it and leave; nobody lives there. Mint its key with `tag:local`: the host and its guests sit on opposite sides of a trust boundary, and the *host* is never managed by the control plane — so an effective **`tag:server` is refused**, same mechanism as `runner`. On a host-class box (`host=yes`), bootstrap finishes the job instead of leaving a to-do: after the role marker is written it **installs the `box` CLI globally and runs box's own `setup-host`**, so the Incus stack is ready for `box new` when bootstrap returns. rig **delegates to box; it never touches Incus itself** — it does not `apt-get install incus`, does not configure the daemon, does not create the `incus` group. It runs box's global installer (`curl … | BOX_YES=1 bash`) as root, and box installs Incus via its `setup-host`; two tools converging one daemon is drift by construction, and box is the single owner. The step is **convergent** (box's installer is a no-op once box is present) and **opt-out** (`RIG_SKIP_BOX_INSTALL=1`, plus a graceful skip with a manual-command pointer when curl or the network is missing — box is the host *extra*, so a failed box install never aborts a bootstrap that otherwise succeeded). Source is pinnable with `BOX_REPO` / `BOX_REF` (default `heavy-duty/box@main`). If `/dev/kvm` is absent, rig warns (a host that exists to run VMs should have it) but does not fail — the shape is rehearsed in containers, which legitimately lack it. (The world-readable global install path — box under `/opt/box` readable by every non-root user — depends on box PR #71; until that merges box's root install lands in `/root`.) > **The box install is unpinned — on purpose, and out loud.** `coolify install` > demands a version pin; the box step tracks a moving `heavy-duty/box@main`. > Not because box self-updates (it doesn't — it has Coolify's shape, not the > runner's) but because there is nothing to pin *to*: box cuts no tags and no > releases, and its installer resolves `refs/heads/` — branches only — so > a `BOX_REF=v0.5.0` would 404 even if the tag existed. Issue #12's call was > that silently tracking `main` on the box that runs the agents is the option > not to pick — hence this paragraph. `BOX_REPO` / `BOX_REF` are the pin > points the day box cuts a tag (or you point at a frozen branch of your own > fork); `RIG_SKIP_BOX_INSTALL=1` opts out entirely for a host whose box you > manage by hand. `dev` is the human-class VM-hosting shape — `tag:local`, box CLI installed as above, a person living on it (`--class server` turns it into the unattended VM-host appliance) — and `workstation` is the machine at the keyboard end of all the SSH connections: human-class, `join=login`, entering the tailnet as *your* device rather than the fleet's. ### `rig bootstrap ` — the box tenants Run as root, **inside** a [box](https://github.com/heavy-duty/box)-minted guest. Convergent — safe to re-run; a second run changes nothing. ```sh rig bootstrap claude # or codex, grok — the agent tenants rig bootstrap staging # the server tenant (docker + sshd hardening) rig bootstrap claude --user dev # when the seed's BOX_USER differs ``` **The layering** (rig#31 ↔ box#81): a box template stops being where tenant content lives. box mints a **thin, creds-free seed** — base image, the `BOX_USER`, rig (+ tmux) preinstalled, and nothing that joins or admits — and everything the guest *becomes* is a rig tenant role. cloud-init is a first-boot one-shot: not convergent, not re-runnable, and only parse-and-grep testable. rig roles are idempotent scripts with effective-state asserts, driven by the same harness as everything else — and re-runnable on an *existing* box to converge it to a new spec instead of re-minting it. One convergence engine; the guests were the hole. It is **one mechanism, parameterized per tenant** (`lib/tenant-config.sh` holds the whole per-tenant table), not four hand-maintained scripts: | tenant | user | what lands | |-----------|----------|------------| | `claude` | `claude` | the agent toolbelt (git, gh, tmux, ripgrep, jq, age, unzip, build-essential), docker, node 22, the Claude Code CLI on the system PATH, zsh + oh-my-zsh, and `~/.claude/CLAUDE.md` | | `codex` | `codex` | the toolbelt, docker, node 22, `@openai/codex` on the system PATH, and `~/.codex/AGENTS.md` | | `grok` | `grok` | the toolbelt, docker, the grok CLI on the system PATH, and `~/.grok/AGENTS.md` | | `staging` | `ops` | box#69's server posture: docker + the same sshd hardening the machine roles get (shared `lib/sshd.sh`, `class=server` acceptance) | Every install is **asserted on effective state**, not exit codes: the CLI must *answer* (`--version`, run as the tenant user — a CLI that exists but cannot run has already cost a drill), docker must answer, `sshd -T` must resolve the hardening. The CLI also lands on the **system** PATH (`/usr/local/bin`): `box exec -- claude …` runs a non-interactive shell that reads no rc files, so a PATH export alone is invisible to it. **Creds-free and non-interactive, by contract.** box auto-runs these at mint (`box exec … rig bootstrap claude`), so nothing here prompts, joins, or admits — no tailnet, no keys (the harness pins this by *absence*: no `tailscale`, no prompt, in the shipped script). The one creds-holding step a staging guest eventually needs — the tailnet workload join — stays **operator-run**, exactly as box#69 designed it: `box shell` → `sudo rig bootstrap workload --hostname ` with a single-use tagged pre-auth key. After that join, re-running `rig bootstrap staging` still converges docker + hardening and leaves the workload marker alone — the machine role is the truer statement of what the box became. **The agent-context file carries the box#80 guard, once.** Every agent tenant writes its agent's instructions file (`CLAUDE.md` / `AGENTS.md`), rendered from one shared template: the creds-free contract, the isolation and disposability facts, and the guard note — **never run `box setup-host`, `box teardown-host`, or the drill inside a box; the box you are in is not a host you own**. A nested box stack claims the guest's own uplink subnet and silently breaks its networking (box#80). The note lives in `lib/tenant-config.sh` exactly once, not copy-pasted per template — that was the point of moving it here. **Tenants and the role marker.** A tenant run writes `role= tenant=yes host=no` — no `class=`, because a guest has no root-door policy of its own (`rig users close-root` fails closed on it, by design). The guard runs the other way too: a box already carrying a **machine** role refuses the agent tenants outright, and *any* tenant refuses a `host=yes` box — a VM host is the opposite of a guest, and a pre-#31 staging *host* re-running its old command is exactly who that refusal catches (it names the new spelling). > **The rig install in the seed is unpinned — same honesty as the box note > above.** The seed preinstalls rig via its curl installer, which resolves > `RIG_REPO`/`RIG_REF` — and since rig#32 the installer defaults to the > **latest release**, with `RIG_REF=` the pin and `RIG_REF=main` the > dev channel. Until rig cuts 0.1.0 there is no release to resolve, so the > seed must set `RIG_REF=main` explicitly (the default channel fails loudly > rather than falling back). That inverts the install edge on this page: > rig installs box on host-class machines, and box guests now install rig. > `RIG_REPO`/`RIG_REF` are the pin points, or point them at a frozen branch > of your own fork. The seed side of this edge is box#81's to document. ### The identity model **Named operators exist on every class, and humans never enter as root.** The tailnet is network-only — no Tailscale SSH — so there is no identity broker at the door: whoever holds a key to an account *is* that account, and a shared root login is unattributable by construction. `rig users apply` puts named operators on every box, server-class included; a human always enters as themself and elevates via sudo. Per role, the whole identity picture at a glance — issue #25's class comparison, translated onto the traits that replaced the class binary: | role | class | host | join | who lives here | root SSH after `rig users apply` | |-----------------|--------|------|---------|--------------------------------------|----------------------------------| | `control-plane` | server | no | authkey | nobody — Coolify runs here | open — the automation door | | `workload` | server | no | authkey | nobody — deployed services run here | open — the automation door | | `runner` | server | no | authkey | nobody — CI jobs as `github-runner` | open — the automation door | | `dev` | human | yes | authkey | operators, minting boxes | closed by `rig users close-root` | | `workstation` | human | yes | login | its owner | closed by `rig users close-root` | (The unattended VM-host appliance — formerly the `staging` preset — is the `class=server host=yes join=authkey` shape: nobody lives there, root SSH stays open as the automation door. The box TENANT roles sit outside this table on purpose: a guest is not a tailnet machine, and its marker carries no `class=`, so `rig users close-root` fails closed on it.) Who installs what, and who runs as what: **bootstrap is always root** and installs everything a role needs — on `host=yes` that includes the box CLI (globally) and box's own `setup-host`. **Humans always run as themselves**: operators land via `rig users apply` on every class and elevate through sudo (roles `admin`/`rig`) or the `incus` group (role `box`) — never by logging in as root. **Machine identities stay machine-shaped**: Coolify's automation SSHes in as root (that is what server-class root *is*), CI jobs run as the unprivileged `github-runner`, and guest VMs are their own server-class boxes, converged from inside by `rig bootstrap workload`. **`class` decides root SSH's fate — after `rig users apply`, never before.** On `class=human`, root SSH closes entirely (`rig users close-root`, below). On `class=server` it stays open — key-only, as bootstrap left it — because root there is the **automation** identity the control plane (Coolify) SSHes in as. It is a machine door, never a human one. **Where this diverges from #17's original table:** that table let the `runner` role close root ("no Coolify involved"). The class model supersedes the per-role call: runner is `class=server` — an automation identity, not a person's box — and on every server-class machine root SSH is the management plane rig itself converges through, so `close-root` refuses there deliberately, runner included. A CI box you mean to administer like a human machine is `--class human` at bootstrap, not an exception carved out of the gate. **The detection side benefit:** once humans never use root, any root login that is not the control plane is anomalous *by definition* — a cheap, high-signal alert that a shared root identity makes impossible to write. **The honest caveat:** on a Docker-running box this buys attribution, not privilege reduction — an operator with sudo is root-equivalent anyway. Attribution is the goal: *who did what* survives, even where *what they could do* is everything. ### `rig coolify install --version ` Control-plane box only. Installs Coolify at exactly the pinned version with `AUTOUPDATE=false` — your deploy tooling is verified against an API surface; the platform must never move underneath it on its own. Upgrading is an explicit re-run with a new pin. The pin is required; there is no default. ### `rig coolify backup install` Control-plane box only. Installs a **nightly age-encrypted dump of Coolify's own database** as a systemd timer. ```sh rig coolify backup install rig coolify backup install --schedule '*-*-* 02:30:00 UTC' --pg-container coolify-db ``` - `--schedule ` — systemd calendar expression (default: `*-*-* 04:00:00 UTC`) - `--pg-container` / `--pg-user` / `--pg-db` — Coolify's postgres (defaults: `coolify-db`, `coolify`, `coolify`) That database holds the GitHub App private key, every registered server's SSH key, and every environment value for every environment the control plane manages. It is `pg_dump`ed straight into `age` — encrypted **client-side, on the box** — and only then shipped to S3. The bucket is never trusted with plaintext. It is **forensics, not a restore path.** A lost control plane is rebuilt fresh and reconciled from your manifest, never restored from this artifact. Which is exactly why the plumbing belongs in rig: there *will* be a next control-plane box, and it should be backed up from birth rather than depending on someone remembering a runbook step mid-incident. **rig installs the machinery; you supply the bindings.** rig writes `/etc/coolify-dump.env` **empty**, `0600`, and never reads it back — no credential ever passes through rig. You fill in the age recipient (a *public* key), the S3 bucket + endpoint, and the S3 credentials. Until you do, the unit **fails loudly on every run**: a silent backup is worse than a missing one. rig cannot verify that the upload works — that needs your credentials. So prove it by hand once, rather than letting the timer discover it at 04:00: ```sh systemctl start coolify-dump.service journalctl -u coolify-dump.service -n 20 --no-pager ``` A backup you have never read back is not yet a backup. ### `rig db ` Ad-hoc PostgreSQL dump/restore for a container running on this box. Run as root. ```sh rig db dump coolify-db # -> coolify-db-20260717T041500Z.sql.gz rig db dump my-app-db /srv/snapshots/pre-migrate.sql.gz rig db restore pre-migrate.sql.gz my-app-db # prompts before overwriting rig db restore umami.sql.gz shared-pg umami --yes ``` This is **imperative on-box tooling** — the "give me a copy of that database right now", "put this artifact back" verbs you reach for by hand. It is the counterpart to [`rig coolify backup install`](#rig-coolify-backup-install), which is the *scheduled, declarative, forensics-only* path; `db` is interactive, targets any container, and (on restore) overwrites live data behind a confirm gate. Declarative convergence lives elsewhere by design — this verb exists precisely for the moments that are not convergent. **`dump`** pipes `pg_dump` straight into `gzip`: ```sh docker exec sh -c \ 'pg_dump -U "$POSTGRES_USER" --clean --if-exists --no-owner --no-acl "$POSTGRES_DB"' | gzip ``` - **`--no-owner --no-acl` is mandatory, not cosmetic.** A cross-instance restore runs as the *target's* superuser, and Coolify randomizes that role per database — so the source's `ALTER OWNER`/`GRANT` statements name a role that does not exist on the target and, under `ON_ERROR_STOP=1`, abort the whole restore on the first one. Stripping ownership and ACLs makes the dump describe *data and schema*, portable onto any instance. - **`$POSTGRES_USER` / `$POSTGRES_DB` are read inside the container** — that is why the command is a *single-quoted* `sh -c`: it evaluates the container's own environment, never the host's. The host never hardcodes `postgres`; on the next container that role name is simply wrong. - With no `[outfile]`, it writes `-.sql.gz` in the current directory (same timestamp shape as the nightly dump). `pipefail` is load-bearing: without it a failing `pg_dump` still exits 0 through the pipe and `gzip` compresses the truncated output into a valid `.gz` that looks exactly like a good backup. rig dumps to a sibling temp, promotes it only on success, and refuses to keep an empty artifact. **`restore [db]`** streams the artifact back in: ```sh gunzip -c | docker exec -i sh -c \ 'psql -U "$POSTGRES_USER" -d "${db:-$POSTGRES_DB}" -v ON_ERROR_STOP=1' ``` - It connects as the container's **own superuser** (`$POSTGRES_USER`), again never a hardcoded role, and runs with `ON_ERROR_STOP=1` so a bad restore fails loudly instead of limping to a half-applied state and reporting success. - **`[db]` targets a named database in a shared container** — e.g. a `umami` database living in a Postgres that also hosts other apps. Omit it to restore into the container's default `$POSTGRES_DB`. rig passes the name *into* the container as an env var rather than splicing it into the command string. - **Restore overwrites the target**, so it prompts `y/N` first. `--yes` (or `--force`) is the automation bypass. The artifact is checked for existence and non-emptiness *before* the prompt and before anything touches the database, so a fat-fingered path fails cheaply. #### Verifying a dump/restore actually works A `.gz` that opens without error is not proof of a good backup: a `pg_dump` truncated mid-stream still compresses into a perfectly valid gzip file that *looks* exactly like a complete one. The same ethos as the nightly dump applies here — **a backup you have never read back is not yet a backup.** The only fully-trustworthy proof is to restore the artifact and read the rows back out. On a real Coolify box you can do that **without touching prod data** by restoring into a fresh *scratch* database rather than over the live one: ```sh # 1. dump the live database (read-only; harmless) rig db dump coolify-db /srv/snapshots/verify.sql.gz # 2. create a throwaway database as the container's OWN superuser docker exec coolify-db sh -c 'createdb -U "$POSTGRES_USER" rig_verify' # 3. restore the artifact INTO the scratch db (not the live one) rig db restore /srv/snapshots/verify.sql.gz coolify-db rig_verify --yes # 4. spot-check a table you expect to see docker exec coolify-db sh -c \ 'psql -U "$POSTGRES_USER" -d rig_verify -c "\dt"' docker exec coolify-db sh -c \ 'psql -U "$POSTGRES_USER" -d rig_verify -c "SELECT count(*) FROM "' # 5. drop the scratch db — live data was never touched docker exec coolify-db sh -c 'dropdb -U "$POSTGRES_USER" rig_verify' ``` If the counts and tables are there, the artifact is real. This is exactly the round-trip `test/db-integration.sh` automates in CI (the `db-integration` job): it seeds a known table in a source container whose superuser is *not* the default, dumps it, restores into a second container whose superuser differs, and asserts the rows and an ordered checksum survived — the same proof, done against throwaway containers on every push. ### `rig runner install --repo ` Runner box only, run after `rig bootstrap runner` (the same two-step rhythm as `bootstrap control-plane` → `coolify install`): ```sh rig bootstrap runner --hostname my-ci-box rig runner install --repo acme/widgets ``` Installs GitHub's official `actions/runner` as a systemd service under an unprivileged user (default `github-runner`, created if absent, never root, no supplementary groups). The runner is an agent, not a server: it long-polls GitHub outbound and receives jobs down that already-established connection, so it needs **zero inbound ports** and works fine behind a deny-all firewall — it can even trigger deploys on hosts only it can reach, like a tailnet-only control plane. No Docker, deliberately: the Docker socket is a root API and `docker` group membership is root-equivalent, which is a gratuitous path to root on a box whose whole point is a narrow blast radius. Add Docker only once a job genuinely needs it, and rethink the isolation model then. - `--version ` — actions/runner release to install (default: the latest release, resolved at install time; e.g. `--version 2.335.1` — the latest as of this writing). Pin it when you need a deterministic, auditable install. - `--name ` — runner name (default: this host's hostname) - `--labels ` — runner labels, replacing the `ci-runner` default — keep any label your workflows' `runs-on` needs (GitHub adds `self-hosted` itself) - `--user ` — the unprivileged service user (default: `github-runner`) **The registration token:** provide it via the `RUNNER_TOKEN` env var or type it at the interactive prompt. It's short-lived, consumed at registration, and never written to disk by rig. Why latest-by-default here when `coolify install` demands a pin: the two tools age differently. Coolify never self-updates (`AUTOUPDATE=false`), so its version is a contract your deploy tooling is verified against — stating it is the point. The runner **self-updates regardless**: GitHub refuses jobs from stale runners, so freezing it would just make it silently stop taking work. The install-time version is a starting point either way; `--version` exists for when you want that starting point deterministic and auditable. Convergent **toward `--repo`** — re-running against the repo this box is already on re-uses the binary, skips registration, and never asks for a token. Pointed at a *different* repo it **refuses**, and names both: skipping there would not be convergence, it would be ignoring the argument — restarting the runner on the **old** repo while reporting success, leaving the repo you asked for with no runner and its `runs-on` jobs queued forever. Moving a runner between repos is a trust-boundary act; that verb is [`rig runner repoint`](#rig-runner-repoint---repo-ownerrepo). ### `rig runner status` ```sh rig runner status ``` What this box's runner is registered to — repo, runner name, labels, install dir, systemd unit and its state. Reads the runner's own on-disk config; no token, no network call. Exits 1 when no runner is installed. The answer to "wait, which repo is this box wired to?" should not require knowing that the config lives in a dotfile under an unprivileged user's home. ### `rig runner remove` ```sh rig runner remove rig runner remove --local # no token; leaves a stale entry to delete by hand ``` Stops and uninstalls the systemd service, then deregisters the runner from GitHub. The binary and its user stay put, so a later `rig runner install` re-registers without downloading anything. **The token here is a *removal* token, not a registration token** — a different endpoint, and mixing them up is the easy mistake: ```sh gh api -X POST repos//actions/runners/remove-token ``` Supply it via `RUNNER_REMOVE_TOKEN` or the prompt; it never touches disk. `--local` is the escape hatch for when the registration is already gone server-side (or you can't mint a token): the box is cleaned, but a stale offline runner stays listed in the repo, for you to delete from Settings → Actions → Runners. The service always comes down *first*, in both paths. GitHub's own removal refuses to run while the service is installed ("Uninstall service first"), and `--local` skips that check entirely — which would otherwise leave a running service pointed at config that no longer exists. Convergent — a box with no runner installed exits 0. ### `rig runner repoint --repo ` ```sh rig runner repoint --repo acme/widgets ``` Moves an installed runner from one repository to another: deregister, re-register, reusing the binary already on the box. It keeps the runner's existing name unless you pass `--name`. This is the verb that was missing. `runner install` can create a runner but never move one — pointed at a repo the box is not on, it fails and sends you here — and re-pointing a box otherwise meant hand-rolled `config.sh`/`svc.sh` incantations against an install path only rig knew. Two short-lived tokens, each minted from **its own** repo — `RUNNER_REMOVE_TOKEN` for the one it's leaving, `RUNNER_TOKEN` for the one it's joining. Both are collected **before** anything is torn down: a token you turn out not to have should fail while the runner is still registered and working, not halfway through the move. If re-registration fails anyway, rig says so plainly and prints the exact `runner install` line that finishes the job. > **Labels do not survive a move on their own.** GitHub holds them; the runner > does not persist them locally. rig now records what it registered with, so > `repoint` and `status` can read it back — but a runner installed before rig > did that has nothing to read, and `repoint` falls back to the `ci-runner` > default and warns loudly before it touches anything. Labels are what > `runs-on` matches, so a silent change there is a workflow that simply stops > finding its runner. Pass `--labels` if yours differ. Convergent — repointing to the repo it is already on changes nothing, exits 0, and never asks for a token. ### `rig users apply --file ` Converges named operator accounts from a declarative users file — on **every** class (see *The identity model*). Run as root. Convergent: a second identical run says "already converged; no changes". ``` # user roles ssh public key dan admin,box ssh-ed25519 AAAA... dan@laptop dan admin,box ssh-ed25519 AAAA... dan@desktop maria rig,box ssh-ed25519 AAAA... maria@mac ``` One line per key — user, comma-joined roles, then the SSH public key (the rest of the line). The format is bash-parseable on purpose: a rig box has no YAML parser and no jq, and gets neither for this. Repeated username lines add authorized keys, and the roles must be identical on each — a repeated line always means "another key", never a quiet role edit hiding mid-file. `root` is refused as a username: this file names operators; root's fate is class policy. `--file -` reads stdin. A bad file exits 2 with **every** error listed at once, before anything changes — one fix cycle, not one round-trip per line. **`@root` — seed keys from the door you came in through (#17).** A key field of exactly `@root` means "this user's `authorized_keys` becomes root's CURRENT `/root/.ssh/authorized_keys`". The point is lockout-avoidance: you provably hold a root private key — you SSHed in with it to run apply at all — so the seeded key is the one key rig can *know* opens for you; any pasted literal can be a key you do not hold. `@root` mixes with literal lines (seeded keys land first, literals append after), re-runs re-seed from root's then-current file — convergent to it, so a seeded key you hand-remove from the admin returns until you switch the line to literal keys — and apply dies if root has no `authorized_keys` to seed. Root's key lines are copied verbatim, options included: a `from=`/`command=` restriction follows the key, and on a Coolify-managed box root's file also carries *Coolify's* key — on `class=server`, prefer literal keys. **Public tool, private state, here too.** The users file lives in *your* private infra repo and is passed per invocation — rig never persists it. It holds nothing secret anyway: usernames, roles, and *public* keys. | role | grants | via group | |---------|----------------------------------------------|-------------| | `admin` | full NOPASSWD sudo | `rig-admin` | | `rig` | NOPASSWD sudo for `/usr/local/bin/rig` only | `rig` | | `box` | Incus **restricted** tier, no sudo | `incus` | **The honest limit of the `rig` role:** its sudo grant is binary-scoped, not argument-scoped — it trusts its holder with every rig verb *except* identity management. The `rig users` commands gate their **invoker**: run under sudo by anyone outside `rig-admin`, they refuse. Without that gate, `sudo rig users apply` against a file naming yourself admin would make the scoped grant silently root-equivalent through the very tool it scopes. Direct root — a bring-up shell, before any admin exists — proceeds. `box` binds where VMs live, and a users file is fleet-wide — its box grants are not. rig never installs Incus — box's `setup-host` owns the daemon — so when the `incus` group is absent, the `host=` trait decides: on `host=yes` apply dies pointing at `box setup-host` (a VM host missing Incus is a real problem) rather than conjure a group the (nonexistent) daemon would never consult; on `host=no` the box role is **skipped with a warning** and everything else — admins included — still converges, because one box-role user somewhere in the fleet must not stop apply everywhere VMs don't live. `incus-admin` is deliberately **not** a role: that group is host-root-equivalent, break-glass by hand only. **All passwords stay locked, always** — created or found. The SSH key at the door is the authentication, and NOPASSWD sudo does not weaken it: there was never a password to guess or rotate. Convergence is exact. Membership in the three rig-managed groups is made to match the file — added *and* removed — while every other group is left alone: not rig's to converge. `authorized_keys` becomes exactly the file's keys, and its ownership and mode (and `.ssh`'s) are converged on **every** run, not just when content changes — sshd's `StrictModes` treats them as load-bearing, so drifted perms are a broken login that "already converged" would lie about. A user dropped from the file is found via the `/etc/rig/users` ledger and **revoked, never deleted**: the account is expired — the switch PAM actually enforces; a locked password alone still lets a pubkey in under Debian's `UsePAM` — and `authorized_keys` is renamed to `authorized_keys.revoked-by-rig`. Access revoked, data kept: deletion frees the uid for reuse and orphans file ownership, so attribution would rot; home stays for the same reason, and re-adding the user to the file brings them back, fresh keys and all. And the sudoers rules land in `/etc/sudoers.d/rig-roles` only after `visudo -c` passes on the candidate — a bad file under `/etc/sudoers.d` can take down *all* of sudo, locking every admin out of the very escalation path apply just granted. ### `rig users status` ```sh rig users status ``` Read-only truth: per rig-managed user, the roles derived from the groups the user is **actually** in — not the ledger's memory of an apply — plus the `authorized_keys` count (`revoked` when only the `.revoked-by-rig` rename remains) and the user's state, **active** or **revoked**. The state is the ledger's word corroborated by the account's real expiry — the switch that actually revokes — and a mismatch is flagged loudly as drift: a box someone changed behind rig's back must never read as healthy. Reads the box only; no network, no writes. Run as root (shadow is read). ### `rig users close-root` ```sh rig users close-root ``` Shuts the root SSH door — `class=human` boxes only, and only once a named admin can already get in. The gates run in order: the `/etc/rig/role` marker must say `class=human` — an absent marker refuses (never shut the root door blind; re-run bootstrap so the box knows what it is), and `class=server` refuses with no `--force`, because root there is the control plane's automation identity and closing it severs fleet management. Then at least one `rig-admin` member must hold a login sshd would plausibly **accept** — a non-empty `authorized_keys` alone proves a file, not a door: the gate checks the `StrictModes` shape (home, `.ssh`, and `authorized_keys` owned by the user and not group/world-writable), a real login shell, and an unexpired account — and then two **reachability** proofs (#17): `sudo -n true` under `runuser` must answer, so NOPASSWD sudo is effective rather than merely written, and `sshd -T -C user=` must resolve a per-user effective config that accepts the login (`pubkeyauthentication yes`, no `DenyUsers` hit — where any pattern or `USER@HOST` entry counts as a hit, fail closed, since `DenyUsers dan*` really denies admin `dan` and rig will not re-implement sshd's pattern engine to prove a miss — `AllowUsers`, if set, names them literally, and the same fail-closed pair for `DenyGroups`/`AllowGroups` judged against the admin's actual groups), so a `Match` block elsewhere cannot quietly exclude the admin while every file looks right. The refusal names which check failed, per candidate. What no local check can prove: that you *hold* the private key, and how a `Match Address` rule treats your real client address (the probe resolves against a synthetic `addr=127.0.0.1`) — which is why the separate-session verification below stays load-bearing. Never close the only door. Before running it, prove the admin door in a **separate** session — `ssh @` while this one stays open. Root SSH is being welded shut; the admin login must be proven, not presumed. > **The drop-in's name is the entire mechanism.** close-root installs > `/etc/ssh/sshd_config.d/00-rig-users.conf` carrying exactly > `PermitRootLogin no`. sshd_config is first-wins, `Include` expands its glob > lexically, and `-` (0x2D) sorts before `.` (0x2E) — so `00-rig-users.conf` > is read *before* bootstrap's `00-rig.conf` and beats its > `prohibit-password`. Bootstrap's effective-config assertion accepts the > closed state (`no` is strictly harder than what it installs), and by the > same first-wins order its own drop-in can never reopen it — a bootstrap > re-run on a closed box leaves it closed. Validate-then-apply as everywhere: > `sshd -t` before the restart, rollback on failure, and success is only > claimed once `sshd -T` resolves `permitrootlogin no`. Convergent — once root is closed, a re-run says "root already closed; nothing to do" and exits 0. > **On `class=server`, root stays — so lock its key instead.** This is README > guidance, deliberately not automation: prefix Coolify's line in root's > `authorized_keys` with a `from=""` clause, so the > automation identity only opens from the one address supposed to use it. rig > will not write that file — Coolify owns its key material on the servers it > registers, and two tools converging one file is drift by construction (the > same argument that keeps rig's hands off Incus). ## What rig deliberately does NOT do - **Provider firewalls** — Docker publishes ports past host firewalls, so the real boundary is your cloud provider's firewall, configured outside this tool. - **Fetch your config** — boxes never receive repo credentials. Everything rig needs arrives as arguments or an interactive prompt. - **Manage deployments** — deploy manifests/executors are separate concerns. (Planned: the `apply`/`diff` executor half joins rig as commands that run on operator machines, never on boxes.) ## Testing `bash test/cli.sh` (dependency-free assertions) + shellcheck run in CI. The versioned install is proven by REAL installer runs: `RIG_INSTALL_SOURCE` points install.sh at the tree under review and the harness drives it against throwaway `RIG_HOME`/`RIG_BIN` roots — fresh install, converge, reinstall, side-by-side upgrade, `use`/rollback, flat-tree migration, symlink healing, the bootstrapped-host warning (via `RIG_ROLE_MARKER` fixtures), and both uninstalls with their absence asserts. The `rig users` family is covered the same way: the harness drives its refusal matrix — users-file parsing, the marker gates, the lexical drop-in-name assertion, the validate-then-apply ordering — through the sourced lib functions, non-root and network-free. The tenant family follows the same split: the harness proves the arg/refusal surface, the marker guards (off fixture markers), the pure parameter table, and the rendered agent-context file — guard note included — plus absence-greps for the creds-free contract; the real converge belongs to the rehearsal. The end-to-end rehearsal is a throwaway VM/container: pristine Debian → install → `bootstrap workload` with a real single-use key → assert the sshd drop-in, tailnet join, and a no-op second run → destroy, remove the node from the tailnet. The tenant rehearsal is the same shape, creds-free: container + seed user → `rig bootstrap claude` / `staging` → assert the CLI answers, docker answers, `sshd -T`, the context file — then re-run and watch it no-op.