heavy-duty/rig

Fork 1

bootstrap: create an admin user, and make the root door role-aware #21

Closed

dan-claude-bot wants to merge 1 commit from feat/bootstrap-admin-user into main

dan-claude-bot commented

2026-07-17 15:29:54 +00:00

(Migrated from github.com)

Closes #17.

rig hardens the SSH door but never created a human to walk through it: every box was administered as root, survivable only because of the prohibit-password drop-in. This adds an admin user on every role and makes closing root's door role-aware — so a uniform "lock root" can't cut Coolify off from the fleet.

Plan

"Lock root" is four different actions, not one

Measured against a live sshd (OpenSSH 10 / Debian 13), not inferred from hardening guides:

Technique	Key-based root SSH after	Verdict
`passwd -l root` (shadow → `!*`)	✅ still works	harmless — locking a password ≠ disabling an account; near no-op where root already has `*`
`PermitRootLogin prohibit-password`	✅ works	safe — what rig does today
`usermod --expiredate 1 root`	❌ PAM denies	breaks
root shell → `/usr/sbin/nologin`	❌ denied	breaks (and `chsh` then fails too)
`PermitRootLogin no`	❌ denied	the only technique `--lock-root` uses

--lock-root means exactly PermitRootLogin no. usermod --expiredate and the nologin shell don't just break interactive root — they break rig's own convergence, since rig runs as root over SSH and a re-run to pick up a fix would find the door bolted from a direction sshd cannot reopen. PermitRootLogin no leaves the account intact and reopenable by deleting one drop-in. passwd -l root buys ~nothing on top of prohibit-password (already no password surface), so rig skips it and says so.

Role-aware policy — why control-plane and workload refuse

Role	`--lock-root`	Why
`control-plane`	refused (exit 2)	Coolify SSHes to its own host (`host.docker.internal`); non-root localhost unsupported upstream (coollabsio/coolify#4245). A uniform lock-root would cut the control plane off from itself.
`workload`	refused (exit 2)	Closing it needs Coolify's experimental non-root mode — a `coolify` user with `NOPASSWD: ALL` (root by another name) rig does not provision. Attribution is cheaper via sshd key-fingerprint logging + auditd. Revisitable.
`runner`	allowed	No Coolify involved.

Refusals are hard errors (exit 2), same spirit as runner refusing tag:server — a flag that silently bricks a box's only door is worse than no flag. The workload refusal is a deliberate scope call: non-root Coolify is a big, experimental dependency whose NOPASSWD: ALL is strictly more privilege than the docker-group membership rig already refuses, so this PR refuses rather than half-provisions it. (dev from #12 isn't on main yet; it joins runner as lockable when it lands.)

The admin user (every role)

Non-root account, sudo group, never docker (root-equivalent socket — same reason runner install refuses it), no other supplementary group. Generic default name admin, refuses root. NOPASSWD sudo — the admin is key-only with no password, so password-required sudo would make it unable to escalate (a non-root user who can't escalate isn't an admin); this is what Debian/Ubuntu cloud images do for their default user, and is right for a human even though it's wrong for Coolify's service user.

Seed-from-root = live proof, seeded once

The admin's authorized_keys is seeded from root's own at creation. The operator is connected as root right now using one of those keys, so the copy is live proof the private key is in their hands — better than any check rig could invent, needs no new argument, and a public key isn't a secret. --admin-key composes an explicit key with the seed.

Seed-once is a named, honest exception to convergence: re-seeding every run would resurrect a key the operator deliberately removed, so rig seeds only at creation and leaves an existing admin user untouched on re-run. It skips command=/from= restricted key lines while seeding (they'd silently follow to the admin) and warns; Coolify's own key in root's authorized_keys is a documented audit caveat.

The lockout checklist — verified before the door closes

Before writing PermitRootLogin no, rig verifies locally and leaves the door open (exit 1, naming the failed check) if any fails:

account exists and is not expired/disabled (a locked password is fine; an expired account is PAM-denied);
real login shell (not nologin/false);
authorized_keys non-empty, owned by the admin, not group/world-writable (sshd silently ignores an over-permissive file);
sudo -n true under runuser -u <admin>;
sshd -T -C user=<admin> resolves to something that permits the login (an AllowUsers/AllowGroups/DenyUsers/Match block elsewhere can silently exclude them).

What rig cannot verify — that the operator holds the private key — is exactly what seeding from root's keys covers. The drop-in is 00-rig-root.conf, sorting before 00-rig.conf on purpose: sshd_config is first-wins, so PermitRootLogin no in a 10-* file would be read after 00-rig.conf's prohibit-password and silently discarded (the same first-wins trap that once cost this repo a month of boxes serving passwordauthentication=yes). It's installed with the same validate-before-restart + sshd -t + rollback + sshd -T effective-assert dance as the base drop-in, and the base assert was widened to accept no so a locked box stays convergent.

What changed

commands/bootstrap.sh — --admin-user/--admin-key/--lock-root flags (validated before the root check); role-gated lock-root refusals; admin-user provisioning (useradd + sudo group + NOPASSWD sudoers via visudo -cf + seed-from-root); lock-root reachability verification + 00-rig-root.conf install; base hardening assert widened to accept permitrootlogin no; sudo added to base packages.
test/cli.sh — arg-surface + refusal tests (below).
README.md — admin-user section, the four-techniques table, the role policy table, and the lockout checklist.

Testing

bash test/cli.sh → 75 passed, 0 failed. shellcheck -x over bin/* **/*.sh → clean.

New unit tests (all reachable without root): --admin-user root refused exit 2; --admin-user/--admin-key need a value; --lock-root refused exit 2 on control-plane (self-SSH message) and workload (non-root-mode message); --lock-root on runner parses and falls through to the non-root must run as root refusal (proving the gate lets runner past); unknown flag still exit 2.

Runtime provisioning + lock-root verification are rehearsal-only (the arg surface is fully unit-tested; the rest needs a live box). The Incus container rehearsal can't see this — no cloud-init drop-in, no Coolify. Real rehearsal on a throwaway cloud box: bootstrap, then from a second terminal confirm inbound SSH as the admin user (the one check rig cannot self-assert), then --lock-root, then re-run rig and confirm it stays convergent through the admin path, asserting effective state (sshd -T -C user=…, sudo -n true) rather than the files rig wrote.

Interactions

Prerequisite for #12 (dev role): box's setup-host.sh needs a sudo-capable non-root user to run under; this PR creates it. dev will join runner as a lockable role.
Textual conflicts with #16 (tag) and #12 (dev role) since all three edit bootstrap.sh's arg parsing / role handling.

Closes #17. rig hardens the SSH door but never created a human to walk through it: every box was administered as `root`, survivable only because of the `prohibit-password` drop-in. This adds an admin user on every role and makes closing root's door role-aware — so a uniform "lock root" can't cut Coolify off from the fleet. ## Plan ### "Lock root" is four different actions, not one Measured against a live sshd (OpenSSH 10 / Debian 13), not inferred from hardening guides: | Technique | Key-based root SSH after | Verdict | |---|---|---| | `passwd -l root` (shadow → `!*`) | ✅ still works | harmless — locking a *password* ≠ disabling an account; near no-op where root already has `*` | | `PermitRootLogin prohibit-password` | ✅ works | safe — what rig does today | | `usermod --expiredate 1 root` | ❌ PAM denies | breaks | | root shell → `/usr/sbin/nologin` | ❌ denied | breaks (and `chsh` then fails too) | | **`PermitRootLogin no`** | ❌ denied | the only technique `--lock-root` uses | `--lock-root` means **exactly `PermitRootLogin no`**. `usermod --expiredate` and the nologin shell don't just break interactive root — they break **rig's own convergence**, since rig runs as root over SSH and a re-run to pick up a fix would find the door bolted from a direction sshd cannot reopen. `PermitRootLogin no` leaves the account intact and reopenable by deleting one drop-in. `passwd -l root` buys ~nothing on top of `prohibit-password` (already no password surface), so rig skips it and says so. ### Role-aware policy — why control-plane and workload refuse | Role | `--lock-root` | Why | |---|---|---| | `control-plane` | **refused (exit 2)** | Coolify SSHes to its **own** host (`host.docker.internal`); non-root localhost unsupported upstream (coollabsio/coolify#4245). A uniform lock-root would cut the control plane off from itself. | | `workload` | **refused (exit 2)** | Closing it needs Coolify's *experimental* non-root mode — a `coolify` user with `NOPASSWD: ALL` (root by another name) rig does not provision. Attribution is cheaper via sshd key-fingerprint logging + auditd. Revisitable. | | `runner` | **allowed** | No Coolify involved. | Refusals are **hard errors (exit 2)**, same spirit as `runner` refusing `tag:server` — a flag that silently bricks a box's only door is worse than no flag. The workload refusal is a deliberate scope call: non-root Coolify is a big, experimental dependency whose `NOPASSWD: ALL` is strictly *more* privilege than the docker-group membership rig already refuses, so this PR refuses rather than half-provisions it. (`dev` from #12 isn't on main yet; it joins `runner` as lockable when it lands.) ### The admin user (every role) Non-root account, `sudo` group, **never `docker`** (root-equivalent socket — same reason `runner install` refuses it), no other supplementary group. Generic default name `admin`, refuses `root`. **NOPASSWD sudo** — the admin is key-only with no password, so password-required sudo would make it unable to escalate (a non-root user who can't escalate isn't an admin); this is what Debian/Ubuntu cloud images do for their default user, and is right for a *human* even though it's wrong for Coolify's *service* user. ### Seed-from-root = live proof, seeded once The admin's `authorized_keys` is seeded from **root's own** at creation. The operator is connected as root *right now* using one of those keys, so the copy is live proof the private key is in their hands — better than any check rig could invent, needs no new argument, and a public key isn't a secret. `--admin-key` composes an explicit key with the seed. **Seed-once is a named, honest exception to convergence:** re-seeding every run would resurrect a key the operator *deliberately removed*, so rig seeds only at creation and leaves an existing admin user untouched on re-run. It skips `command=`/`from=` restricted key lines while seeding (they'd silently follow to the admin) and warns; Coolify's own key in root's `authorized_keys` is a documented audit caveat. ### The lockout checklist — verified before the door closes Before writing `PermitRootLogin no`, rig verifies **locally** and leaves the door **open** (exit 1, naming the failed check) if any fails: 1. account exists and is not expired/disabled (a locked *password* is fine; an expired *account* is PAM-denied); 2. real login shell (not nologin/false); 3. `authorized_keys` non-empty, owned by the admin, not group/world-writable (sshd silently ignores an over-permissive file); 4. `sudo -n true` under `runuser -u <admin>`; 5. `sshd -T -C user=<admin>` resolves to something that **permits** the login (an `AllowUsers`/`AllowGroups`/`DenyUsers`/`Match` block elsewhere can silently exclude them). What rig *cannot* verify — that the operator holds the private key — is exactly what seeding from root's keys covers. The drop-in is `00-rig-root.conf`, sorting **before** `00-rig.conf` on purpose: sshd_config is *first-wins*, so `PermitRootLogin no` in a `10-*` file would be read after `00-rig.conf`'s `prohibit-password` and silently discarded (the same first-wins trap that once cost this repo a month of boxes serving `passwordauthentication=yes`). It's installed with the same validate-before-restart + `sshd -t` + rollback + `sshd -T` effective-assert dance as the base drop-in, and the base assert was widened to accept `no` so a locked box stays convergent. ## What changed - `commands/bootstrap.sh` — `--admin-user`/`--admin-key`/`--lock-root` flags (validated before the root check); role-gated lock-root refusals; admin-user provisioning (useradd + sudo group + NOPASSWD sudoers via `visudo -cf` + seed-from-root); lock-root reachability verification + `00-rig-root.conf` install; base hardening assert widened to accept `permitrootlogin no`; `sudo` added to base packages. - `test/cli.sh` — arg-surface + refusal tests (below). - `README.md` — admin-user section, the four-techniques table, the role policy table, and the lockout checklist. ## Testing `bash test/cli.sh` → **75 passed, 0 failed**. `shellcheck -x` over `bin/* **/*.sh` → **clean**. New unit tests (all reachable without root): `--admin-user root` refused exit 2; `--admin-user`/`--admin-key` need a value; `--lock-root` refused exit 2 on control-plane (self-SSH message) and workload (non-root-mode message); `--lock-root` on runner *parses* and falls through to the non-root `must run as root` refusal (proving the gate lets runner past); unknown flag still exit 2. **Runtime provisioning + lock-root verification are rehearsal-only** (the arg surface is fully unit-tested; the rest needs a live box). The Incus container rehearsal can't see this — no cloud-init drop-in, no Coolify. Real rehearsal on a throwaway cloud box: `bootstrap`, then **from a second terminal confirm inbound SSH as the admin user** (the one check rig cannot self-assert), *then* `--lock-root`, then re-run rig and confirm it stays convergent **through the admin path**, asserting effective state (`sshd -T -C user=…`, `sudo -n true`) rather than the files rig wrote. ## Interactions - **Prerequisite for #12** (`dev` role): `box`'s `setup-host.sh` needs a sudo-capable non-root user to run under; this PR creates it. `dev` will join `runner` as a lockable role. - **Textual conflicts** with #16 (tag) and #12 (dev role) since all three edit `bootstrap.sh`'s arg parsing / role handling.

claude-bot-andresmgsl (Migrated from github.com) reviewed 2026-07-17 15:29:54 +00:00

codex-bot-andresmgsl (Migrated from github.com) reviewed 2026-07-17 15:29:54 +00:00

grok-bot-andresmgsl (Migrated from github.com) reviewed 2026-07-17 15:29:54 +00:00

dan-claude-bot commented

2026-07-18 16:43:28 +00:00

(Migrated from github.com)

Closing as superseded by #30, per its description: main's fleet-users redesign (#27) reshaped #17, and #30 re-delivers the remaining pieces (reachability proofs, @root key seeding, runner-row reconciliation) on the current design. This branch embodies the old single-flag model and is CONFLICTING against main. See the merge-order plan on #29/#30.

Pull request closed

This pull request cannot be reopened because the branch was deleted.

No reviewers

claude-bot-andresmgsl

codex-bot-andresmgsl

grok-bot-andresmgsl

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: heavy-duty/rig#21

No description provided.