class decides root SSH's fate, and this is human's: install
/etc/ssh/sshd_config.d/00-rig-users.conf (PermitRootLogin no), where the NAME
is the mechanism — sshd_config is first-wins, the Include glob expands
lexically, and '-' sorts before '.', so it is read before bootstrap's
00-rig.conf and wins. Gated three ways, no --force: a marker must exist
(never shut the root door blind), it must say class=human (on a server root
is the control plane's automation identity — closing it severs fleet
management), and some rig-admin member must already hold a non-empty
authorized_keys (never close the only door). The gate's policy lives in the
lib as assert_marker_human so the harness proves every refusal against
fixture markers as non-root; RIG_ROLE_MARKER keeps the command pointable at
the same fixtures. Apply is bootstrap's validate-then-apply shape verbatim —
cmp-guard, sshd -t on the merged config before the restart with rollback,
then the sshd -T effective assertion. Bootstrap's own permitrootlogin
assertion widens to accept 'no': the closed door is strictly harder, never
broken, and by first-wins bootstrap cannot reopen it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Operators become a declared fact, not an accumulation of adduser runs: a
line-based, bash-parseable users file (no YAML, no jq — a rig box has
neither) names each user, their roles, and their keys, and apply converges
the box to exactly that. Roles map to groups (admin→rig-admin with full
NOPASSWD sudo, rig→rig sudo for the rig binary only, box→incus with no
sudo — box's setup-host owns Incus, rig only asserts the group). Every
password stays locked always; the SSH key at the door is the
authentication. A user dropped from the file is found via the /etc/rig/users
ledger and locked, never deleted — deleting frees the uid and rots
attribution. The sudoers drop-in lands only after visudo -c passes, because
a bad file under sudoers.d takes down all of sudo. Class never gates apply
(#26: a shared root login is unattributable, so operators belong on every
class); the marker only colors what root SSH does next. The whole file is
validated in one pass before the root check, every error named with its
line, so refusals are provable in the non-root harness through the sourced
parser.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Roles become presets over three orthogonal traits declared in one map:
class (who lives here), host (runs VMs), join (authkey or interactive
login). Every per-role behavior now keys off the traits — the /dev/kvm
advisory rides host=yes, the next-steps log rides class and host — and
tag:server is derived policy, not a trait: only control-plane and
workload are shapes the control plane manages, so every other role
refuses the effective tag. join=login inverts the tag assertion (a
user-owned node must come up untagged; a tag is refused and backed out
on first join, refused without back-out on a box already joined) and
refuses a set TS_AUTHKEY before the root check. The verified shape is
recorded convergently in /etc/rig/role as ground truth for rig users.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The db PR only unit-tested arg parsing; this adds executable proof that
dump/restore actually works end to end.
- test/db-integration.sh: stands up two throwaway Postgres containers whose
superusers DIFFER by construction (src_super vs dst_super), seeds a known
checksummable fixture, runs the real `rig db dump`/`rig db restore`, and
reads the rows back out — proving both invariants db.sh cares about: the
code reads the container's OWN $POSTGRES_USER/$POSTGRES_DB (a hardcoded
`postgres` would break on the non-default source superuser), and
--no-owner --no-acl makes the dump portable across differing superusers (a
plain dump would abort under ON_ERROR_STOP=1 on the missing role). Also
asserts default-outfile naming, restore idempotency (--clean --if-exists),
and the named-[db] scratch-database path. Skips cleanly (exit 0) when Docker
is absent/unreachable or root is unobtainable; always cleans up via trap.
- ci.yml: separate `db-integration` job on ubuntu-latest (Docker preinstalled),
kept apart from the fast shellcheck+cli.sh `check` job so an image pull can't
slow lint feedback.
- README: "Verifying a dump/restore actually works" — the safe manual
round-trip against a real Coolify container via a fresh scratch db, echoing
"a backup you have never read back is not yet a backup."
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
rig used to pass --ts-tag to `tailscale up --advertise-tags`, stating the
tailnet tag a second time with no way to know whether its request and the
key's own tags agreed. It asserted the tag it REQUESTED, never the tag control
GRANTED — the sshd first-wins bug in a different hat, and the same scar (both
M900s joined tag:server, retagged by hand, unnoticed).
Collapse the two sources of truth onto one: the key.
- `tailscale up` drops --advertise-tags; the key's tags apply.
- After join, poll `tailscale status --json` for `.Self.Tags` (netmap ground
truth, not `debug prefs`) until tags appear or BackendState=Running, on BOTH
the fresh-join and already-joined paths.
- UNTAGGED -> hard refusal: `tailscale logout` to back the user-owned node out,
then die naming the fix (mint a tagged key).
- Role policy moves onto the effective tag: a runner must not have tag:server
among the tags the key actually granted. Strictly stronger than before.
- --ts-tag is removed, and dies exit 2 with a message pointing at the key
(consuming its value), not an "unknown flag".
- New array-aware reader json_string_array in lib/runner-config.sh (jq-free,
never fails under set -e), with its own unit tests; bootstrap sources the lib.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `rig db dump <container> [outfile]` and
`rig db restore <artifact> <container> [db] [--yes]` — imperative on-box
PostgreSQL tooling, the interactive counterpart to the scheduled,
declarative `coolify backup install`.
Key decisions:
- Dumps carry `--clean --if-exists --no-owner --no-acl`. `--no-owner
--no-acl` is mandatory for cross-instance restores: the target's
superuser differs (Coolify randomizes it), so a plain dump aborts under
ON_ERROR_STOP=1 on the first GRANT/ALTER OWNER for a missing role.
- $POSTGRES_USER/$POSTGRES_DB are read INSIDE the container (single-quoted
`sh -c`), never hardcoded to `postgres` on the host.
- restore connects as the container's own superuser and runs with
ON_ERROR_STOP=1; the optional [db] arg targets a NAMED database in a
shared container, passed in via a container env var rather than string
splicing.
- restore overwrites the target, so it prompts y/N; --yes/--force is the
automation bypass. Artifact existence/non-emptiness is checked before
the confirm gate and before anything touches the DB.
- dump uses pipefail + a sibling temp promoted only on success, and
refuses to keep an empty artifact — a failed pg_dump must never leave a
plausible-looking .gz behind.
Args are validated before the root check (testable without root); guards
are root, Debian-family warn, docker, and gzip/gunzip. Adds CLI tests and
a `### rig db` README section.
Closes#15
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`rig runner install --repo <B>` on a box already registered to repo A
treated the mere existence of .runner as "already registered", skipped
configure, restarted the service still pointed at A, and reported success.
--repo was accepted, validated, and then ignored — leaving B with zero
runners and its `runs-on` jobs queued against one that will never come.
This is the natural next command after a partial `repoint`, and the failure
is worse than a no-op: moving a runner between repos is a trust-boundary
act, so quietly putting it back on the old one defeats the point of the move.
Gate install on the repo .runner actually names. Convergence — the property
worth keeping — is untouched: re-running against the repo the box is already
on still skips registration, never prompts for a token, and exits 0.
Skipping when the repo *differs* was never convergence, only a silently
ignored argument, so it now fails and names both repos, pointing at
`runner repoint` (move) or `runner remove` (start over). An unreadable
.runner is refused too — it is no licence to assume a match.
The .runner reader that `status` and `repoint` each carried is lifted into
commands/lib/runner-config.sh, which now also holds the guard. Its json_field
no longer dies bare under `set -o pipefail` when a key is missing, which is
what `status`'s own ${REPO_URL:-unknown} fallback always assumed.
Tests: the guard is exercised against a fixture .runner (refuses another repo
naming both, points at repoint, no-ops on the same repo, passes an
unregistered box, refuses an unreadable one) plus an ordering assertion that
it precedes svc.sh start — reaching it through the CLI would need root and a
really-registered runner, which the dependency-free harness cannot fabricate.
All three mutants (guard deleted, guard comparing nothing, guard moved below
the service start) go red.
Closes#13
runner install is convergent by skipping: it sees a registered runner and
leaves it alone. So rig could create a runner and never move or destroy one,
and re-pointing a box at a different repo meant hand-rolled config.sh/svc.sh
incantations against an install layout only rig knew about.
- status: repo, name, labels, dir, unit — read-only, no token, no network.
- remove: service down, then deregister. --local wipes the box without
contacting GitHub, leaving a stale entry to delete by hand.
- repoint: remove + re-register in one act, keeping the runner's name and
reusing the binary already on the box.
The service always comes down before deregistration in both paths: GitHub's
removal throws "Uninstall service first" while the service is configured, and
--local bypasses that check entirely, which would strand a running service
pointed at deleted config.
repoint collects both tokens up front — a token you turn out not to have must
fail while the runner is still registered, not halfway through the move.
Labels are the sharp edge: GitHub holds them, the runner does not persist
them, and they are what runs-on matches. install now records what it
registered with so repoint and status can read it back; a runner installed
before that has nothing to read, so repoint falls back to the ci-runner
default and warns before it touches anything.
Both found by the first run on a real control-plane box — neither was
reachable by the argument-parsing tests.
$EDITOR is unset on a freshly-bootstrapped server, which is precisely rig's
target environment. The printed next-step `$EDITOR /etc/coolify-dump.env`
expanded to nothing, so bash tried to EXECUTE the 0600 bindings file and said
"Permission denied" — an error that reads like a filesystem problem and is not
one. Print `nano`.
A bare bucket name in S3_BUCKET reads to `aws` as a LOCAL path, so the upload
died with "Invalid argument type" and a usage dump — after pg_dump had run and
age had encrypted 14MB, with nothing in the error pointing at the actual
mistake. The script now validates the bindings up front: S3_BUCKET must be an
s3:// URI, S3_ENDPOINT must carry a scheme. Both fail with the value quoted and
the reason stated, before a database is read.
Note what still cannot be validated, and now says so in the script: age's X25519
header does not reveal its recipient, so a valid-but-WRONG key (staging's
instead of prod's) yields a flawless backup nobody can open. Only decrypting an
artifact proves the recipient. The printed next-steps now walk through that
read-back explicitly, from a machine holding the private key — never the box.
The dump script ships as an embedded heredoc, so a typo in it would first
surface at 04:00 on a live control plane. test/cli.sh now extracts it and
asserts it is valid bash and that both new guards fire.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Coolify control-plane database holds the GitHub App private key, every
registered server's SSH key, and every environment value for every environment
it manages. Backing it up was a manual runbook step, and the dump script lived
in cast — the off-box tool, whose src never references it. It runs on the box,
as root, under a scheduler: that is rig's job description.
It matters beyond tidiness. The dump is forensics, not a restore path — a lost
control plane is rebuilt fresh and reconciled from the manifest. So there will
be a next control-plane box, and as a runbook step it was born un-backed-up,
depending on someone remembering mid-incident. Now it is backed up from birth.
rig installs the machinery and templates /etc/coolify-dump.env empty at 0600,
never reading it back — no credential passes through rig. The script's own
guards make an unfilled file fail the unit loudly rather than ship plaintext.
systemd timer over cron: EnvironmentFile is the right idiom for 0600 secrets,
failures surface in systemctl status instead of being mailed into the void, and
Persistent=true catches a run missed while the box was down.
Two hazards the cast script missed, carried into the unit:
- aws-cli >= 2.23 enables default upload checksums that S3-compatible backends
reject; Debian 13 ships 2.23.6, so the unit defaults both checksum knobs to
when_required.
- A failed pg_dump piped into age still yields a valid, tiny, encrypted file
that uploads cleanly every night and looks exactly like a working backup. The
script now refuses to upload an empty artifact.
Closes#8
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On Debian, /etc/os-release defines VERSION="13 (trixie)". runner-install
sourced it into the main shell for the Debian-family guard, overwriting the
script's empty $VERSION: the latest-release resolution was skipped and the
download URL became .../v13 (trixie)/... -> curl (3) malformed URL. A
--version pin was clobbered the same way (guards run after arg parsing).
Read ID/ID_LIKE via a subshell in both runner-install and bootstrap (same
pattern, no collision there yet), and add a harness guard that fails on any
future main-shell sourcing of os-release.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
