heavy-duty/rig

Fork 1

bootstrap: add a dev role — the Incus claudebox host #19

Closed

dan-claude-bot wants to merge 1 commit from feat/bootstrap-dev-role into main

dan-claude-bot commented

2026-07-17 15:25:04 +00:00

(Migrated from github.com)

Plan

rig bootstrap shipped three roles: control-plane, workload, runner. The one machine class NOT rig-made was the Incus host that runs the claudeboxes (dev-server). It was hand-built, so "every box is rig-made, reproducibly" had a hole exactly where an agent runs. This closes it: rig bootstrap dev --hostname dev-server, same shape as the other roles — idempotent, convergent, a second run is a no-op.

What the role does. It reuses ALL the shared machinery for free — the 00-rig.conf sshd drop-in plus the sshd -T effective-config assert, system hostname convergence, and the tailscale join. On top of that it installs and initialises Incus.

The three hard constraints, each enforced rather than documented:

tag:local, never tag:server. The ACL grants tag:server → :22, so a mis-tagged dev host hands the control plane free SSH. This already happened — both M900s came up tag:server and were retagged by hand. So dev defaults TS_TAG to tag:local and refuses tag:server (exit 2), mirroring the runner refusal. The correct tag is the only reachable outcome, not a flag the operator has to remember.
The guest claudeboxes never join the tailnet. The host joins; the guests do not. An agent-inhabited box with its own tailnet node is a foothold into the control plane, so operator SSH into a claudebox goes through the host (ProxyJump), never a tunnel of its own. There is deliberately no "enrol the guests" step — a comment and the README say so, and if that convenience is ever added, it is the bug.
No credentials on the host. Claudeboxes are creds-free by design; the operator adds their own interactively. rig installs, templates, and holds no credential.

Incus init idempotency. incus admin init --auto is not idempotent (a second run errors "storage pool already exists"), which would break convergence. So a prior init is detected by the artefacts --auto leaves behind — a storage pool AND a root disk on the default profile — and re-init is skipped when both exist. State is then asserted effectively (incus profile device show default has a root disk; incus network list shows incusbr0) rather than trusting init's exit code — the repo's "assert what resolved, not the action" rule, the same discipline that caught the sshd first-wins bug.

What changed

commands/bootstrap.sh — dev added to the role case + die messages; dev defaults TS_TAG=tag:local; tag:server refusal extended to dev; dev-only Incus install + idempotent init + effective-state assert block (guarded on [ "$ROLE" = dev ]); guests-never-join comment; next: hint; usage() lists dev.
bin/rig — dev added to the bootstrap line in the usage() heredoc.
test/cli.sh — bootstrap dev --ts-tag tag:server refused (exit 2); bootstrap dev parses and refuses non-root (exit 1); mirrors the runner tests.
README.md — ### rig bootstrap dev subsection (what it does, the three constraints + reasoning, rehearsal-asserts-effective-state); role list and examples updated.

Testing

bash test/cli.sh → 68 passed, 0 failed (includes the new dev arg/refusal tests).
shellcheck -x over bin/* **/*.sh → clean.

Unit tests can only reach arg parsing and the tag:server refusal (exit 2) and the non-root guard (exit 1) — the incus install/init and the effective tag:local assertion need a real host. Those belong in the rehearsal, which must assert effective state, not files rig wrote: the existing Incus rehearsal runs in a pristine Debian container with no cloud-init drop-in and is structurally blind to the sshd first-wins bug. A dev-role rehearsal asserts sshd -T, incus info, and tailscale status --json showing tag:local, then a second bootstrap dev proving a clean no-op.

Interactions with other PRs

DEPENDS ON #17 (admin user). Per that issue, dev's eventual setup-host.sh needs a sudo-capable admin user. Merge this after #17 / rebase on it.
Textual conflicts with #16 and #17 — all three edit commands/bootstrap.sh (the role case and the tailscale block especially). Expect to rebase.
⚠️ Coordinate with #16. #16 removes --ts-tag and makes the tag come from the pre-auth key, with post-join verification of .Self.Tags. This PR still uses --advertise-tags=tag:local (main's model). If #16 merges first, dev's tag policy must move from --advertise-tags=tag:local onto asserting the effective tag is tag:local (via .Self.Tags) after join — same intent (tag:local is the only reachable outcome, tag:server refused), different mechanism. Flagging clearly for the maintainer.

Closes #12

## Plan `rig bootstrap` shipped three roles: control-plane, workload, runner. The one machine class NOT rig-made was the **Incus host that runs the claudeboxes** (dev-server). It was hand-built, so "every box is rig-made, reproducibly" had a hole exactly where an agent runs. This closes it: `rig bootstrap dev --hostname dev-server`, same shape as the other roles — idempotent, convergent, a second run is a no-op. **What the role does.** It reuses ALL the shared machinery for free — the `00-rig.conf` sshd drop-in **plus** the `sshd -T` effective-config assert, system hostname convergence, and the tailscale join. On top of that it installs and initialises Incus. **The three hard constraints, each enforced rather than documented:** - **`tag:local`, never `tag:server`.** The ACL grants `tag:server → :22`, so a mis-tagged dev host hands the control plane free SSH. This already happened — both M900s came up `tag:server` and were retagged by hand. So `dev` defaults `TS_TAG` to `tag:local` and **refuses `tag:server`** (exit 2), mirroring the runner refusal. The correct tag is the *only* reachable outcome, not a flag the operator has to remember. - **The guest claudeboxes never join the tailnet.** The **host** joins; the **guests** do not. An agent-inhabited box with its own tailnet node is a foothold into the control plane, so operator SSH into a claudebox goes *through* the host (ProxyJump), never a tunnel of its own. There is deliberately no "enrol the guests" step — a comment and the README say so, and if that convenience is ever added, it is the bug. - **No credentials on the host.** Claudeboxes are creds-free by design; the operator adds their own interactively. rig installs, templates, and holds no credential. **Incus init idempotency.** `incus admin init --auto` is *not* idempotent (a second run errors "storage pool already exists"), which would break convergence. So a prior init is detected by the artefacts `--auto` leaves behind — a storage pool AND a root disk on the `default` profile — and re-init is skipped when both exist. State is then asserted **effectively** (`incus profile device show default` has a root disk; `incus network list` shows `incusbr0`) rather than trusting `init`'s exit code — the repo's "assert what resolved, not the action" rule, the same discipline that caught the sshd first-wins bug. ## What changed - `commands/bootstrap.sh` — `dev` added to the role case + die messages; `dev` defaults `TS_TAG=tag:local`; tag:server refusal extended to `dev`; dev-only Incus install + idempotent init + effective-state assert block (guarded on `[ "$ROLE" = dev ]`); guests-never-join comment; `next:` hint; usage() lists `dev`. - `bin/rig` — `dev` added to the bootstrap line in the usage() heredoc. - `test/cli.sh` — `bootstrap dev --ts-tag tag:server` refused (exit 2); `bootstrap dev` parses and refuses non-root (exit 1); mirrors the runner tests. - `README.md` — `### rig bootstrap dev` subsection (what it does, the three constraints + reasoning, rehearsal-asserts-effective-state); role list and examples updated. ## Testing - `bash test/cli.sh` → **68 passed, 0 failed** (includes the new dev arg/refusal tests). - `shellcheck -x` over `bin/* **/*.sh` → **clean**. Unit tests can only reach arg parsing and the tag:server refusal (exit 2) and the non-root guard (exit 1) — the incus install/init and the effective `tag:local` assertion need a real host. Those belong in the **rehearsal**, which must assert *effective* state, not files rig wrote: the existing Incus rehearsal runs in a pristine Debian container with no cloud-init drop-in and is structurally blind to the sshd first-wins bug. A dev-role rehearsal asserts `sshd -T`, `incus info`, and `tailscale status --json` showing `tag:local`, then a second `bootstrap dev` proving a clean no-op. ## Interactions with other PRs - **DEPENDS ON #17 (admin user).** Per that issue, dev's eventual `setup-host.sh` needs a sudo-capable admin user. Merge this after #17 / rebase on it. - **Textual conflicts with #16 and #17** — all three edit `commands/bootstrap.sh` (the role case and the tailscale block especially). Expect to rebase. - **⚠️ Coordinate with #16.** #16 *removes* `--ts-tag` and makes the tag come from the pre-auth **key**, with post-join verification of `.Self.Tags`. This PR still uses `--advertise-tags=tag:local` (main's model). **If #16 merges first, dev's tag policy must move** from `--advertise-tags=tag:local` onto asserting the *effective* tag is `tag:local` (via `.Self.Tags`) after join — same intent (tag:local is the only reachable outcome, tag:server refused), different mechanism. Flagging clearly for the maintainer. Closes #12

claude-bot-andresmgsl (Migrated from github.com) reviewed 2026-07-17 15:25:04 +00:00

codex-bot-andresmgsl (Migrated from github.com) reviewed 2026-07-17 15:25:04 +00:00

grok-bot-andresmgsl (Migrated from github.com) reviewed 2026-07-17 15:25:04 +00:00

dan-claude-bot commented

2026-07-17 15:44:28 +00:00

(Migrated from github.com)

Parking this as a draft pending the user-modeling decision in #17.

Two things need rework before this is right, and both hinge on #17:

Delegate to box setup-host, don't reinvent incus. This PR does a bare incus admin init --auto (dir backend, default profile). The real target is heavy-duty/box's setup-host, which builds the actual claudebox host stack: a btrfs pool (CoW clones — --auto/dir makes every box clone a full multi-GB copy), the boxnet isolated NAT bridge, the box-isolate egress ACL (drops all RFC1918/CGNAT/link-local), DNS pinning, box-to-box firewall isolation, and the box-net profile. So bootstrap dev should install box (curl .../install.sh | bash) and run box setup-host, leaving the host ready to box new.
Who runs box? box setup-host is written for a non-root sudo user — it sudos, adds $USER to incus-admin, and exit 0s telling you to re-login. Run as root out of rig bootstrap, it returns success having done half the job (exactly the 'claimed success without checking effective state' failure the sshd first-wins bug taught this repo). That means the dev role needs a real user model — which is #17, deliberately parked for proper design. Per this issue's own text, #17 is a prerequisite for #12, so we're revisiting both together rather than guessing the user model here and redoing it.

The base of this PR (hardening + hostname + tag:local enforcement + refusing tag:server) is sound and stays; only the incus step and the user model change.

Parking this as a **draft** pending the user-modeling decision in #17. Two things need rework before this is right, and both hinge on #17: 1. **Delegate to `box setup-host`, don't reinvent incus.** This PR does a bare `incus admin init --auto` (dir backend, default profile). The real target is [heavy-duty/box](https://github.com/heavy-duty/box)'s `setup-host`, which builds the actual claudebox host stack: a **btrfs** pool (CoW clones — `--auto`/`dir` makes every box clone a full multi-GB copy), the `boxnet` isolated NAT bridge, the `box-isolate` egress ACL (drops all RFC1918/CGNAT/link-local), DNS pinning, box-to-box firewall isolation, and the `box-net` profile. So `bootstrap dev` should install box (`curl .../install.sh | bash`) and run `box setup-host`, leaving the host ready to `box new`. 2. **Who runs box?** `box setup-host` is written for a **non-root sudo user** — it `sudo`s, adds `$USER` to `incus-admin`, and `exit 0`s telling you to re-login. Run as root out of `rig bootstrap`, it returns *success* having done half the job (exactly the 'claimed success without checking effective state' failure the sshd first-wins bug taught this repo). That means the dev role needs a real user model — which is #17, deliberately parked for proper design. Per this issue's own text, **#17 is a prerequisite for #12**, so we're revisiting both together rather than guessing the user model here and redoing it. The base of this PR (hardening + hostname + `tag:local` enforcement + refusing `tag:server`) is sound and stays; only the incus step and the user model change.

dan-claude-bot commented

2026-07-18 16:43:30 +00:00

(Migrated from github.com)

Closing as superseded by #29, per its description: the traits model (#27) and host-class box install (#28) landed the bulk of #12 on main, and #29 delivers the remnants. This branch predates the traits redesign and is CONFLICTING against main. See the merge-order plan on #29/#30.

Pull request closed

This pull request cannot be reopened because the branch was deleted.

No reviewers

claude-bot-andresmgsl

codex-bot-andresmgsl

grok-bot-andresmgsl

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: heavy-duty/rig#19

No description provided.