diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..845da81 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,32 @@ +name: ci +on: + push: + branches: [main] + pull_request: +jobs: + check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: shellcheck + # -x follows `source`/`.` directives; box has no lib split today, but the + # flag costs nothing and keeps the invocation identical to rig's. + # globstar so a script in a new subdirectory is linted without anyone + # remembering to edit this list; bin/* covers the extensionless entrypoint + # (bin/box). The file list is printed so under-coverage shows up in the log. + run: | + shopt -s globstar + files=(bin/* **/*.sh) + printf 'shellcheck: %s\n' "${files[@]}" + shellcheck -x "${files[@]}" + - name: cli tests + run: bash test/cli.sh + + # NOT run here: the drill (drill/drill.sh) and the multi-user rehearsal + # (drill/multiuser.sh). Both stand up a real Incus, rearrange the host's + # network/firewall, and — for the rehearsal — create throwaway system users. + # None of that belongs in CI: it needs a disposable host and root, and it is + # the answer to #72 Task 0's substrate question, which a container runner + # cannot answer. CI stays static + dependency-free (shellcheck + cli.sh), + # exactly as this repo's design demands ("isolation claims are tested on a + # real host, never reasoned about" — docs/box-design.md). diff --git a/CHANGELOG.md b/CHANGELOG.md index 8715206..ed2eded 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,97 @@ History before 0.5.0 lives in git and in [drill/RUNS.md](drill/RUNS.md), which records not just what changed but what each drill run proved. +## Unreleased + +### Added + +- **Global / root install** (#71) — run as root, box installs *once* to + `/opt/box` (world-readable) with the `box` symlink on `/usr/local/bin`, so + every operator on a shared host runs the same tree. Per-user installs are + unchanged (`$HOME/.local`); `BOX_HOME`/`BOX_BIN` still override. A per-user + tree under `/root` is `0700` and unreadable to everyone else — the whole fleet + got `command not found` — so the root branch lands in a system location and + `chmod -R a+rX`'s it (read for files, +search on dirs), guarded on root. This + unblocks "rig installs box" (rig#24's `box` role). +- **CI + a test suite** — `.github/workflows/ci.yml` (a `check` job: globstar + `shellcheck -x` over `bin/* **/*.sh`, then `bash test/cli.sh`) and `test/cli.sh`, + dependency-free and runnable by a non-root user with no Incus. It exercises the + `install.sh` DEST/BINDIR branch functionally (both tiers + `BOX_HOME`/`BOX_BIN` + overrides), the CLI contract, and grep-guards the daemon-gated invariants and + tmux in every template — the box was the repo with "no tests and no CI". + +### Fixed + +- **`box tmux` works on every template** (#65) — `box tmux` runs + `tmux new-session` *inside* the box, but the templates did not install tmux, so + it failed with `tmux: command not found`. `tmux` is now in each template's + cloud-init package list (`blank`/`claude`/`codex`/`grok`). + +- **`box setup-host` finishes in one run** (#63). When it had to add you to + `incus-admin` it stopped there and told you to re-login and re-run — an + `exit 0` that reported success having built none of the stack: no `boxnet`, + no ACL, no `box-net` profile, no firewall. It now re-execs itself under + `sg incus-admin` and completes in that one invocation. The membership check + was also asking the wrong question: `id -nG "$USER"` reads the group + database, which lists the group the moment `usermod` returns, so a + same-session re-run passed the check with credentials that still lacked the + group and died further down on a bare permission error from `incus`. Argless + `id -nG` asks the process what it actually holds. + +- **`setup-host` works as root, with or without `sudo`** — every privileged + call was a hardcoded `sudo`, so on a minimal root image (no `sudo` package) + it died on `sudo: command not found` before doing anything. Privilege is now + resolved once: nothing at UID 0, `sudo` otherwise, and a clear error if + neither is possible. This is what made `install.sh`'s root path real rather + than nominal. +- **`setup-host` grants `incus-admin` to the human, not to root** — under + `sudo install.sh` it would have added `root` to the group: a no-op (UID 0 + opens the socket regardless) that also left the actual user locked out of + their own boxes. It now derives the login user from `SUDO_USER`. +- **`box-firewall.service` now reports its state honestly** — the unit is + `Type=oneshot` and was missing `RemainAfterExit=yes`, so it went + `inactive (dead)` the instant it succeeded: a host whose isolation was + perfectly live read as one whose firewall unit had died. drill.sh sends you + to `systemctl status box-firewall` to diagnose exactly that, and + setup-host.sh's own comment already asserted the unit "is RemainAfterExit" — + it was not. Found by running the drill on a real host and mistrusting the + green: `nft list table bridge box` showed the drop live while the unit read + dead. `restart` was and remains correct either way. +- **`setup-host`'s apt calls can no longer hang** — a fresh cloud image has + `apt-daily`/`unattended-upgrades` holding the dpkg lock, and a plain + `apt-get install` waits on it silently and indefinitely. Now bounded + (`DPkg::Lock::Timeout=300`) and non-interactive, which matters because + `install.sh` runs it with nobody watching. + +### Changed + +- **`drill.sh` proves the new contract instead of masking it** — the drill ran + `setup-host` itself right after installing, so the stack existed by its own + hand and a run passed identically whether or not `install.sh` had done a + thing; a fresh run converged the stack three times, while the messages still + described the pre-#63 "first pass may only add you to the group" behaviour. + It now asserts the post-install stack in-group before touching the host, and + runs `setup-host` exactly once more — after the clean, which deliberately + unsets `dns.mode` and so has to be converged back. `DRILL_OWNS_SETUP=1` + hands sequencing back to the drill. Pre-setup tripwires now read *before* + `install.sh`, since that is what triggers setup now. +- **`install.sh` asks, sets up the host, and no-ops on re-run** (#64) — it now + prompts *"Install box?"*, then on a fresh host installs the tree and asks a + second question, *"Set up this machine as a box host now?"*, running the whole + isolation stack if you say yes (previously it only printed a warning and left + you a command, so the install reported success and `box new` died on a host + with no Incus). Prompts read `/dev/tty`, since under `curl | bash` the script + itself is stdin; `BOX_YES=1` answers yes unattended (required where there is + no terminal), `BOX_SKIP_SETUP_HOST=1` declines the host-setup step. +- **`install.sh` never overwrites an existing install** — if box is already + installed it says so and changes nothing, so a stray re-run can no longer + clobber a working tree or rebuild the host stack under live boxes. Upgrading + is explicit: uninstall (`rm -rf ~/.local/share/box ~/.local/bin/box`, boxes + preserved first) and install fresh. This replaces the earlier version-diff + refusal with a simpler rule that dissolves the same class of errors. The + version-aware upgrade that migrates boxes instead is #67; a portable + `box export` so a box survives its own deletion is #70. + ## 0.5.0 — 2026-07-15 The release the project was renamed in: the repo is `heavy-duty/box`, matching diff --git a/README.md b/README.md index 94aa408..8317fe0 100644 --- a/README.md +++ b/README.md @@ -40,14 +40,56 @@ design rationale. curl -fsSL https://raw.githubusercontent.com/heavy-duty/box/main/install.sh | bash ``` -Installs the tree to `~/.local/share/box` and links `box` onto your -`PATH`. Re-run any time to upgrade — upgrading from a pre-0.4.0 install also -retires the old `claudebox` symlink. (No `git clone` needed.) +It asks first — **"Install box?"** — then, if box is not already installed, +downloads the tree to `~/.local/share/box`, links `box` onto your `PATH`, and +asks a second question: **"Set up this machine as a box host now?"** Say yes and +it builds the whole isolation stack for you (it may ask for `sudo`); say no and +you can run `box setup-host` later. (No `git clone` needed.) + +**Re-running is a safe no-op.** If box is already installed, the installer tells +you so and changes nothing — a stray re-run can never clobber your install or +rebuild the stack under your boxes. Upgrading is therefore explicit: uninstall +what you have and install fresh. Preserve any boxes first — `box down `, +copy out anything you need (a portable `box export` is +[#70](https://github.com/heavy-duty/box/issues/70)), then `box rm ` +(which deletes the box *and* its snapshots) — then: + +```sh +rm -rf ~/.local/share/box ~/.local/bin/box # uninstall +curl -fsSL https://raw.githubusercontent.com/heavy-duty/box/main/install.sh | bash +``` + +A version-aware upgrade that migrates boxes instead of asking you to is +[#67](https://github.com/heavy-duty/box/issues/67). For unattended installs +(CI, images), `BOX_YES=1` answers every prompt yes and `BOX_SKIP_SETUP_HOST=1` +declines the host-setup step. + +### Global vs per-user install + +Where box lands depends on **who runs the installer**, because on a shared host +box's tree is *executed by other users* — so it cannot hide in one user's home: + +- **As root → global.** The tree goes to `/opt/box` (world-readable) and the + `box` symlink to `/usr/local/bin` (already on every login `PATH`). One + install, every operator on the host runs the same `box`. This is the fleet + path: [rig](https://github.com/heavy-duty/rig)'s `box` role + ([rig#24](https://github.com/heavy-duty/rig/issues/24)) installs box once at + host bootstrap ([#71](https://github.com/heavy-duty/box/issues/71)). +- **As a normal user → per-user.** The tree goes to `~/.local/share/box` and + the symlink to `~/.local/bin` — the solo path, unchanged. Nobody else needs + to run your box. + +`BOX_HOME` / `BOX_BIN` override the destination on either path. A per-user +install under `/root` would be `0700` and unreadable to everyone else — which +is exactly the bug the root branch fixes. ## One-time host setup (Ubuntu 24.04 / Debian 13) +The installer already does this. Run it directly to set up a host you +installed with `BOX_SKIP_SETUP_HOST=1`, or to re-apply the stack by hand: + ```sh -box setup-host # run twice if it adds you to incus-admin (re-login between) +box setup-host # one run is enough ``` Idempotent. Installs Incus and creates the isolation stack: the `boxnet` NAT diff --git a/bin/box b/bin/box index 63be4de..5613ded 100755 --- a/bin/box +++ b/bin/box @@ -387,10 +387,11 @@ EOF Prepare this host to mint boxes — one time. Installs Incus and builds the isolation stack: the boxnet NAT bridge (resolver pinned), the box-isolate ACL, the box-net profile, and the firewall rules, all re-applied at boot. -Idempotent — safe to re-run after a box upgrade to pick up stack changes. +Idempotent — safe to re-run after a box upgrade to pick up stack changes; +install.sh runs it for you, so this is for re-applying by hand. -If it has to add you to the incus-admin group it will say so and exit; log -back in (or 'sg incus-admin') and run it again. +One run is enough. If it has to add you to the incus-admin group it re-runs +itself under that group — no re-login, no second invocation. box setup-host EOF @@ -640,6 +641,13 @@ load_template() { *) die "template '$t': not a KEY=\"value\" line: $line" ;; esac val="${val#\"}"; val="${val%\"}" + # T_DESC is parsed for symmetry with the other BOX_* keys, but cmd_templates + # re-reads BOX_DESCRIPTION straight from the file (a box is listed without + # ever loading its template), so the parsed value here is never read. Keep the + # row — deleting it would turn box.env's own key into an "unknown key" error at + # mint time. (SC2034 disabled for the branch below; the directive must sit on + # the whole case, not an individual arm.) + # shellcheck disable=SC2034 case "$key" in BOX_DESCRIPTION) T_DESC="$val" ;; BOX_IMAGE) T_IMAGE="$val" ;; @@ -650,7 +658,12 @@ load_template() { *) die "template '$t': unknown key '$key' — a template sets image, user and resources, nothing else (there is no key for a network, on purpose)" ;; esac done <"$dir/box.env" - [ -n "$T_IMAGE" ] && [ -n "$T_USER" ] || die "template '$t': BOX_IMAGE and BOX_USER are required" + # Not 'A && B || die': if T_IMAGE is set but T_USER is not, that idiom still + # dies (which is what we want) — but it reads as an if-then-else it is not, so + # spell the guard out (SC2015). + if [ -z "$T_IMAGE" ] || [ -z "$T_USER" ]; then + die "template '$t': BOX_IMAGE and BOX_USER are required" + fi # Resolution, most specific wins: inline flag (--cpu/--memory/--disk, #57) # > BOX_* environment (how a small host or the drill shrinks every box it # mints) > the template's file > defaults. Values pass to Incus verbatim — @@ -792,7 +805,7 @@ box_net_ip() { pfx="$(incus network get boxnet ipv4.address 2>/dev/null | cut -d/ -f1 | cut -d. -f1-3)" [ -n "$pfx" ] || return 1 incus list "$1" --format csv --columns 4 2>/dev/null \ - | tr -d '"' | tr ' ,' '\n\n' | grep -E "^${pfx//./\\.}\.[0-9]+$" | head -n1 | grep . + | tr -d '"' | tr ' ,' '\n' | grep -E "^${pfx//./\\.}\.[0-9]+$" | head -n1 | grep . } # VIRTUAL-MACHINE is a mouthful in a table; anything unexpected passes through. diff --git a/drill/doctor.sh b/drill/doctor.sh index 546c7cd..b0e7ff7 100755 --- a/drill/doctor.sh +++ b/drill/doctor.sh @@ -14,6 +14,11 @@ # # This script is the answer to "what state is the host actually in?" — the # question that kept getting answered by hand. +# +# ok/no/inf/head_ all return 0, so the 'A && ok "…" || no "…"' idiom this file +# is built on cannot hit the C-may-run-when-A-is-true trap SC2015 warns about +# (same reasoning as drill.sh's ok/no). +# shellcheck disable=SC2015 set -u FIX=0; PIN=0 diff --git a/drill/drill.sh b/drill/drill.sh index f638b3a..9bb7dbe 100755 --- a/drill/drill.sh +++ b/drill/drill.sh @@ -172,6 +172,47 @@ EOF fi phase "Installing box ($REPO@$REF)" + + # Sudo, up front and out loud. Later calls run unattended, and a password + # prompt swallowed by a '-qq' redirect looks exactly like a hang. This now + # has to precede the install too: install.sh runs the host setup itself, so + # the first thing needing root is no longer further down — it is inside the + # very next command. + sudo -v || { echo "drill: need sudo (the host setup installs packages and firewall rules)"; exit 1; } + + # Pre-setup observations must be READ BEFORE install.sh, because install.sh + # is now what runs setup-host. Read after it and setup has already had its + # chance to act, so the observation says nothing. + # setup-host.sh installs nftables itself when neither nft nor UFW exists + # (a stock Debian 13 cloud image ships neither). This is a tripwire: if it + # fires, that fix regressed. + fw_absent_pre=0 + if ! command -v nft >/dev/null 2>&1 && ! command -v ufw >/dev/null 2>&1; then + fw_absent_pre=1 + fi + + # DRILL_OWNS_SETUP=1 opts out of the installer's automatic setup and puts the + # drill back in charge of sequencing it (install, then clean, then converge). + # The DEFAULT deliberately does not: a drill that runs setup-host itself right + # after installing cannot tell you whether install.sh did its job, because the + # drill's own call would build the stack either way. The default path exercises + # what a user actually runs, and asserts the result in-group below. + OWNS="${DRILL_OWNS_SETUP:-0}" + if [ "$OWNS" = 1 ]; then + export BOX_SKIP_SETUP_HOST=1 + fi + + # The installer is a no-op when box is already installed — upgrading is + # uninstall-then-install, by design. The drill re-proves a tree from scratch + # every run, so it does the uninstall itself: clear any prior tree and symlink + # before installing, or install.sh would correctly refuse to touch them. + rm -rf "$HOME/.local/share/box" "$HOME/.local/bin/box" + + # The installer prompts (install? set up host?) and reads /dev/tty. The drill + # runs unattended with no tty, so it answers yes to everything via BOX_YES. + # OWNS still suppresses the setup prompt via BOX_SKIP_SETUP_HOST above. + export BOX_YES=1 + BOX_REPO="$REPO" BOX_REF="$REF" \ bash -c "$(curl -fsSL "https://raw.githubusercontent.com/$REPO/$REF/install.sh")" \ || { echo "install failed"; exit 1; } @@ -195,17 +236,10 @@ EOF inf "installed tree confirms: $got" phase "Host setup (Incus, boxnet, ACL, profile, firewall)" - # setup-host.sh installs nftables itself when neither nft nor UFW exists - # (a stock Debian 13 cloud image ships neither). This guard is a tripwire: - # if it fires, that fix regressed. - if ! command -v nft >/dev/null 2>&1 && ! command -v ufw >/dev/null 2>&1; then + if [ "$fw_absent_pre" = 1 ]; then note "neither nft nor ufw present pre-setup — setup-host.sh must install nftables itself (it fixed this once; watch that it still does)" fi - # Sudo, up front and out loud. Later calls run unattended, and a password - # prompt swallowed by a '-qq' redirect looks exactly like a hang. - sudo -v || { echo "drill: need sudo (the host setup installs packages and firewall rules)"; exit 1; } - # apt's lock is held by apt-daily / unattended-upgrades on a fresh cloud # image, and 'apt-get -qq >/dev/null' waits for it in COMPLETE SILENCE — # which is how run 5 looked stuck for minutes right after this header. @@ -224,16 +258,57 @@ EOF inf "incus already installed — skipping apt" fi - inf "running setup-host.sh (first pass: may only add you to incus-admin)…" - ~/.local/share/box/host/setup-host.sh || true - # The group we were just added to isn't in this shell's credentials yet. + # No setup-host call here any more. It used to run a full "first pass" that + # the comment described as "may only add you to incus-admin" — behaviour that + # no longer exists (setup-host converges in one run now, #63) and that, since + # install.sh runs setup itself (#64), was simply the stack being built a + # second time before the drill had asserted the first. + if [ "$OWNS" = 1 ]; then + # We opted out of the installer's setup, so nobody has joined us to the + # group yet. usermod ONLY: the stack build waits until after the clean + # below, which is the entire reason for owning the sequence. + inf "DRILL_OWNS_SETUP=1 — the drill owns the host setup" + id -nG | grep -qw incus-admin || sudo usermod -aG incus-admin "$USER" + else + inf "install.sh ran the host setup — asserting what it left, in-group, next" + fi + # setup-host's own sg re-exec was a CHILD of install.sh; this shell's + # credentials are untouched, so we still have to enter the group ourselves — + # once, for the remainder of the drill. inf "re-entering inside the incus-admin group…" - exec sg incus-admin -c "IN_GROUP=1 BOX_REPO='$REPO' BOX_REF='$REF' KEEP=$KEEP bash '$SELF' --in-group" + exec sg incus-admin -c "IN_GROUP=1 DRILL_OWNS_SETUP='$OWNS' BOX_REPO='$REPO' BOX_REF='$REF' KEEP=$KEEP bash '$SELF' --in-group" fi export PATH="$HOME/.local/bin:$PATH" KEEP="${KEEP:-0}" +# PROVE THE INSTALLER'S CONTRACT (#64) — first, before the clean or anything +# else on this host mutates the stack, and before the drill runs setup-host +# itself further down. That ordering is the whole point: the old flow ran +# setup-host immediately after installing, so the stack existed by the drill's +# own hand and the run passed identically whether or not install.sh had done a +# thing. This is read-only, so it is safe with a previous run's boxes still +# attached. +if [ "${DRILL_OWNS_SETUP:-0}" != 1 ]; then + phase "Asserting the stack that install.sh built" + missing="" + incus network show boxnet >/dev/null 2>&1 || missing="$missing boxnet" + incus network acl show box-isolate >/dev/null 2>&1 || missing="$missing box-isolate" + incus profile show box-net >/dev/null 2>&1 || missing="$missing box-net" + # Last thing setup-host does, so it doubles as "it ran to the end". + sudo nft list table bridge box >/dev/null 2>&1 || missing="$missing nft-bridge-box" + if [ -n "$missing" ]; then + echo "drill: FATAL — install.sh reported success but left an INCOMPLETE stack:$missing" >&2 + echo " install.sh is supposed to run the host setup itself (#64), and setup-host" >&2 + echo " is supposed to converge in one run (#63). One of those did not happen." >&2 + echo " reproduce with the output visible:" >&2 + echo " ~/.local/share/box/host/setup-host.sh" >&2 + echo " or hand setup back to the drill: DRILL_OWNS_SETUP=1 $SELF" >&2 + exit 1 + fi + ok "install.sh left a complete host stack (boxnet, box-isolate, box-net, nft bridge drop) — no second setup needed" +fi + # CLEAN BEFORE SETUP, not after. setup-host.sh reconfigures the network's ACLs, # and a previous run's boxes are still ATTACHED to that network — 'incus network # set' then has to push the change onto every live NIC, which is how run 6 @@ -277,7 +352,15 @@ done left="$(incus list --format csv --columns n 2>/dev/null | tr '\n' ' ')" [ -n "$left" ] && inf "instances still on this host (not ours, left alone): $left" -inf "running setup-host.sh (in-group pass: network, ACL, profile, firewall)…" +# This call stays, and it is NOT the install's setup repeated for its own sake: +# the clean above deliberately unsets dns.mode, which is part of the SHIPPED +# stack, and drops a previous run's phase-D mutations. Something has to put the +# host back together afterwards, and setup-host is that something — this is the +# "converge against a clean slate" the block above is ordered for. On the +# default path the install's setup has already been asserted, so what this +# proves is idempotency: a second run over a cleaned host is a no-op that +# restores the stack rather than a fresh build. +inf "running setup-host.sh (post-clean convergence: restores dns.mode and any reverted mutations)…" if ! timeout -k 10 300 ~/.local/share/box/host/setup-host.sh; then echo "drill: setup-host.sh failed or timed out (>5 min)." >&2 echo " it should take seconds on a host that already has incus. usual causes:" >&2 diff --git a/drill/wipe.sh b/drill/wipe.sh index e368f5a..a6c4ed0 100644 --- a/drill/wipe.sh +++ b/drill/wipe.sh @@ -19,6 +19,11 @@ # NOT 'set -e': on a wipe, a step that finds nothing to remove is success, # not failure. Every removal states what it did; silence is never trusted # (the exit-code lesson, again). +# +# The file is one long 'removal && say "did X" || say "X failed"'. say always +# returns 0, so the C-may-run-when-A-is-true trap SC2015 warns about cannot fire +# here (same reasoning as drill.sh's ok/no). +# shellcheck disable=SC2015 set -u YES=0; PURGE_STORAGE=0 diff --git a/host/box-firewall.service b/host/box-firewall.service index 9b78370..de949a5 100644 --- a/host/box-firewall.service +++ b/host/box-firewall.service @@ -6,6 +6,13 @@ Wants=network.target [Service] Type=oneshot +# The unit's state must mean "the rules are applied", because that is the +# question everyone asks it — drill.sh sends you to 'systemctl status +# box-firewall' to diagnose exactly that. Without this, a oneshot goes +# 'inactive (dead)' the moment it succeeds, so a host whose isolation is +# perfectly live reads as a host whose firewall unit died. setup-host.sh's +# comment already assumed this was set; it was not. +RemainAfterExit=yes ExecStart=/usr/local/sbin/box-firewall [Install] diff --git a/host/migrate-host.sh b/host/migrate-host.sh index 688e920..e7bf238 100644 --- a/host/migrate-host.sh +++ b/host/migrate-host.sh @@ -19,9 +19,13 @@ # # NOT 'set -e' around the per-box work: a box that fails one step is reported # and skipped, not a crash that abandons the rest mid-migration. +# +# The report idiom is 'action && say "did X" || warn/die': say and warn always +# return 0, so the C-may-run-when-A-is-true trap SC2015 warns about cannot fire +# on those lines (same reasoning as drill.sh's ok/no). +# shellcheck disable=SC2015 set -u -GW_NEW=10.88.0.1 say() { printf 'migrate: %s\n' "$*"; } warn() { printf 'migrate: WARNING: %s\n' "$*" >&2; } die() { printf 'migrate: ERROR: %s\n' "$*" >&2; exit 1; } @@ -86,9 +90,9 @@ rehome_one() { # 3. VERIFY THE EFFECT, not the exit codes (the whole repo's lesson). The box # must be on 10.88 and actually resolve+reach the internet on its new leg # before we call it migrated. - local i ip + local _i ip ip="" - for i in $(seq 1 30); do + for _i in $(seq 1 30); do ip="$(incus exec "$b" -- ip -4 -o addr show scope global /dev/null \ | awk '{for(i=1;i/dev/null; then - sudo apt-get update - sudo apt-get install -y incus +# How we reach root, decided once. 'sudo' cannot be hardcoded: at UID 0 it is +# unnecessary, and on a minimal root image it is not installed at all — this +# script died on 'sudo: command not found' before doing anything, which made +# install.sh's deliberate root path unusable on exactly the hosts it was for. +if [ "$(id -u)" -eq 0 ]; then + SUDO="" +elif command -v sudo >/dev/null 2>&1; then + SUDO="sudo" +else + echo "ERROR: host setup needs root and 'sudo' was not found." >&2 + echo " re-run this as root: $self" >&2 + exit 1 fi -if ! id -nG "$USER" | grep -qw incus-admin; then - sudo usermod -aG incus-admin "$USER" - echo "NOTE: added $USER to incus-admin — re-login (or 'sg incus-admin') and re-run." - exit 0 +# apt, unattended-safe. install.sh now runs us without a human watching, and +# a fresh cloud image has apt-daily/unattended-upgrades holding the dpkg lock +# for the first minutes of its life — plain 'apt-get install' then waits on it +# in complete silence, indefinitely. Bound the wait and never prompt. +# 'env', not a bare VAR=val prefix: bash recognises assignments at PARSE time, +# so with $SUDO empty (we are root) 'DEBIAN_FRONTEND=x apt-get' would have +# already been parsed as a plain word and bash would try to EXECUTE it — +# 'DEBIAN_FRONTEND=noninteractive: command not found'. env is immune. +apt_get() { + $SUDO env DEBIAN_FRONTEND=noninteractive apt-get -o DPkg::Lock::Timeout=300 "$@" +} + +if ! command -v incus >/dev/null; then + apt_get update + apt_get install -y incus +fi + +if [ "$(id -u)" -eq 0 ]; then + # Root needs no group: UID 0 opens /var/lib/incus/unix.socket regardless of + # who owns it, and there is nothing to re-exec into. The HUMAN needs it — and + # under 'sudo install.sh' that is SUDO_USER, not the root we are running as. + # Adding root to incus-admin would be a no-op that also left the actual user + # locked out of their own boxes. + # NOTE: 'id -nG "$name"' here is deliberate and NOT the bug fixed below. That + # bug was asking the DATABASE about our own process; this asks the database + # about someone else's account, which is the only thing it can be asked. + login_user="${SUDO_USER:-}" + if [ -n "$login_user" ] && [ "$login_user" != root ]; then + if ! id -nG "$login_user" | grep -qw incus-admin; then + usermod -aG incus-admin "$login_user" + echo "added $login_user to incus-admin — log out and back in for your shell to pick it up" + fi + fi +# Group membership is a property of THIS PROCESS's credentials, not of the group +# database — and the two disagree for exactly as long as it matters here. +# 'id -nG "$USER"' names a user, so it reads /etc/group and reports incus-admin +# the instant usermod returns; the running shell's own credentials still lack +# it, because supplementary groups are fixed at login. So the old check passed +# on a same-session re-run, sailed into the incus calls below, and died on a +# permission error that named neither the group nor the re-login. Argless +# 'id -nG' asks the process what it actually holds, which is what incus checks +# when it opens /var/lib/incus/unix.socket. +elif ! id -nG | grep -qw incus-admin; then + $SUDO usermod -aG incus-admin "$USER" + # Then finish the job rather than adjourning it. Exiting 0 here was a + # success-shaped no-op: no boxnet, no ACL, no box-net profile, no firewall — + # and the burden of knowing that on the reader of a NOTE (#63). 'sg' runs us + # again with the new group in our credentials, no re-login, one invocation. + # The guard makes that at most one hop: if sg somehow lands without the + # group, we fail loudly instead of forking forever. + if [ -z "${BOX_SETUP_HOST_REEXEC:-}" ]; then + echo "added $USER to incus-admin — re-running under the new group (no re-login needed)" + export BOX_SETUP_HOST_REEXEC=1 + exec sg incus-admin -c "$(printf '%q ' bash "$self" "$@")" + fi + echo "ERROR: still not in incus-admin after usermod + sg." >&2 + echo " log out and back in, then re-run: box setup-host" >&2 + exit 1 fi # Storage pool + base config (safe to re-run: skipped once the pool exists). @@ -28,7 +92,7 @@ fi # says so. if ! incus storage show default >/dev/null 2>&1; then driver=btrfs - command -v mkfs.btrfs >/dev/null 2>&1 || sudo apt-get install -y btrfs-progs || driver=dir + command -v mkfs.btrfs >/dev/null 2>&1 || apt_get install -y btrfs-progs || driver=dir if ! incus admin init --preseed </dev/null 2>&1 && ! command -v nft >/dev/null 2>&1; then - sudo apt-get install -y nftables + apt_get install -y nftables fi -sudo install -m 755 "$here/host/box-firewall.sh" /usr/local/sbin/box-firewall -sudo install -m 644 "$here/host/box-firewall.service" /etc/systemd/system/ -sudo systemctl daemon-reload -sudo systemctl enable box-firewall.service +$SUDO install -m 755 "$here/host/box-firewall.sh" /usr/local/sbin/box-firewall +$SUDO install -m 644 "$here/host/box-firewall.service" /etc/systemd/system/ +$SUDO systemctl daemon-reload +$SUDO systemctl enable box-firewall.service # RESTART, not 'enable --now'. The unit is RemainAfterExit, so once it has run # it stays "active" forever — and 'enable --now' does nothing to an active unit. # Re-running setup-host after upgrading the tool therefore installed the new # rules to /usr/local/sbin and never applied them: the host kept the old # firewall, silently, and the box→box hole stayed open through a release that # claimed to close it. Restart re-runs the script, which is idempotent by design. -sudo systemctl restart box-firewall.service +$SUDO systemctl restart box-firewall.service # Profile — box-net, the placement contract: the isolated NIC and the root # disk, nothing a template controls (resources are stamped per-instance from @@ -143,7 +207,7 @@ incus profile edit box-net < "$here/profiles/box-net.yaml" # The sibling drop is the one rule whose absence is invisible: everything keeps # working, and boxes can simply reach each other. Assert it landed. -if sudo nft list table bridge box >/dev/null 2>&1; then +if $SUDO nft list table bridge box >/dev/null 2>&1; then echo "Isolation: box-to-box drop is live (nft bridge table 'box')." else echo "WARNING: the box-to-box drop is NOT active — boxes can reach each other." >&2 diff --git a/install.sh b/install.sh index 1d269e0..a894f7d 100755 --- a/install.sh +++ b/install.sh @@ -10,17 +10,69 @@ set -euo pipefail REPO="${BOX_REPO:-heavy-duty/box}" REF="${BOX_REF:-main}" -DEST="${BOX_HOME:-$HOME/.local/share/box}" -BINDIR="${BOX_BIN:-$HOME/.local/bin}" +# Root installs GLOBALLY, non-root installs per-user. box's install tree is +# EXECUTED by other users (the multi-user host path: rig installs box once, every +# incus-group operator runs it) — unlike rig, which is root-only and can hide in +# /root. So a root install must land in a system location, not $HOME: /root is +# 0700, so a $HOME/.local tree there is unreadable to everyone else and the whole +# fleet gets 'command not found' (#71). /opt/box is the world-readable system +# tree; /usr/local/bin is already on every login PATH. BOX_HOME/BOX_BIN still win. +if [ "$(id -u)" -eq 0 ]; then + DEST="${BOX_HOME:-/opt/box}" + BINDIR="${BOX_BIN:-/usr/local/bin}" +else + DEST="${BOX_HOME:-$HOME/.local/share/box}" + BINDIR="${BOX_BIN:-$HOME/.local/bin}" +fi log() { printf 'box-install: %s\n' "$*"; } warn() { printf 'box-install: WARNING: %s\n' "$*" >&2; } die() { printf 'box-install: ERROR: %s\n' "$*" >&2; exit 1; } +# Ask a yes/no question and echo the answer. The wrinkle: under the intended +# 'curl … | bash', THIS SCRIPT is stdin — so a plain 'read' would consume the +# installer's own remaining lines, not the user's keystroke. Prompts therefore +# read the terminal directly via /dev/tty. When there is no terminal at all (CI, +# a pipe with no tty), there is nobody to ask: BOX_YES=1 means "assume yes to +# every prompt" and is how automation and the drill drive this unattended; +# without it we refuse rather than silently assume consent. +confirm() { # $1 = question + [ -n "${BOX_YES:-}" ] && return 0 + if ! { true >/dev/tty; } 2>/dev/null; then + die "no terminal to confirm on. Re-run with BOX_YES=1 to proceed non-interactively (assumes yes to all prompts)." + fi + local reply + printf 'box-install: %s [y/N] ' "$1" >/dev/tty + read -r reply /dev/null 2>&1 || die "curl is required but was not found. Please install curl and re-run." command -v tar >/dev/null 2>&1 || die "tar is required but was not found. Please install tar and re-run." +# --- confirm, then no-op if already installed ------------------------------ +# Prompt BEFORE downloading anything: the first thing a curl|bash should do is +# ask whether you meant to. Then, if box is already installed, this run changes +# nothing and says so — a re-run is a safe no-op, which dissolves the whole +# "curl clobbered my working install / rebuilt the stack under my boxes" class +# of failures. Upgrading is deliberately NOT an in-place overwrite: you uninstall +# what you have (dealing with your boxes as you do) and install fresh. +confirm "Install box from $REPO@$REF?" || die "cancelled — nothing was changed." + +if [ -e "$BINDIR/box" ] || [ -x "$DEST/bin/box" ]; then + cur="$(cat "$DEST/INSTALLED_FROM" 2>/dev/null || echo '')" + cur_ver="$(cat "$DEST/VERSION" 2>/dev/null || echo '?')" + log "box is already installed ($cur, version $cur_ver) — nothing to do." + log "To install a different version, remove the current one first:" + log " · preserve any boxes you care about — 'box down ', then keep them" + log " (a portable 'box export' is #70; for now copy what you need OUT via" + log " 'box shell'/'box exec'), and 'box rm ' when you are done" + log " · uninstall: rm -rf \"$DEST\" \"$BINDIR/box\"" + log " · then re-run this installer" + exit 0 +fi + # --- temp workspace -------------------------------------------------------- TMPDIR="$(mktemp -d)" cleanup() { rm -rf "$TMPDIR"; } @@ -47,14 +99,25 @@ EXTRACTED="$(find "$TMPDIR" -mindepth 1 -maxdepth 1 -type d | head -n1)" [ -n "$EXTRACTED" ] || die "could not find the extracted source directory in archive" [ -f "$EXTRACTED/bin/box" ] || die "archive does not contain bin/box — is $REPO@$REF correct?" -# --- atomically replace $DEST --------------------------------------------- +# --- install into $DEST ---------------------------------------------------- +# Reached only on a host with no existing install (the no-op check above +# exits otherwise), so this is always a fresh tree, never an overwrite. log "installing into $DEST" -rm -rf "$DEST" mkdir -p "$(dirname "$DEST")" mv "$EXTRACTED" "$DEST" chmod +x "$DEST/bin/box" +# A global (root) install is run by OTHER users, but mv preserves the tarball's +# root:root ownership and GitHub's archives carry no world bits on some paths — so +# without this, a non-root caller cannot even traverse into $DEST to reach bin/box. +# Root owns the tree, nobody else writes it, everybody reads it. a+rX: read on +# files, +search (x) on directories only. Guarded on root so the per-user install +# stays byte-identical to before. +if [ "$(id -u)" -eq 0 ]; then + chmod -R a+rX "$DEST" +fi + # --- put box on PATH ------------------------------------------------------- mkdir -p "$BINDIR" ln -sf "$DEST/bin/box" "$BINDIR/box" @@ -84,16 +147,51 @@ case ":$PATH:" in ;; esac -# --- environment check ----------------------------------------------------- -if ! command -v incus >/dev/null 2>&1; then - warn "incus was not found. box needs Incus on the host." - warn " run the one-time host setup: $DEST/host/setup-host.sh" -fi - # Record WHAT was installed, so a caller can assert it got what it asked for. # Without this, an installer invoked with stale env vars (the CLAUDEBOX_* names # retired in 0.5.0) silently falls back to the defaults and installs main — # and the caller drills the wrong tree, believing it drilled its branch. +# Written BEFORE host setup: this records the install, which has now happened, +# and it must not hinge on whether the host stack came up. printf '%s@%s\n' "$REPO" "$REF" > "$DEST/INSTALLED_FROM" -log "done ($REPO@$REF) — try: box new --name test" +# --- host setup (second prompt) -------------------------------------------- +# The tool is installed; the machine is not yet a box host. Offer to finish the +# job — build Incus and the isolation stack — rather than leave 'box new' to die +# later on a host with no boxnet and no profile (#64). This is its own decision: +# you might be installing the CLI on a workstation and hosting boxes elsewhere. +# BOX_SKIP_SETUP_HOST=1 answers "no" without prompting (image builds, a host set +# up by hand); BOX_YES answers "yes". +setup_ok="" +setup_declined="" +if [ -n "${BOX_SKIP_SETUP_HOST:-}" ]; then + log "skipping host setup (BOX_SKIP_SETUP_HOST is set)." + setup_declined=1 +elif [ "$(id -u)" -ne 0 ] && ! command -v sudo >/dev/null 2>&1; then + warn "cannot set up the host: it needs root and sudo was not found." + warn " run this as root to finish: $DEST/host/setup-host.sh" + setup_declined=1 +elif confirm "Set up this machine as a box host now? (installs Incus + the isolation stack; needs sudo)"; then + # +# Runs cmd, asserts exit code and (if non-empty) that combined output +# contains want_substr. +check() { + local desc="$1" want="$2" substr="$3"; shift 3 + local out rc + out="$("$@" 2>&1)"; rc=$? + if [ "$rc" -ne "$want" ]; then + echo "FAIL: $desc — exit $rc, wanted $want" + printf '%s\n' "$out" | sed 's/^/ /' + FAIL=$((FAIL + 1)); return + fi + if [ -n "$substr" ] && ! printf '%s' "$out" | grep -qF -e "$substr"; then + echo "FAIL: $desc — output missing '$substr'" + printf '%s\n' "$out" | sed 's/^/ /' + FAIL=$((FAIL + 1)); return + fi + echo "ok: $desc"; PASS=$((PASS + 1)) +} + +BOX="$ROOT/bin/box" + +# --------------------------------------------------------------------------- +# The CLI contract: dispatch, help, usage errors. No incus needed — these all +# resolve before any daemon call. Exit codes are box's own (0 ok / 1 wrong / +# 2 you-asked-wrong), read straight from bin/box and confirmed by running it. +# --------------------------------------------------------------------------- +# box with no args is 'help' (cmd="${1:-help}"), which prints the general usage +# and exits 0 — NOT rig's exit-2 bare-usage. Assert box's actual contract. +check "no args → general help, exit 0" 0 "USAGE" "$BOX" +check "no args help names the command form" 0 "box " "$BOX" +check "--help exits 0" 0 "USAGE" "$BOX" --help +check "-h exits 0" 0 "USAGE" "$BOX" -h +check "help exits 0" 0 "USAGE" "$BOX" help +check "help → that command's usage" 0 "usage: box new" "$BOX" help new +check "--version exits 0" 0 "box" "$BOX" --version +# Unknown command is a usage error (2), and it says so — the suggester may add a +# 'did you mean', but the stem is stable. +check "unknown command exits 2" 2 "unknown command" "$BOX" frobnicate +check "unknown command points at help" 2 "box help" "$BOX" zzzzzz +# Options before the command are the classic mistake; box names the fix. +check "option before command exits 2" 2 "options come after the command" "$BOX" --json list +# A missing required positional is a usage error carrying that command's synopsis. +check "new without --name exits 2" 2 "usage: box new" "$BOX" new +check "shell without a box exits 2" 2 "usage: box shell" "$BOX" shell +check "restore without arg2 needs a box first" 2 "usage: box restore" "$BOX" restore +# An unknown flag is refused, not swallowed as a positional (the --labl bug). +check "unknown flag on list exits 2" 2 "unknown option" "$BOX" list --nope +# A flag that needs a value and gets none. +check "--name with no value exits 2" 2 "--name needs a value" "$BOX" new --name + +# --------------------------------------------------------------------------- +# A shim `id` on PATH: lets us drive install.sh's DEST branch with a canned uid + +# group output, exactly the way rig drives assert_runner_repo against fixtures. +# --------------------------------------------------------------------------- +SHIMDIR="$(mktemp -d)" +cat > "$SHIMDIR/id" <<'SHIM' +#!/usr/bin/env bash +# Fake `id`: -u prints $FAKE_UID, -nG prints $FAKE_GROUPS. Just enough for +# install.sh's DEST branch, which only ever asks these two. +case "${1:-}" in + -u) printf '%s\n' "${FAKE_UID:-1000}" ;; + -nG) printf '%s\n' "${FAKE_GROUPS:-}" ;; + *) exit 0 ;; +esac +SHIM +chmod +x "$SHIMDIR/id" + +# --------------------------------------------------------------------------- +# install.sh — #71 global/root install. bash -n first, then drive the actual +# DEST/BINDIR branch with the shim id (the functional proof the contract asks +# for), then grep the root-only pieces that a daemon-free run cannot exercise. +# --------------------------------------------------------------------------- +check "install.sh is valid bash" 0 "" bash -n "$ROOT/install.sh" +# Extract EXACTLY the DEST/BINDIR if/else/fi (the first `id -u -eq 0` block) and +# print what it resolved — the same "run the pure block in isolation" trick rig +# uses for its embedded dump script. Fail closed: a mangled extraction is caught +# by the /opt/box grep below before any resolution is trusted. +DBLOCK="$(mktemp)" +awk '/id -u.*-eq 0/{f=1} f{print} f&&/^fi$/{exit}' "$ROOT/install.sh" > "$DBLOCK" +# The $DEST/$BINDIR here are LITERAL text appended into the extracted block — they +# must expand when that block RUNS, not when this printf writes it. Hence single +# quotes; SC2016 is the intent. +# shellcheck disable=SC2016 +printf '\nprintf "DEST=%%s BINDIR=%%s\\n" "$DEST" "$BINDIR"\n' >> "$DBLOCK" +check "install.sh: DEST block extracted (guards the awk)" 0 "/opt/box" cat "$DBLOCK" +check "install.sh: the extracted DEST block is valid bash" 0 "" bash -n "$DBLOCK" + +dest() { # dest [extra env assignments...] — resolve DEST/BINDIR + local uid="$1"; shift + FAKE_UID="$uid" HOME=/home/tester PATH="$SHIMDIR:$PATH" env "$@" bash "$DBLOCK" +} +# Root: the global path — a system tree other users can read (#71). +check "install.sh: root → DEST=/opt/box" 0 "DEST=/opt/box" dest 0 +check "install.sh: root → BINDIR=/usr/local/bin" 0 "BINDIR=/usr/local/bin" dest 0 +# Non-root: unchanged, the solo path. +check "install.sh: non-root → DEST=\$HOME/.local" 0 "DEST=/home/tester/.local/share/box" dest 1000 +check "install.sh: non-root → BINDIR=\$HOME/.local" 0 "BINDIR=/home/tester/.local/bin" dest 1000 +# BOX_HOME / BOX_BIN still win on BOTH branches — the scripting override. +check "install.sh: BOX_HOME overrides the root default" 0 "DEST=/srv/box" dest 0 BOX_HOME=/srv/box +check "install.sh: BOX_BIN overrides the root default" 0 "BINDIR=/srv/bin" dest 0 BOX_BIN=/srv/bin +check "install.sh: BOX_HOME overrides the non-root default" 0 "DEST=/srv/box" dest 1000 BOX_HOME=/srv/box +rm -f "$DBLOCK" +# The root-only world-readable chmod (#71): the tree is EXECUTED by other users, +# so root must open read+traverse. Grep it, and that it is root-guarded so the +# per-user install stays byte-identical to before. +# $DEST is a LITERAL in the grep pattern (install.sh's own variable) — single +# quotes intended. +# shellcheck disable=SC2016 +check "install.sh: root makes the tree world-readable (a+rX)" 0 "" \ + grep -qF 'chmod -R a+rX "$DEST"' "$ROOT/install.sh" +check "install.sh: the a+rX is root-guarded" 0 "" \ + bash -c 'grep -B2 "chmod -R a+rX" "'"$ROOT"'/install.sh" | grep -q "id -u.*-eq 0"' +# #66's flow, preserved: confirm-before-download, and no-op if already installed. +check "install.sh: still confirms before downloading (#66)" 0 "" \ + grep -qF 'confirm "Install box from' "$ROOT/install.sh" +check "install.sh: still no-ops on an existing install (#66)" 0 "" \ + grep -qF 'already installed' "$ROOT/install.sh" + +# --------------------------------------------------------------------------- +# Templates — #65 tmux. `box tmux` runs `tmux new-session` INSIDE the box, so a +# template that never installs tmux fails with "tmux: command not found". Every +# template must carry it in its cloud-init package list. +# --------------------------------------------------------------------------- +for t in blank claude codex grok; do + check "template '$t': installs tmux (#65)" 0 "" \ + grep -qE '^[[:space:]]*-[[:space:]]+tmux$' "$ROOT/templates/$t/user-data.yaml" +done + + +echo "---" +echo "$PASS passed, $FAIL failed" +rm -rf "$SHIMDIR" +[ "$FAIL" -eq 0 ]