box/host/migrate-host.sh
dan-claude-bot b429f5b65c feat(setup-host): refuse a claimed subnet, and BOX_SUBNET to move off one (#80)
Run inside a box, setup-host built a nested boxnet claiming the exact
subnet and gateway of the guest's own uplink: the guest then held
10.88.0.1 as a LOCAL address while its default route pointed at 10.88.0.1
as its GATEWAY, carried two connected routes for the subnet, and suffered
intermittent, self-recovering egress blackouts (~24-36 s, roughly hourly)
that no host-side check could attribute. The flagship use case funnels
agents toward exactly this — working on box, in a box — so the guard, not
the operator, has to catch it.

setup-host now scans the target subnet BEFORE any mutation: the default
route's gateway inside it (the smoking gun — that is this machine's own
uplink), or any non-boxnet interface holding an address in it, refuses
with the way out named (BOX_SUBNET). A prior boxnet owning the subnet is
the one claimant that is NOT a collision — the legitimate re-run — and an
existing bridge on a DIFFERENT subnet refuses too: setup-host converges a
bridge, it never re-addresses one under live leases.

BOX_SUBNET=<a.b.c.0/24> (validated at the gate, alongside BOX_DNS) moves
the whole stack from one place:
  · the bridge address derives (ipv4.address=<gw>/24)
  · the ACL's gateway carve-out derives — and the ruleset is now converged
    via 'network acl edit', not created once: the #80 escape hatch (moving
    the bridge) left the old /32 behind, stranding box DNS inside the
    10.0.0.0/8 drop, which presents as a dead resolver, never as a stale
    ACL
  · box-firewall reads the gateway off the live bridge ('|| true' because
    under pipefail an absent bridge would kill the script instead of
    answering "no bridge yet")
  · the drill, multiuser rehearsal and migrate-host probes derive the
    prefix from 'incus network get boxnet ipv4.address' instead of
    hardcoding 10.88 — a BOX_SUBNET host must not fail its own rehearsals

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-18 19:46:58 +00:00

159 lines
7.9 KiB
Bash

#!/usr/bin/env bash
# migrate-host.sh — move a host from the pre-0.4.0 'claudebox' stack to 'box'.
#
# The zero-ceremony transition is just install.sh + setup-host.sh: that leaves
# a DUAL-STACK host where legacy boxes (tag user.claudebox=1, claudenet/10.87,
# claude-dev) keep working while new mints land on boxnet. This script is
# the two things that path does not do:
#
# migrate-host.sh --box <name> re-home ONE legacy box onto the new stack
# migrate-host.sh --all-boxes re-home every legacy box
# migrate-host.sh --retire-legacy remove the legacy stack (refuses while any
# legacy box still exists)
#
# One action per invocation, idempotent, loud about what it did. Re-homing
# PRESERVES the box's authed state (Claude login, git creds — the expensive
# thing); it does not re-mint. The order is load-bearing: tag first (additive,
# reversible), profile last, and verify the box works on its new leg BEFORE
# calling it migrated — a box must never end up tagless or profileless.
#
# NOT 'set -e' around the per-box work: a box that fails one step is reported
# and skipped, not a crash that abandons the rest mid-migration.
#
# The report idiom is 'action && say "did X" || warn/die': say and warn always
# return 0, so the C-may-run-when-A-is-true trap SC2015 warns about cannot fire
# on those lines (same reasoning as drill.sh's ok/no).
# shellcheck disable=SC2015
set -u
say() { printf 'migrate: %s\n' "$*"; }
warn() { printf 'migrate: WARNING: %s\n' "$*" >&2; }
die() { printf 'migrate: ERROR: %s\n' "$*" >&2; exit 1; }
mode=""
target=""
while [ $# -gt 0 ]; do
case "$1" in
--box) mode=box; target="${2:-}"; shift 2 || die "--box needs a name" ;;
--all-boxes) mode=all; shift ;;
--retire-legacy) mode=retire; shift ;;
-h|--help) sed -n '2,20p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) die "unknown argument: $1 (see --help)" ;;
esac
done
[ -n "$mode" ] || die "pick one: --box <name> | --all-boxes | --retire-legacy"
command -v incus >/dev/null || die "incus is not installed on this host"
# The new stack must exist before any box can be re-homed onto it. setup-host
# creates it; refuse rather than move a box onto a network that isn't there.
require_new_stack() {
incus network show boxnet >/dev/null 2>&1 || die "boxnet does not exist — run host/setup-host.sh first"
incus profile show box-net >/dev/null 2>&1 || die "box-net profile does not exist — run host/setup-host.sh first"
}
# A box is LEGACY only if it still carries the old tag and has NOT been
# re-homed. Counting every user.claudebox=1 box was a bug: re-homing ADDS
# user.box=1 without removing the old tag, so --retire-legacy saw its own
# migrated boxes as un-migrated and refused forever.
legacy_boxes() {
local b
for b in $(incus list "user.claudebox=1" -f csv -c n 2>/dev/null); do
[ "$(incus config get "$b" user.box 2>/dev/null)" = 1 ] && continue # already re-homed
echo "$b"
done
}
# Re-home one box. Legacy boxes are all claude boxes (the only template the old
# tool minted), so the new metadata is the claude template's.
rehome_one() {
local b="$1" st
incus config get "$b" user.claudebox >/dev/null 2>&1 || { warn "$b is not a legacy box (no user.claudebox tag) — skipping"; return 1; }
if [ "$(incus config get "$b" user.box 2>/dev/null)" = 1 ]; then
say "$b already carries user.box=1 — already re-homed, skipping"; return 0
fi
say "re-homing $b"
# 1. TAG FIRST — additive and reversible. A box that stops here is still a
# valid legacy box (the old tag is untouched) AND now a new one.
incus config set "$b" user.box=1 user.box.template=claude user.box.user=claude \
|| { warn "$b: could not set new metadata — left untouched"; return 1; }
# 2. Stop, reassign the profile (this is the network move), restart. Incus
# won't reassign a profile on a running instance's NIC cleanly, and the
# box needs a fresh DHCP lease on boxnet anyway.
st="$(incus list "$b" -f csv -c s 2>/dev/null | head -1)"
case "$st" in RUNNING|Running|running) incus stop "$b" >/dev/null 2>&1 || warn "$b: stop was not clean" ;; esac
incus profile assign "$b" box-net \
|| { warn "$b: profile assign failed — it still has user.box=1 but is on the OLD network; fix by hand"; return 1; }
incus start "$b" >/dev/null 2>&1 || { warn "$b: did not restart — start it by hand"; return 1; }
# 3. VERIFY THE EFFECT, not the exit codes (the whole repo's lesson). The box
# must be on boxnet's subnet (read off the network — BOX_SUBNET moves it,
# #80) and actually resolve+reach the internet on its new leg before we
# call it migrated.
local _i ip pfx
pfx="$(incus network get boxnet ipv4.address 2>/dev/null | cut -d/ -f1)"; pfx="${pfx%.*}."
ip=""
for _i in $(seq 1 30); do
ip="$(incus exec "$b" -- ip -4 -o addr show scope global </dev/null 2>/dev/null \
| awk -v p="$pfx" '{for(i=1;i<NF;i++) if($i=="inet" && index($(i+1),p)==1){split($(i+1),a,"/"); print a[1]; exit}}')"
[ -n "$ip" ] && break
sleep 2
done
[ -n "$ip" ] || { warn "$b: never got a boxnet address after restart — re-home INCOMPLETE, inspect: incus console $b"; return 1; }
if incus exec "$b" -- getent hosts deb.debian.org </dev/null >/dev/null 2>&1; then
# LAST, and only once the move is VERIFIED: drop the legacy tag. Until this
# point the box wears both tags, so a failure anywhere above leaves it a
# valid box under one name or the other — never orphaned. Now it is simply
# a box, and --retire-legacy can see the old stack is empty.
incus config unset "$b" user.claudebox >/dev/null 2>&1 \
|| warn "$b: migrated, but the legacy tag could not be removed — retire-legacy will still see it"
say "$b re-homed: on boxnet ($ip), resolves + reachable, authed state preserved"
return 0
fi
warn "$b is on boxnet ($ip) but cannot resolve — check the new stack's resolver (box doctor)"
return 1
}
case "$mode" in
box)
[ -n "$target" ] || die "--box needs a name"
require_new_stack
rehome_one "$target"
;;
all)
require_new_stack
boxes="$(legacy_boxes)"
[ -n "$boxes" ] || { say "no legacy boxes to re-home"; exit 0; }
rc=0
for b in $boxes; do rehome_one "$b" || rc=1; done
[ "$rc" = 0 ] && say "all legacy boxes re-homed" || warn "some boxes need attention (above)"
exit "$rc"
;;
retire)
# Refuse while any legacy box still references the old stack — removing an
# in-use profile/network fails anyway, and a half-removed stack is worse
# than an intact one.
remaining="$(legacy_boxes)"
if [ -n "$remaining" ]; then
die "legacy boxes still exist: $(echo "$remaining" | tr '\n' ' ')
re-home them first (--all-boxes), or delete them, then retire."
fi
say "no legacy boxes remain — removing the legacy stack"
incus profile delete claude-dev >/dev/null 2>&1 && say "deleted profile claude-dev"
incus network delete claudenet >/dev/null 2>&1 && say "deleted network claudenet"
incus network acl delete claude-isolate >/dev/null 2>&1 && say "deleted ACL claude-isolate"
sudo systemctl disable --now claudebox-firewall.service >/dev/null 2>&1 && say "disabled claudebox-firewall.service"
sudo rm -f /etc/systemd/system/claudebox-firewall.service /usr/local/sbin/claudebox-firewall
sudo systemctl daemon-reload
sudo nft delete table inet claudebox >/dev/null 2>&1 && say "deleted nft table inet claudebox"
sudo nft delete table bridge claudebox >/dev/null 2>&1 && say "deleted nft table bridge claudebox"
# Assert the absence — don't trust the removals' exit codes.
left=""
incus network show claudenet >/dev/null 2>&1 && left="$left claudenet"
incus profile show claude-dev >/dev/null 2>&1 && left="$left claude-dev"
sudo nft list table bridge claudebox >/dev/null 2>&1 && left="$left nft-bridge"
[ -z "$left" ] && say "legacy stack retired — this host is now single-stack (box only)" \
|| die "legacy stack NOT fully removed:$left"
;;
esac