The tool underneath was already generic: a thin, honest wrapper over
Incus. What was Claude-specific was welded on — one image, one profile,
one cloud-init file, one hardcoded 'sudo -u claude'. The weld is now a
template.
The mechanic: 'box new' stamps the template's identity onto the
instance (user.box=1, user.box.template, user.box.user); shell/exec/
tmux read the user back off the instance, and 'incus copy' carries
user.* keys (audit B2), so a clone knows what it is without consulting
the template. Templates are box.env (parsed against a strict allowlist,
never sourced — no key for a network exists, on purpose) plus a
verbatim cloud-init. Every template launches with the shared box-net
profile: the isolated NIC and root disk, nothing template-controlled —
resources land per-instance from box.env, overridable via BOX_CPU/
BOX_MEMORY/BOX_DISK (which is also how the drill shrinks boxes on a
small host now that profile edits can't).
The three open calls, taken as recommended: clean cut at 0.4.0 (no
claudebox shim; the installer retires the old symlink); default
template = claude (muscle memory survives); repo stays heavy-duty/
claudebox, binary is box.
Compat is the tag, not the name: resolve_box and list honor the legacy
user.claudebox=1 forever, and the legacy tag maps to the claude user —
a pre-rename box lists, shells, clones, unchanged.
Deliberate divergence from #17's table: the host-stack resource names
(claudenet, claude-isolate, nft tables, claudebox-firewall.*) are NOT
renamed — they are host-internal, invisible to users, and renaming
them breaks every provisioned host for zero user-visible gain.
claude-dev is no longer created; setup-host creates box-net, teardown
removes both.
Closes#17
Every fault the drill's doctor diagnoses is a user's fault first: a
wedged Incus daemon, a dnsmasq that silently isn't serving, a VPN
resolver boxes inherit, isolation claimed by config but off in the
kernel — each has killed a cold mint or weakened a boundary, with a
cloud-init error that names none of them. The CLI half-admitted it:
cmd_new's failure path hand-pointed at issue #33, doing one special
case of a doctor's job inline.
The verb delegates to the installed drill/doctor.sh (the tree ships
whole) — one hardened script, two audiences. Its flags pass through;
the verdict now reads 'fit to mint boxes (and to drill)'; the mint-
failure hint ends with 'claudebox doctor' instead of the hand-rolled
diagnosis.
Closes#46
The bridge's dnsmasq forwarded to whatever sat in the host's
/etc/resolv.conf at that moment. On a Tailscale host that is MagicDNS:
box DNS flapped with the tailnet (killing cold mints), and tailnet peer
names and split-DNS zones resolved from inside a box — name-level
reconnaissance of a private network, the same class as the sibling
enumeration dns.mode=none already closes.
setup-host.sh now sets raw.dnsmasq to no-resolv + pinned public
upstreams (BOX_DNS overrides the default 1.1.1.1 8.8.8.8), answering
the issue's three open questions from live measurement: raw.dnsmasq is
the lever (no first-class upstream key on the bridge; verified by
doctor --pin-dns followed by a box resolving), upstreams are a setting
with a sane default, and the pin is unconditional.
The doctor's unpinned-state messages now point at setup-host.sh as the
durable fix, keeping --pin-dns as the quick test.
Closes#33
claudebox-firewall.sh drops everything from a box to the host except
DNS (53) and DHCP (67); ICMP to 10.87.0.1 dies in that trailing drop on
every healthy host. The doctor used exactly that ping as its routing
probe, so it reported 'cannot even reach the gateway' — and a NOT-fit-
to-drill verdict — on a host whose very next line proved DNS working
through that same gateway.
Probe routing the way the contract states it: a box reaches the public
internet. curl to 1.1.1.1 by address, reusing the one probe for the
DNS-failure diagnosis instead of running it twice.
With a TTY on stdin, 'incus exec' goes interactive and puts the terminal
in raw mode. The DNS probes then hang forever: timeout's bare TERM never
lands (no -k escalation), and ^C is forwarded into the box as a keystroke
instead of killing the script. Observed live: doctor hung 15+ minutes at
'Can a box actually resolve DNS?' and survived Ctrl-C; the operator had
to kill the shell.
The drill already learned this exact lesson in #22 (exec_in pins stdin
and uses 'timeout -k'); the doctor's probe section was added later in
#34/#35 and never got the cure. All four probes now pin stdin to
/dev/null and escalate to SIGKILL.
Run the doctor on a healthy host and it reported three problems, all of
them its own:
· dns.mode=none was flagged as leftover rehearsal dirt. It is now the
SHIPPED setting — it is what stops a box enumerating its siblings.
Its absence is the fault; its presence was being "fixed" away.
· the nft check used 'sudo -n', which fails without cached credentials,
so it reported the box-to-box rule MISSING on a host where 'sudo nft
list' plainly shows it.
· the kernel's bridge view — the one fact that settles the isolation
question — was skipped with "'bridge' not installed". It is installed;
it lives in /usr/sbin, which is not on a normal user's PATH.
· and the DNS probe looked only for the drill's own box names, so it
said "no box to probe with" while two boxes sat there RUNNING.
A diagnostic that cries wolf is worse than no diagnostic: it costs the
same trust as a real failure and teaches you to ignore it. All four now
report what is actually true.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two bugs, one in each direction.
setup-host's new assertion ran 'nft list table bridge claudebox' without
sudo. nft needs root, so it failed with permission denied and printed
"the box-to-box drop is NOT active" about a rule that was demonstrably
there. A check that cries wolf is worse than no check.
And the deeper one: every check so far has asked the CONFIG whether
boxes are isolated. The config is a claim. Incus can accept
security.port_isolation and the kernel can still leave 'isolated off' on
the tap — and then boxes reach each other while every config in sight
says they cannot. That is precisely the shape of the original bug: the
ACL looked airtight and never saw the traffic.
So the doctor now reads the kernel's own view — 'bridge -d link show'
on claudenet's ports — and reports the isolated flag as the fact it is.
If the profile says true and the kernel says off, we learn that in a
second instead of after another ten-minute drill.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The nft bridge-family rule from the previous commit is LIVE on the host
and boxes still reach each other:
table bridge claudebox {
chain forward { ... meta ibrname "claudenet" meta obrname "claudenet" drop }
}
FAIL BOX A REACHES BOX B — sibling isolation does NOT hold [tcp: refused]
So the rule is not wrong about intent, it is wrong about mechanism —
whatever path these frames take, that hook does not stop them. Rather
than reason harder about netfilter (reasoning is what put the hole there
in the first place), use the mechanism Incus provides for exactly this:
security.port_isolation on the bridged NIC, which sets the kernel bridge
port's isolated flag so two isolated ports cannot exchange frames at all.
The nft rule stays as a second layer — it costs nothing — but the
profile flag is what carries the guarantee. doctor.sh checks it, because
the absence of this one is invisible: everything works and boxes can
simply reach each other.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The box-to-box drop shipped, the drill still found boxes reaching each
other, and the rule was simply not on the host. setup-host.sh ended with
'systemctl enable --now claudebox-firewall.service' — but the unit is
RemainAfterExit, so once it has run it stays "active" forever, and
'--now' does nothing to an active unit. Re-running setup-host after
upgrading claudebox therefore installed the new script to
/usr/local/sbin and never executed it. The host silently kept its old
firewall, and the box-to-box hole stayed open through the release that
claimed to close it.
This is worse than the original bug: every future firewall change would
have landed only on hosts that had never run setup-host before.
Restart the unit instead — the script is idempotent by design. Then
ASSERT the rule is live rather than assume it, because the absence of
this particular rule is invisible: everything keeps working and boxes
can simply reach each other. doctor.sh checks it too.
Also: dns.mode=none is now part of the shipped stack, so the drill must
stop treating it as leftover rehearsal dirt and reverting it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two cold mints died in cloud-init with "Temporary failure resolving
deb.debian.org", on a host the doctor had just certified clean. The
cause was not DNS forwarding, not leftover mutations, and not the drill:
claudenet had NO dnsmasq. It never respawned after the SIGKILL that
recovered the wedged daemon in run 6, so boxes got no DHCP lease at all
— no address, no gateway, no DNS.
Incus does not surface this. The bridge is up, the config is pristine,
'incus network show' says status: Created. Only the process table knows.
So the doctor now asks the process table, and --fix restarts incus to
respawn it.
An hour of hunting and two dead mints went into learning this. It is a
five-second check.
RUNS.md gains trap 11: a network incus calls "Created" may have nothing
serving it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Two cold mints in a row died with cloud-init 'status: error' on a host
the doctor had just certified clean — so the earlier "leftover mutations
poisoned the network" theory is dead, and the DNS failure is
reproducible rather than transient.
'claudebox new' printed four hundred dots and the word "error", leaving
the user with nothing to act on: the reason was in the box's own log and
nobody was told the log existed. It now prints cloud-init's status, the
fetch/resolve errors from the box's log, and how to inspect the box —
which is left running, because a box that failed to build is evidence,
not garbage. It also names the usual culprit: the host's resolver.
doctor.sh gains the diagnosis that keeps being done by hand:
· what the HOST resolves through, and whether that is a CGNAT/Tailscale
resolver the boxes inherit (issue #33);
· whether claudenet's resolver is pinned;
· and inside a box, the question that settles it — DNS is broken, but
can it still reach 1.1.1.1 BY ADDRESS? If yes, egress is fine and the
fault is purely the inherited forwarder.
· --pin-dns applies the #33 fix (raw.dnsmasq: no-resolv + public
servers) so the hypothesis can be TESTED rather than argued.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Run 8's cold mint failed with 'cloud-init status: error' — the box could
not resolve deb.debian.org, or claude.ai, or anything. The cause was not
in that run at all: run 7's phase D set dns.mode=none on claudenet, the
run ended before reverting it, and every box minted afterwards came up
with no DNS.
This is the worst failure mode the drill has: a poisoned host does not
fail the next run honestly, it produces confident wrong answers. It is
how a false design veto against #16 got posted, and it wasted a cold
mint plus an hour of diagnosis that had nothing to do with the code
under test.
Three defences:
· the phase-D revert is armed with a trap BEFORE the first mutation,
so it fires on any exit, Ctrl-C included;
· the revert is VERIFIED rather than fired into /dev/null, so a failed
unset can no longer masquerade as a successful one;
· the drill refuses to start on a host still carrying the mutations.
And drill/doctor.sh answers the question that kept being answered by
hand: what state is this host actually in? Network, profile, ACL,
leftover boxes, and whether a box can still resolve DNS — with --fix to
revert the leftovers.
RUNS.md gains trap 10.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
