test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
# The drill
|
|
|
|
|
|
chore: finish the debrand — env vars, install dir, docs are 'box', not 'claudebox'
The 0.4.0 rename left surface leftovers the user hit through the
CLAUDEBOX_* env vars. Sweep them, drawing a clean line:
box = everything the user touches — env vars (BOX_REPO/REF/HOME/
BIN), installer messages (box-install:), the install tree
(~/.local/share/box, with the installer sweeping the old
~/.local/share/claudebox on upgrade), tool prose, and the
docs (docs/box-{design,recipe}.md).
claudebox = the GitHub repo name (URLs, the claudebox-<ref> tarball
dir, issue refs), the legacy user.claudebox=1 tag, the
old-stack cleanup code (claudenet/claude-dev/claude-isolate/
claudebox-firewall), and the .claudebox/ runbook convention
— a deliberate v1 hold, since renaming it breaks consuming
repos.
Renamed the two doc files and their links; updated drill.sh/doctor.sh
paths and BOX_REPO/BOX_REF; fixed the claude template's in-box briefing
to say 'box'. RUNS.md left as-is (append-only history). No behavior
change beyond the install-dir move, which the installer migrates.
2026-07-14 17:44:24 +00:00
|
|
|
An end-to-end rehearsal of box against a **real** Incus: install the CLI,
|
test: fold the #15 audit into the drill — sibling probes and a hardening rehearsal
The drill covered the CLI surface and basic egress but never probed the
one claim everything hinges on: it only ever had one box running, so
sibling isolation — #12's central, never-reproduced finding — went
untested. Phase C now clones a peer and probes box-to-box reachability
(with a listener, so "refused" cannot masquerade as "dropped"), DNS
enumeration, IPv6-off, and host-to-box inbound. A new phase D applies
the exact changes #16 proposes (dns.mode=none, NIC filtering, @internal)
and watches what breaks — a FAIL there is a design veto caught before
the code is written. The run ends with an answers block to paste into
issue #15.
Also: defaults now install heavy-duty/claudebox@main (the old default
pointed at a fork branch that stopped moving when #13 merged), the
version check compares against the installed VERSION file instead of a
hardcoded 0.3.0, and the blind sleep after start became an agent poll.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:30:47 +00:00
|
|
|
set up the host, mint boxes, drive the whole surface, check that the isolation
|
|
|
|
|
actually holds — and run the full
|
|
|
|
|
[#15 audit](https://github.com/heavy-duty/claudebox/issues/15), including a live
|
|
|
|
|
rehearsal of the hardening
|
|
|
|
|
[#16](https://github.com/heavy-duty/claudebox/issues/16) proposes. It ends with
|
|
|
|
|
a block of audit answers to paste into #15.
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
|
|
|
|
|
> ⚠ **It rearranges the host it runs on.** Incus, a systemd unit, a network, an
|
test: fold the #15 audit into the drill — sibling probes and a hardening rehearsal
The drill covered the CLI surface and basic egress but never probed the
one claim everything hinges on: it only ever had one box running, so
sibling isolation — #12's central, never-reproduced finding — went
untested. Phase C now clones a peer and probes box-to-box reachability
(with a listener, so "refused" cannot masquerade as "dropped"), DNS
enumeration, IPv6-off, and host-to-box inbound. A new phase D applies
the exact changes #16 proposes (dns.mode=none, NIC filtering, @internal)
and watches what breaks — a FAIL there is a design veto caught before
the code is written. The run ends with an answers block to paste into
issue #15.
Also: defaults now install heavy-duty/claudebox@main (the old default
pointed at a fork branch that stopped moving when #13 merged), the
version check compares against the installed VERSION file instead of a
hardcoded 0.3.0, and the blind sleep after start became an agent poll.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:30:47 +00:00
|
|
|
> ACL, a profile, rewritten firewall rules — and, in the last phase, deliberate
|
|
|
|
|
> mutations to the network and profile. **Run it on a machine you can format** —
|
|
|
|
|
> a spare server, a cloud VM you'll destroy, a VM on your laptop.
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
> Not your workstation.
|
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
git clone https://github.com/heavy-duty/claudebox && cd claudebox
|
test: fold the #15 audit into the drill — sibling probes and a hardening rehearsal
The drill covered the CLI surface and basic egress but never probed the
one claim everything hinges on: it only ever had one box running, so
sibling isolation — #12's central, never-reproduced finding — went
untested. Phase C now clones a peer and probes box-to-box reachability
(with a listener, so "refused" cannot masquerade as "dropped"), DNS
enumeration, IPv6-off, and host-to-box inbound. A new phase D applies
the exact changes #16 proposes (dns.mode=none, NIC filtering, @internal)
and watches what breaks — a FAIL there is a design veto caught before
the code is written. The run ends with an answers block to paste into
issue #15.
Also: defaults now install heavy-duty/claudebox@main (the old default
pointed at a fork branch that stopped moving when #13 merged), the
version check compares against the installed VERSION file instead of a
hardcoded 0.3.0, and the blind sleep after start became an agent poll.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:30:47 +00:00
|
|
|
bash drill/drill.sh --yes # run and forget; omit --yes to be asked first
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
```
|
|
|
|
|
|
test: fold the #15 audit into the drill — sibling probes and a hardening rehearsal
The drill covered the CLI surface and basic egress but never probed the
one claim everything hinges on: it only ever had one box running, so
sibling isolation — #12's central, never-reproduced finding — went
untested. Phase C now clones a peer and probes box-to-box reachability
(with a listener, so "refused" cannot masquerade as "dropped"), DNS
enumeration, IPv6-off, and host-to-box inbound. A new phase D applies
the exact changes #16 proposes (dns.mode=none, NIC filtering, @internal)
and watches what breaks — a FAIL there is a design veto caught before
the code is written. The run ends with an answers block to paste into
issue #15.
Also: defaults now install heavy-duty/claudebox@main (the old default
pointed at a fork branch that stopped moving when #13 merged), the
version check compares against the installed VERSION file instead of a
hardcoded 0.3.0, and the blind sleep after start became an agent poll.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:30:47 +00:00
|
|
|
Useful flags: `--ref <branch>` (drill a branch rather than `main`),
|
|
|
|
|
`--keep-boxes` (leave the boxes up to poke at — note the last phase's network
|
|
|
|
|
and profile mutations stay applied with them).
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
|
test: fold the #15 audit into the drill — sibling probes and a hardening rehearsal
The drill covered the CLI surface and basic egress but never probed the
one claim everything hinges on: it only ever had one box running, so
sibling isolation — #12's central, never-reproduced finding — went
untested. Phase C now clones a peer and probes box-to-box reachability
(with a listener, so "refused" cannot masquerade as "dropped"), DNS
enumeration, IPv6-off, and host-to-box inbound. A new phase D applies
the exact changes #16 proposes (dns.mode=none, NIC filtering, @internal)
and watches what breaks — a FAIL there is a design veto caught before
the code is written. The run ends with an answers block to paste into
issue #15.
Also: defaults now install heavy-duty/claudebox@main (the old default
pointed at a fork branch that stopped moving when #13 merged), the
version check compares against the installed VERSION file instead of a
hardcoded 0.3.0, and the blind sleep after start became an agent poll.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:30:47 +00:00
|
|
|
Exit 0 means every check passed. Roughly 20 minutes, most of it the cold box.
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
|
fix(drill): stop poisoning the host, and add a doctor to prove it
Run 8's cold mint failed with 'cloud-init status: error' — the box could
not resolve deb.debian.org, or claude.ai, or anything. The cause was not
in that run at all: run 7's phase D set dns.mode=none on claudenet, the
run ended before reverting it, and every box minted afterwards came up
with no DNS.
This is the worst failure mode the drill has: a poisoned host does not
fail the next run honestly, it produces confident wrong answers. It is
how a false design veto against #16 got posted, and it wasted a cold
mint plus an hour of diagnosis that had nothing to do with the code
under test.
Three defences:
· the phase-D revert is armed with a trap BEFORE the first mutation,
so it fires on any exit, Ctrl-C included;
· the revert is VERIFIED rather than fired into /dev/null, so a failed
unset can no longer masquerade as a successful one;
· the drill refuses to start on a host still carrying the mutations.
And drill/doctor.sh answers the question that kept being answered by
hand: what state is this host actually in? Network, profile, ACL,
leftover boxes, and whether a box can still resolve DNS — with --fix to
revert the leftovers.
RUNS.md gains trap 10.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-14 00:32:03 +00:00
|
|
|
**Something wrong with the host?** `bash drill/doctor.sh` — it reports whether
|
|
|
|
|
the host is fit to drill (network, profile, ACL, leftover boxes, whether a box
|
|
|
|
|
can still resolve DNS), and `--fix` reverts what an aborted run left behind.
|
|
|
|
|
The drill mutates the host in phase D; an aborted run can leave a network that
|
|
|
|
|
mints boxes with **no DNS**.
|
|
|
|
|
|
docs(drill): keep the run log in the repo, not in PR descriptions
Five runs of hard-won knowledge — which probes are answered, which bugs
the drill found in claudebox, and the seven traps the drill itself fell
into — has been living in PR bodies and chat, where the next person
debugging this cannot find it.
RUNS.md carries: the audit scoreboard (A3 still unanswered, and why that
matters), the claudebox findings, a traps section to read BEFORE adding
a probe, stall diagnostics, how to run a single probe by hand instead of
paying for a full run, and the B3 flip-flop as a standing lesson about
verdicts drawn from one observation of a system with restart semantics.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:06:38 +00:00
|
|
|
**Iterating on the drill?** Read [RUNS.md](RUNS.md) first — it is the run log:
|
chore: finish the debrand — env vars, install dir, docs are 'box', not 'claudebox'
The 0.4.0 rename left surface leftovers the user hit through the
CLAUDEBOX_* env vars. Sweep them, drawing a clean line:
box = everything the user touches — env vars (BOX_REPO/REF/HOME/
BIN), installer messages (box-install:), the install tree
(~/.local/share/box, with the installer sweeping the old
~/.local/share/claudebox on upgrade), tool prose, and the
docs (docs/box-{design,recipe}.md).
claudebox = the GitHub repo name (URLs, the claudebox-<ref> tarball
dir, issue refs), the legacy user.claudebox=1 tag, the
old-stack cleanup code (claudenet/claude-dev/claude-isolate/
claudebox-firewall), and the .claudebox/ runbook convention
— a deliberate v1 hold, since renaming it breaks consuming
repos.
Renamed the two doc files and their links; updated drill.sh/doctor.sh
paths and BOX_REPO/BOX_REF; fixed the claude template's in-box briefing
to say 'box'. RUNS.md left as-is (append-only history). No behavior
change beyond the install-dir move, which the installer migrates.
2026-07-14 17:44:24 +00:00
|
|
|
what the audit has answered so far, the bugs the drill has found in box,
|
docs(drill): keep the run log in the repo, not in PR descriptions
Five runs of hard-won knowledge — which probes are answered, which bugs
the drill found in claudebox, and the seven traps the drill itself fell
into — has been living in PR bodies and chat, where the next person
debugging this cannot find it.
RUNS.md carries: the audit scoreboard (A3 still unanswered, and why that
matters), the claudebox findings, a traps section to read BEFORE adding
a probe, stall diagnostics, how to run a single probe by hand instead of
paying for a full run, and the B3 flip-flop as a standing lesson about
verdicts drawn from one observation of a system with restart semantics.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 23:06:38 +00:00
|
|
|
the traps this script has already fallen into (every one cost a run), how to
|
|
|
|
|
diagnose a stall, and how to run a single probe by hand instead of paying for a
|
|
|
|
|
whole run.
|
|
|
|
|
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
## Why it exists
|
|
|
|
|
|
|
|
|
|
The repo has no tests and no CI, and the CLI is a shell script that shells out to
|
|
|
|
|
`incus`. That means the interesting failures are not in the bash — they are in
|
|
|
|
|
what Incus actually does, which is exactly what unit tests would stub out and get
|
|
|
|
|
wrong. The drill runs the real thing.
|
|
|
|
|
|
|
|
|
|
## What it checks
|
|
|
|
|
|
chore: finish the debrand — env vars, install dir, docs are 'box', not 'claudebox'
The 0.4.0 rename left surface leftovers the user hit through the
CLAUDEBOX_* env vars. Sweep them, drawing a clean line:
box = everything the user touches — env vars (BOX_REPO/REF/HOME/
BIN), installer messages (box-install:), the install tree
(~/.local/share/box, with the installer sweeping the old
~/.local/share/claudebox on upgrade), tool prose, and the
docs (docs/box-{design,recipe}.md).
claudebox = the GitHub repo name (URLs, the claudebox-<ref> tarball
dir, issue refs), the legacy user.claudebox=1 tag, the
old-stack cleanup code (claudenet/claude-dev/claude-isolate/
claudebox-firewall), and the .claudebox/ runbook convention
— a deliberate v1 hold, since renaming it breaks consuming
repos.
Renamed the two doc files and their links; updated drill.sh/doctor.sh
paths and BOX_REPO/BOX_REF; fixed the claude template's in-box briefing
to say 'box'. RUNS.md left as-is (append-only history). No behavior
change beyond the install-dir move, which the installer migrates.
2026-07-14 17:44:24 +00:00
|
|
|
**A. Incus semantics.** The assumptions box is built on, probed directly:
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
that `incus config get <inst> user.claudebox` returns `1` (this is on the path of
|
|
|
|
|
*every* box command — if it lies, everything fails closed); that the
|
|
|
|
|
`user.claudebox=1` list filter selects our instances and excludes an untagged
|
|
|
|
|
one; that `--columns nstS` gives four clean CSV fields; that the state column
|
test: fold the #15 audit into the drill — sibling probes and a hardening rehearsal
The drill covered the CLI surface and basic egress but never probed the
one claim everything hinges on: it only ever had one box running, so
sibling isolation — #12's central, never-reproduced finding — went
untested. Phase C now clones a peer and probes box-to-box reachability
(with a listener, so "refused" cannot masquerade as "dropped"), DNS
enumeration, IPv6-off, and host-to-box inbound. A new phase D applies
the exact changes #16 proposes (dns.mode=none, NIC filtering, @internal)
and watches what breaks — a FAIL there is a design veto caught before
the code is written. The run ends with an answers block to paste into
issue #15.
Also: defaults now install heavy-duty/claudebox@main (the old default
pointed at a fork branch that stopped moving when #13 merged), the
version check compares against the installed VERSION file instead of a
hardcoded 0.3.0, and the blind sleep after start became an agent poll.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:30:47 +00:00
|
|
|
reads `RUNNING`; that `incus rename` really does refuse a running instance; that
|
|
|
|
|
snapshot-list's first CSV field is the label; that an **unset** config key reads
|
|
|
|
|
as empty with exit 0 (#15 B4); and that `incus copy` **preserves `user.*` keys**
|
|
|
|
|
(#15 B2 — the whole template-metadata design in #17 rests on it).
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
|
|
|
|
|
**B. The surface.** Mint, list, info, snapshot, clone-from-a-snapshot-of-a-
|
|
|
|
|
renamed-box, rename (running must refuse, stopped must work), the escape hatch
|
|
|
|
|
and its isolation warning, the `rm` confirmation guard, and the CLI contract
|
|
|
|
|
(typo'd command, typo'd flag, `list <box>`).
|
|
|
|
|
|
|
|
|
|
**The boundary** gets its own treatment: the drill launches an instance
|
chore: finish the debrand — env vars, install dir, docs are 'box', not 'claudebox'
The 0.4.0 rename left surface leftovers the user hit through the
CLAUDEBOX_* env vars. Sweep them, drawing a clean line:
box = everything the user touches — env vars (BOX_REPO/REF/HOME/
BIN), installer messages (box-install:), the install tree
(~/.local/share/box, with the installer sweeping the old
~/.local/share/claudebox on upgrade), tool prose, and the
docs (docs/box-{design,recipe}.md).
claudebox = the GitHub repo name (URLs, the claudebox-<ref> tarball
dir, issue refs), the legacy user.claudebox=1 tag, the
old-stack cleanup code (claudenet/claude-dev/claude-isolate/
claudebox-firewall), and the .claudebox/ runbook convention
— a deliberate v1 hold, since renaming it breaks consuming
repos.
Renamed the two doc files and their links; updated drill.sh/doctor.sh
paths and BOX_REPO/BOX_REF; fixed the claude template's in-box briefing
to say 'box'. RUNS.md left as-is (append-only history). No behavior
change beyond the install-dir move, which the installer migrates.
2026-07-14 17:44:24 +00:00
|
|
|
box did *not* mint, aims `down`, `rm` and the escape hatch at it, and
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
requires all three to refuse — and the instance to still be standing afterwards.
|
|
|
|
|
|
test: fold the #15 audit into the drill — sibling probes and a hardening rehearsal
The drill covered the CLI surface and basic egress but never probed the
one claim everything hinges on: it only ever had one box running, so
sibling isolation — #12's central, never-reproduced finding — went
untested. Phase C now clones a peer and probes box-to-box reachability
(with a listener, so "refused" cannot masquerade as "dropped"), DNS
enumeration, IPv6-off, and host-to-box inbound. A new phase D applies
the exact changes #16 proposes (dns.mode=none, NIC filtering, @internal)
and watches what breaks — a FAIL there is a design veto caught before
the code is written. The run ends with an answers block to paste into
issue #15.
Also: defaults now install heavy-duty/claudebox@main (the old default
pointed at a fork branch that stopped moving when #13 merged), the
version check compares against the installed VERSION file instead of a
hardcoded 0.3.0, and the blind sleep after start became an agent poll.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-13 21:30:47 +00:00
|
|
|
**C. Isolation baseline (#15 section A).** From inside a real box: public egress
|
|
|
|
|
works; the box cannot reach a listener on the host's claudenet gateway; RFC1918
|
|
|
|
|
is dropped; a **sibling box is unreachable** (a listener runs on the peer so
|
|
|
|
|
"refused" — the packet arrived — cannot masquerade as "dropped"); whether DNS
|
|
|
|
|
**enumerates** the sibling is recorded (#12 predicts it leaks today; that is
|
|
|
|
|
audit data, not a failure); IPv6 is off; and the host cannot connect **into**
|
|
|
|
|
a box.
|
|
|
|
|
|
|
|
|
|
**D. Hardening rehearsal (#15 section B).** The host is disposable, so the drill
|
|
|
|
|
applies the exact changes #16 proposes and watches what breaks: `dns.mode=none`
|
|
|
|
|
(must kill sibling resolution, must not kill egress), `security.mac_filtering` +
|
|
|
|
|
`security.ipv4_filtering` on the NIC (the box and its in-box Docker must keep
|
|
|
|
|
working), and `@internal` as an ACL drop destination on a bridge network (if
|
|
|
|
|
accepted and egress survives, #16's sibling drop is renumber-proof by
|
|
|
|
|
construction; if not, #16 derives the subnet instead). A FAIL in this phase is
|
|
|
|
|
a **design veto** for #16, caught before the code is written.
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
|
|
|
|
|
## What it does not check
|
|
|
|
|
|
docs: extend the agent-agnostic reframe to design, recipe, and drill docs
Same treatment as the README, applied to the prose that stood in "Claude" for
"the coding agent": box-recipe.md and box-design.md now describe the `.box/`
runbook and creds-free flow around whichever agent the box was minted with, and
name the per-template context file (`~/.claude/CLAUDE.md`, `~/.codex/AGENTS.md`,
`~/.grok/AGENTS.md`) instead of hardcoding Claude's. drill/README.md's
"does not check" note says the drill confirms each template's CLI, not just
Claude Code. `claude` stays as the concrete login example throughout.
Left untouched (out of scope, literal identifiers): the legacy isolation-stack
names (claudenet/claude-dev/claude-isolate, user.claudebox), the claude template
files themselves, and drill/RUNS.md history.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 00:21:22 +00:00
|
|
|
agent login (e.g. `claude /login`) — it's interactive by design, and the box is
|
|
|
|
|
creds-free by design. The drill confirms each coding-agent template's CLI is
|
|
|
|
|
installed and runnable; authenticating is yours.
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
|
chore: finish the debrand — env vars, install dir, docs are 'box', not 'claudebox'
The 0.4.0 rename left surface leftovers the user hit through the
CLAUDEBOX_* env vars. Sweep them, drawing a clean line:
box = everything the user touches — env vars (BOX_REPO/REF/HOME/
BIN), installer messages (box-install:), the install tree
(~/.local/share/box, with the installer sweeping the old
~/.local/share/claudebox on upgrade), tool prose, and the
docs (docs/box-{design,recipe}.md).
claudebox = the GitHub repo name (URLs, the claudebox-<ref> tarball
dir, issue refs), the legacy user.claudebox=1 tag, the
old-stack cleanup code (claudenet/claude-dev/claude-isolate/
claudebox-firewall), and the .claudebox/ runbook convention
— a deliberate v1 hold, since renaming it breaks consuming
repos.
Renamed the two doc files and their links; updated drill.sh/doctor.sh
paths and BOX_REPO/BOX_REF; fixed the claude template's in-box briefing
to say 'box'. RUNS.md left as-is (append-only history). No behavior
change beyond the install-dir move, which the installer migrates.
2026-07-14 17:44:24 +00:00
|
|
|
If the host has no `/dev/kvm`, box falls back to container mode. The drill
|
test: add an end-to-end drill against a real Incus
The repo has no tests and no CI, and everything in #9/#10/#13 was verified
against a stubbed incus. The interesting failures aren't in the bash — they're
in what Incus actually does, which is exactly what a stub gets wrong. So the
drill runs the real thing on a host you can format.
A. Incus semantics — the assumptions claudebox is built on, probed directly:
the user.claudebox=1 tag read (now on the path of every box command in
#13), the list filter, the CSV shape, the state string, whether rename
really refuses a running instance, the snapshot-list field order.
B. The claudebox surface — mint, list, info, snapshot, clone-from-snapshot-of-
a-renamed-box, rename, the escape hatch, the rm guard, the CLI contract —
and the boundary: an instance claudebox did NOT mint has down/rm/hatch
aimed at it, must refuse all three, and must still be standing after.
C. Isolation — from inside a real box: public egress works, box → host is
blocked, box → RFC1918 is dropped.
Destructive by design, and it says so before it starts: it asks for consent
(--yes to skip), and refuses outright with no TTY. Without /dev/kvm it reports
that the VM trust boundary was NOT validated rather than passing on a container.
2026-07-13 21:15:56 +00:00
|
|
|
still runs, but it says loudly that **the VM trust boundary was not validated**
|
|
|
|
|
rather than passing quietly on a weaker one.
|
2026-07-18 04:09:48 +00:00
|
|
|
|
|
|
|
|
## The multi-user rehearsal (`multiuser.sh`)
|
|
|
|
|
|
|
|
|
|
The restricted tier (#74) has its own rehearsal — the drill proves one
|
|
|
|
|
operator's host; this proves a *shared* one:
|
|
|
|
|
|
|
|
|
|
```sh
|
|
|
|
|
sudo BOX_MULTIUSER_REHEARSAL=1 bash drill/multiuser.sh --yes
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Root only, opt-in twice (it creates system users and edits the group
|
|
|
|
|
database). It creates two throwaway users, grants them the tier through the
|
|
|
|
|
real `box grant`, mints real boxes as them, and measures — from inside those
|
|
|
|
|
boxes — that each user is confined to their own project (a), the full
|
|
|
|
|
lifecycle works (b), no cross-user visibility (c), names don't collide (d),
|
|
|
|
|
`expose`/`setup-host`/`doctor` answer honestly at the tier (e/f), the boxes
|
|
|
|
|
ride `boxnet` under the full isolation contract including the cross-user
|
|
|
|
|
sibling drop (g), the private-bridge escape hatches are closed (h), the grant
|
|
|
|
|
survives an incus-user restart (k), and `revoke --purge` erases one user
|
|
|
|
|
without touching the other (l). Everything it makes, it deletes.
|
|
|
|
|
|
|
|
|
|
`--container` skips VM mints (CI runs it this way on every PR — the tier's
|
|
|
|
|
semantics are instance-type-independent); on real hardware run it bare so the
|
|
|
|
|
boxes are VMs. `--keep` leaves the users and boxes up for inspection.
|