🔴 Boxes are NOT isolated from each other — sibling isolation does not hold (confirmed live) #16
Labels
No labels
blocked
blocker:ci-red
blocker:conflict
blocker:drill-pending
blocker:unrequested
bug
claimed
documentation
enhancement
epic
merge-next
needs-triage
ready
release
scope:cli
scope:drill
scope:host
scope:installer
scope:templates
scope:tiers
stale
state:addressing
state:bots-reviewing
state:building
state:needs-human
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: heavy-duty/box#16
Loading…
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Split from #12 (part 2 of 3: #15 audit → isolation hardening → #17 box + templates). Blocked by #15 — its results pin this issue's design points.
Boxes are already isolated from each other, but by accident
The question that prompted #12: multiple boxes share
claudenet— can they reach each other?By code reading: they cannot (the audit verifies it live). But for a reason nobody wrote down, and nothing tests.
host/setup-host.sh:31drops all egress to10.0.0.0/8; the claudenet subnet is10.87.0.0/24— inside10.0.0.0/8. So box A's packets to box B die on A's own NIC, andsecurity.acls.default.ingress.action=drop(setup-host.sh:39) would drop them again at B. Belt-and-braces — and incidental.docs/claudebox-design.md:95-97describes those drops as being about RFC1918 and the host's LAN; sibling boxes appear nowhere in the design, the code, or a test. Renumber the subnet to192.168.50.0/24for some perfectly good reason and box-to-box isolation silently evaporates — while every stated design goal still reads as satisfied. That is the bug: not today's behavior, but that today's behavior is not load-bearing on purpose.The same shape of accident has a second instance: IPv6 is off (
ipv6.address=none,setup-host.sh:24) and every ACL rule is IPv4-only. Airtight today; re-enable IPv6 and not one rule covers it.Two gaps survive even with the drop working as intended:
setup-host.sh:30) keeps dnsmasq on10.87.0.1serving DNS — and that dnsmasq holds records for every instance on the network. Box A can resolve box B's name and address: not connect, but enumerate (names, existence, count).profiles/claude-dev.yaml:7-11sets nosecurity.mac_filtering/security.ipv4_filtering. A hostile box poisoning the gateway's ARP cannot MITM siblings (their replies die on its own ingress drop) but can blackhole them — a cross-box denial of service still breaks "isolated even between them".Why not a network per box
10.87.0.0/16(256 of them, real bookkeeping, a leak every time anrmgoes wrong).host/claudebox-firewall.shhardcodesNET=claudenet/GW=10.87.0.1; its UFW / nft /DOCKER-USERhole-punching would become per-network and re-run on every mint — today it is a one-shot boot unit.The right forward-looking axis is named networks a box opts into, defaulting to the isolated one — not built here, just not foreclosed (and when it comes, it comes as a CLI flag, never a template key).
Proposed
10.87.0.0/24written into it recreates the original bug one renumbering later. Per the audit's B1:@internalas destination if bridge networks support it; elsesetup-host.shderives the subnet at rule-creation time (masking the gateway CIDR thatipv4.addressreturns —10.87.0.1/24, host bits included). Rule ordered after the gateway carve-out (order is load-bearing; the carve-out itself stays literal — if it drifts, DNS breaks loudly, the good failure mode). One comment stating intent: boxes must not reach boxes, independent of the RFC1918 drops.security.mac_filtering=true+security.ipv4_filtering=trueon the profile NIC. Ends the L2 spoofing class. Caveat to document: pins each box to its own address; fine for every current workload including in-box Docker (NAT hides behind eth0), forecloses extra-MAC/IP-on-the-wire workloads. Existing boxes pick it up on profile re-apply but may need a restart — stated, not automated.dns.mode=noneonclaudenet. Closes the enumeration leak; instances keep upstream resolution (audit B3).ipv6.address=noneand an assertion in the test.tests/isolation.sh— the repo's first test; the audit's section-A probes, mechanized:Plain bash,
set -euo pipefail, PASS/FAIL per line, nonzero exit on any MUST-fail that passed,trapcleanup of its two boxes. Not CI-wired (no Incus in CI); a header documents how to run it by hand.docs/claudebox-design.md— the isolation section gains the missing sentence: siblings mutually unreachable by design, DNS does not enumerate, IPv6-off and L2 filtering are contract, the test is the proof.Acceptance
tests/isolation.shin the repo and green on a live host (output pasted in the PR). Thegetentlines are the ones that currently fail to fail.@internal, or derived at creation).bin/claudebox.One PR, a morning's work once the audit is in. The box+templates issue then lands against a repo whose isolation is explicit and tested — the rename cannot silently regress what a test asserts.
Two of this issue's four proposed changes are now settled by the live drill (#15, run 2) — design points, before any code:
dns.mode=noneis vetoed as written. On a real bridge it didn't just stop sibling-record resolution — it broke all DNS in the box (curl: Resolving timed outto a public host). So proposal 3 can't bedns.mode=nonealone; it needs a mechanism that drops instance records while keeping the upstream resolver. Options to evaluate:dns.mode=noneplus an explicit resolver handed to the box (static/etc/resolv.confor adns.nameserversequivalent), or dropping only the enumeration path rather than the whole DNS mode. This is exactly the kind of thing the rehearsal exists to catch.@internalis unsupported on bridge networks —Error: Unsupported nftables subject "@internal". So proposal 1 takes the fallback already written into it: derive the subnet insetup-host.shby masking the gateway CIDR thatincus network get claudenet ipv4.addressreturns (10.87.0.1/24→10.87.0.0/24), so the drop rule and the network can't disagree.Confirmed on the way: L2 filtering (
security.mac_filtering+security.ipv4_filtering) leaves box networking intact (proposal 2 is safe); IPv6 is off; and the DNS-enumeration leak this issue targets is real and reproduced (a box resolves its sibling by name). Still pending run 3: A3, the sibling-reachability baseline itself.⚠️ Retracting half of my previous comment — the
dns.mode=noneveto is not established.Run 3 contradicted run 2 on exactly that point:
B3 egress under dns.mode=none: BROKEN — design vetoB3 egress under dns.mode=none: intact — safe to shipSetting
dns.moderestarts the network's dnsmasq, and both runs probed egress ~2s later — run 2 caught the restart window, run 3 didn't. So the "veto" was very likely an artifact of when the probe fired, not a property of the knob. PR #21 makes the probe distinguish a transient outage (recovers within 30s) from a real break, so the next run gives a verdict this issue can actually design on.Unchanged and still solid:
@internalis genuinely unsupported on bridge ACLs (Unsupported nftables subject) ⇒ proposal 1 derives the subnet by masking the gateway CIDR. L2 filtering leaves box networking intact ⇒ proposal 2 is safe. The DNS-enumeration leak is real and reproduced on every run ⇒ this issue's premise holds.The lesson is one worth keeping in the issue itself: a design verdict from a single observation of a system with restart semantics is not a verdict. Whatever mechanism proposal 3 lands on, its test must tolerate the dnsmasq restart window rather than race it.
🔴 The premise of this issue is wrong — and the truth is worse
The #15 audit ran on a live host. Sibling isolation does not hold.
refusedmeans A's SYN arrived at B and B answered with a RST. Boxes reach each other, bidirectionally, on a stock host, today.So this issue's framing must change:
The code reading in #12 was plausible and wrong: yes, the ACL drops egress to
10.0.0.0/8, and yes10.87.0.0/24sits inside it — but two boxes on one bridge share an L2 segment, so their traffic is switched, not routed, and never meets the rule that would drop it. The "belt and braces" was neither.The tool's stated contract is currently false. "A box reaches the public internet and nothing else" — it also reaches every other box on the host. Two agents, in two boxes, can talk to each other and neither the docs, the code, nor a test says otherwise.
The four proposals, now settled by live evidence
@internalis rejected on bridge ACLs (Unsupported nftables subject "@internal"), so derive the subnet insetup-host.sh(mask the gateway CIDR:ipv4.addressreturns10.87.0.1/24, you want10.87.0.0/24). ⚠️ But an egress ACL rule may not see intra-bridge traffic either — that is precisely the bug above. If it does not, this needs a bridge-family/ebtables rule, orsecurity.aclsapplied at the NIC, or per-box networks after all. The acceptance test is thatrefusedbecomestimeout. Do not ship this on a code reading; that is what got us here.security.mac_filtering+security.ipv4_filteringipv4_filteringVETOED as written — it breaks the box's networking. Measured on a baseline that had already passed the egress check, so this verdict is sound (an identical FAIL in an earlier run was a broken box lying, and is void). Symptom: box loses networking;dockerdis up but cannot pull or run a container — consistent withipv4_filteringpermitting only the instance's own address as source, which in-box Docker's172.17.xNAT does not satisfy. Split it:mac_filteringalone is probably fine and still kills ARP spoofing;ipv4_filteringneeds a way to accommodate the box's own container NAT, or it is out.dns.mode=noneipv6.address: none), just needs stating — every ACL rule is IPv4-only, so IPv6-on would be an uncovered path.tests/isolation.shdrill/drill.sh.Two related findings from the same audit
100.64.0.0/10addresses but the gateway DNS carve-out forwards to MagicDNS. Same shape as the sibling leak: connection blocked, enumeration wide open. Whatever mechanism fixes proposal 3 should fix this too.Createdmay have nothing serving it (#35) — worth knowing before you debug an isolation rule that "does nothing".Priority
This is no longer hardening. It is a security fix against a false claim in the README, and it should be treated as such. #17 (the
boxrename) can wait behind it; this cannot.Closing on run 11's evidence. The hardening this issue asked for is shipped and measured, not reasoned:
security.port_isolation: "true"in the profile (#38) + the nft bridge-family drop (#37, kept as the second belt), with the design written down inclaudebox-firewall.sh's comment block — the L2 explanation this issue said existed nowhere.A3 sibling: BLOCKED — tcp dropped + no icmp reply (security.port_isolation), plus box→host, RFC1918, and host→box all dropped. The kernel's view (isolated onper tap) is read directly by the doctor since #39/#40.dns.mode=noneshipped; run 11 confirms a box cannot resolve its sibling's name, and public-name forwarding survives.setup-host.sh(the IPv4-only ACLs have no uncovered path) — run 11 asserts it.security.ipv4_filteringVETOED (breaks the box — measured on a passing baseline, run 10),@internalunsupported on bridge ACLs.The detour was the finding: the original 10.0.0.0/8 belt-and-braces reasoning was wrong (L2 frames never meet an L3 ACL — confirmed live in run 10 by a RST from a sibling), and the fix that closed it is the port-isolation flag this issue's audit trail produced. #17 is unblocked.
🤖 Generated with Claude Code