setup-host: refuse a claimed subnet (#80); BOX_SUBNET end-to-end; doctor learns the gateway-squat signature #86
No reviewers
Labels
No labels
blocked
blocker:ci-red
blocker:conflict
blocker:drill-pending
blocker:unrequested
bug
claimed
documentation
enhancement
epic
merge-next
needs-triage
ready
release
scope:cli
scope:drill
scope:host
scope:installer
scope:templates
scope:tiers
stale
state:addressing
state:bots-reviewing
state:building
state:needs-human
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: heavy-duty/box#86
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/nested-host-guard"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Fixes #80.
Running
box setup-host(or the drill, whose install path runs setup-host) inside a box built a nestedboxnetclaiming the exact subnet and gateway of the guest's own uplink: the guest then held10.88.0.1as a local address while its default route pointed at10.88.0.1as its gateway, carried two conflicting10.88.0.0/24connected routes, and suffered the issue's intermittent, self-recovering egress blackouts. This lands the issue's suggested fixes 1–3; suggested fix 4 (the agent-context guard note for theclaude/codex/groktemplates) lands in heavy-duty/rig#31's bootstrap roles per the thin-templates split (#81).1.
setup-hostrefuses a claimed subnet — before any mutationRight after the sudo resolution (nothing mutating precedes it), setup-host scans the target subnet: the default route's gateway inside it (the smoking gun — that is this machine's own uplink, i.e. the inside of a box), or any non-
boxnetinterface holding an address in it, refuses with exit 1, names issue #80 and the way out (BOX_SUBNET). A priorboxnetowning the subnet is the one claimant that is not a collision — the legitimate converge re-run — and an existing bridge on a different subnet refuses too: setup-host converges a bridge, it never re-addresses one under live leases. The drill is covered by the same guard (its install path runs setup-host).2.
BOX_SUBNET— the subnet is an input, end to endBOX_SUBNET=<a.b.c.0/24>(validated at the gate; garbage and wrong shapes die by name), alongside the existingBOX_DNS. Every consumer derives from it:ipv4.address=<gw>/24<gw>/32— and the ruleset is now converged vianetwork acl edit, not created once: the #80 escape hatch (moving the bridge) left the old/32behind, stranding box DNS inside the10.0.0.0/8dropbox-firewallreads the gateway off the live bridge instead of hardcoding itmigrate-hostderive their subnet prefix fromincus network get boxnet ipv4.address(no more hardcoded10.88)3.
box doctorknows the signaturegw_squat_signature(pureip route/ip addrtext → findings, unit-drivable): a default gateway held as a LOCAL address, and duplicate connected routes for the uplink subnet.probe_sig), where the signature actually lives.boxnet's actual gateway (the stale-/32failure mode above).Verification
Suite:
bash test/cli.sh→ 265 passed, 0 failed (211 on main + 54 new);shellcheck -x bin/* **/*.shclean. New coverage: both guard functions extracted and driven (shimip, canned tables — including the issue's captures verbatim); the whole setup-host driven end to end under logging shims, asserting the refusals made zero incus/sudo calls, the converge path still reaches "Host ready", and a freshBOX_SUBNET=10.89.0.0/24build landsipv4.address=10.89.0.1/24+destination: 10.89.0.1/32; the doctor signature driven against the poisoned, workaround-remapped, healthy-host, no-default-route, gateway-only and duplicate-only fixtures; line-order guards on refuse-before-mutation and signature-before-daemon.Live, on a box that IS the issue's environment (uplink
10.88.0.202/24via gw10.88.0.1, nested stack present with the workaround applied —boxnetremapped to10.89.0.1/24):The patched setup-host refuses before touching anything:
The patched doctor (read-only, admin path) on the same machine:
Both verdicts are the truth of that machine: the duplicate-route/captured-gateway signature is absent (the workaround moved the bridge — the pre-workaround state is covered by the synthetic fixtures, which reproduce the issue's
ip routecapture verbatim), and the stale carve-out the workaround left behind is real and now has a name and a fix.Not verified locally:
incus network acl editagainst a live daemon (the only writable daemon in reach is the live test asset this PR must not mutate) — CI's rehearsal job runs the real setup-host on a fresh daemon and the doctor baseline asserts the carve-out matches, so a bad edit turns that job red.🤖 Generated with Claude Code
Verdict: Approve
Fixes #80 as claimed: refuse-before-mutation,
BOX_SUBNETplumb-through, doctor gateway-squat signature + ACL carve-out check, and tests that pin the load-bearing seams. CIcheck+rehearsalgreen. No prior comments/reviews.What lands cleanly
1. Claimed-subnet refusal (setup-host)
valid_subnet+subnet_claimantsit immediately after sudo resolution, before apt/usermod/incus. Smoking gun (default GW inside target) and foreign-iface claimants both refuse with #80 +BOX_SUBNETnamed; priorboxneton the same subnet is allowed (converge); existing bridge on a different subnet refuses re-address under live leases. Pure overipoutput so the suite can drive it.2.
BOX_SUBNETend-to-endBridge
ipv4.address, ACL allow/32, and derived GW all follow one validateda.b.c.0/24. ACL is now full-ruleset converge vianetwork acl edit(not create-once) — correct fix for the escape-hatch stale-/32stranding DNS inside10.0.0.0/8. Drill / multiuser / migrate-host drop hardcoded10.88and read prefix fromincus network get boxnet ipv4.address.box-firewallreads GW off the live bridge (nft path rebuilds correctly).3. Doctor
gw_squat_signaturePure text→findings; judged on this machine first, both tiers, before any daemon call (nested impostor daemon);
probe_sigon both tiers; egress-broken/DNS-fine names itself as #80 fingerprint; admin ACL section compares carve-out to live bridge GW. Fixtures include the issue capture, remapped workaround, healthy host, and each signature line alone.4. Tests
Refuse paths assert no
incus/sudolog files (exit 1 ontest -e); converge still reaches "Host ready"; freshBOX_SUBNET=10.89.0.0/24logs bridge + ACL destinations; line-order guards on guard-before-mutation and signature-before-daemon. Shellcheck-clean path covered by CI.Suggested fix 4 (template agent-context) correctly deferred to heavy-duty/rig#31 per #81.
Non-blocking nits
UFW does not converge when the bridge GW moves.
box-firewall.shstill skips the whole UFW block onceDENYonboxnetexists. nft flush/rebuild tracks a remapped GW; UFW leaves the oldallow … to <old-gw> port 53. That is the same class of failure the PR carefully fixed for the ACL. First install withBOX_SUBNETis fine; remapped-then-re-setup UFW hosts are not fully end-to-end. Follow-up: converge UFW (or document "UFW hosts: re-add DNS allows after a bridge move").Doctor fix text hardcodes
10.89.0.1/24. Fine as the issue's known-good workaround; wrong if10.89is already claimed. Optional: prefer "pick a free/24viaBOX_SUBNET=…" without a fixed target, or note collision risk.Boot fallback
GW=10.88.0.1whenboxnetis absent is only a paper-over for default-subnet hosts; harmless for first install (After=incus.service+ setup-host order), slightly misleading if a custom-subnet host ever hits the no-bridge window on a first boot before UFW rules exist. Low practical risk.Not blockers / out of scope as stated
acl editnot hand-probed on a poisoned host — honest in the PR body; rehearsal runs real setup-host + doctor on a fresh daemon.drill/drill.shstill not in CI (pre-existing); multiuser rehearsal covers the derived-prefix probes on the default subnet.index, not real CIDR — acceptable given/24+ trailing-dot validation, and covered by the10.8vs10.88test.No request-changes items. Approve.
🔧 Reviewed — I agree with most; feedback below.
Read the diff end to end and reproduced the guard/signature logic by hand against the fixtures. The core is correct and the test coverage is genuinely load-bearing, not decorative:
subnet_claimantprefix-matches on${1%0/24}with the trailing dot terminating (10.8.0.≠10.88.x), the default-gateway-inside-target case fires whiledev == boxnetand boxnet's own address are correctly exempted, and the command-substitution exit status (if hit="$(subnet_claimant …)") makes "clean → don't refuse" fall out right. TheA_HOSTSTACKconverge case is a true negative, not luck.gw_squat_signature: both lines keyed offip -o addr$4and kernel-proto$2=="dev"routes match the real formats;upis pinned to the default route'sdev, so a healthy host with boxnet on its own subnet stays clean. Each line proven to fire alone.network acl editconverge (not create-once) is the right call for the stale-/32stranding, and the doctor carve-out check reads the allow rule position-independently.Two concerns, both non-blocking — but they're the same class of bug this PR fixes for the ACL, left unfixed in the UFW/boot path:
UFW doesn't converge on a bridge move, and doctor can't see that it didn't.
host/box-firewall.sh:16still guards the whole UFW block behind "does aDENY on boxnetrule already exist" — so a host that remaps boxnet off a colliding subnet keeps its oldallow … to <old-gw> port 53and never gains one for the new gateway. That's exactly the stale-carve-out failure the ACL path was rewritten to prevent. Worse, the new doctor check (drill/doctor.sh:290) only compares the incus ACL carve-out to the live gateway — which setup-host now converges — so a remapped UFW host gets a cleanOK the gateway carve-out matchesbill while box→gateway DNS is actually being dropped by the stale UFW rule. The safety net (rehearsal + doctor) has a blind spot precisely for UFW hosts that use the #80 escape hatch. Either converge the UFW allows off the live bridge too, or have the doctor check also inspect the UFW DNS-allow gateway.box-firewall.sh:19boot fallbackGW=10.88.0.1is wrong on aBOX_SUBNEThost. The comment argues the fallback only papers over the no-bridge-yet window, but on a10.89.xhost that hits that window at boot (bridge not yet addressed when the service runs, e.g. nft DOCKER-USER rebuild), the carve-out is built for the wrong gateway. Harmless on the default subnet; a latent DNS-drop on a custom-subnet host. Worth either ordering the service after boxnet has an address, or failing closed rather than defaulting to10.88.0.1.Everything else — the refuse-before-mutation ordering (asserted on the shims' absence of calls, nicely), the plumb-through to every derived value, deferring template fix 4 to rig#31 — I agree with. CI
check+rehearsalgreen.— automated review by
claude-bot-andresmgsl· heavy-duty-review-botVerdict: I agree with everything and have no additional feedback.
Post-merge follow-up: @claude-bot-andresmgsl's two non-blocking items (UFW converge on a bridge move + the doctor's UFW blind spot, and the
GW=10.88.0.1boot fallback) are addressed in #89 — UFW allows now converge off the live bridge every run, doctor judges UFW's own DNS carve-out against the live gateway, and the boot window fails closed instead of guessing a gateway.