codex-bot-andresmgsl/box
1
0
Fork 0
forked from heavy-duty/box
Code Pull requests Activity

Merge pull request #36 from claude-hdb/drill/run10-results

Browse source 
docs(drill): the audit is complete — boxes are not isolated from each other
This commit is contained in:
Daniel Marin 2026-07-14 12:28:10 +01:00 committed by GitHub
commit 3b79f304c0
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
1 changed files with 26 additions and 23 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

49
drill/RUNS.md
View file

@ -4,28 +4,28 @@ What the drill has actually found, what has broken *in the drill itself*, and
how to diagnose the next stall without starting from zero. Append a section per how to diagnose the next stall without starting from zero. Append a section per
run; keep the traps table current — it is the part that saves time. run; keep the traps table current — it is the part that saves time.
The audit this feeds is [#15](https://github.com/heavy-duty/claudebox/issues/15). The audit this fed, [#15](https://github.com/heavy-duty/claudebox/issues/15), is
**complete** (run 10, 48/49).
## Where the audit stands ## The audit's answer
| Probe | Answer | Run | | Probe | Answer |
| --- | --- | --- | | --- | --- |
| A1/A5 egress + gateway DNS | **PASS** | 2, 3, 4 | | A1/A5 egress + gateway DNS | PASS |
| A2 box → host | **dropped** | 2, 3, 4 | | A2 box → host | dropped |
| A2 box → RFC1918 | **dropped** | 2, 3, 4 | | A2 box → RFC1918 | dropped |
| **A3 sibling isolation** | ⏳ **still unanswered** — blocked by a drill bug in every run so far | — | | **A3 sibling isolation** | 🔴 **FAIL — tcp REFUSED, i.e. the packet ARRIVED.** Boxes reach each other. #12's central claim was wrong; **#16 is a fix, not a formalization** |
| A4 DNS enumeration | **LEAKS** — a box resolves its sibling by name (as #12 predicted) | 2, 3, 4 | | A4 DNS enumeration | LEAKS — a box resolves its sibling by name and address |
| A6 IPv6 off | `none` ✓ | 2, 3, 4 | | A6 IPv6 off | `none` ✓ |
| A7 inbound host → box | **dropped** | 3, 4 | | A7 inbound host → box | dropped |
| B1 `@internal` on a bridge ACL | **REJECTED** — `Unsupported nftables subject` ⇒ #16 derives the subnet | 2, 3 | | B1 `@internal` on a bridge ACL | REJECTED — `Unsupported nftables subject` ⇒ #16 derives the subnet |
| B2 `incus copy` preserves `user.*` | **YES** ⇒ #17's metadata design holds | 2, 3, 4 | | B2 `incus copy` preserves `user.*` | YES ⇒ #17's metadata design holds |
| B3 `dns.mode=none` | closes the leak; **egress verdict FLIPPED between runs** (see below) | 2, 3 | | B3 `dns.mode=none` | VIABLE — closes the leak, egress survives, no outage window |
| B4 `config get` on unset key | **empty + exit 0** ⇒ #17 must use `${var:-}`, never `\|\|` | 2, 3, 4 | | B4 `config get` unset key | empty + exit 0 ⇒ #17 must use `${var:-}` |
| B5 L2 filtering | box networking **intact**; in-box docker unverified | 2, 3 | | B5 L2 filtering | 🔴 `ipv4_filtering` **BREAKS the box** — design veto (measured on a healthy baseline) |
**A3 is the whole point and it has never fired.** It is the one claim #12 could **The headline:** the tool's contract — *"a box reaches the public internet and
not verify from a code reading. Until it is answered, #16 does not know whether nothing else"* — is **false today**. It also reaches every other box on the host.
it is writing a *formalization* or a *fix*.
## Findings in claudebox (not in the drill) ## Findings in claudebox (not in the drill)
@ -158,14 +158,17 @@ No listener is needed, and none should be started: see trap 3.
| 4 | hung at C4 | trap 2 again, this time via `claudebox exec` in a command substitution | | 4 | hung at C4 | trap 2 again, this time via `claudebox exec` in a command substitution |
| 5 | stalled in host setup | trap 6 — silence through apt/sudo | | 5 | stalled in host setup | trap 6 — silence through apt/sudo |
| 6 | stalled in `setup-host.sh` | trap 8 — cleanup ran *after* setup. Recovering the host exposed **two real claudebox bugs**: `setup-host` deadlocks the incus daemon when re-run with boxes up (#26), and clones inherit their source's machine-id → same DHCP lease → **two boxes, one IP** (#27) | | 6 | stalled in `setup-host.sh` | trap 8 — cleanup ran *after* setup. Recovering the host exposed **two real claudebox bugs**: `setup-host` deadlocks the incus daemon when re-run with boxes up (#26), and clones inherit their source's machine-id → same DHCP lease → **two boxes, one IP** (#27) |
| **10** | **48/49 — the audit is complete** | 🔴 **A3 answered: sibling isolation DOES NOT HOLD** (tcp refused = the packet arrived). B5's `ipv4_filtering` veto confirmed on a *healthy* baseline. B3 cleared. Every #15 probe answered |
| 9 | aborted: cold mint failed, twice | **not** the drill and **not** the host's mutations (doctor was green): `claudenet` had **no dnsmasq** — it never respawned after the SIGKILL in run 6's recovery. Boxes got no DHCP lease at all. Trap 11 | | 9 | aborted: cold mint failed, twice | **not** the drill and **not** the host's mutations (doctor was green): `claudenet` had **no dnsmasq** — it never respawned after the SIGKILL in run 6's recovery. Boxes got no DHCP lease at all. Trap 11 |
| 8 | aborted: cold mint failed | `cloud-init status: error` — **the box had no DNS at all**. Run 7's phase-D `dns.mode=none` survived the run and poisoned the host. Trap 10, and the reason `doctor.sh` exists | | 8 | aborted: cold mint failed | `cloud-init status: error` — **the box had no DNS at all**. Run 7's phase-D `dns.mode=none` survived the run and poisoned the host. Trap 10, and the reason `doctor.sh` exists |
| 7 | 41/49 | the clone-identity fix could not reboot (systemd needs a valid machine-id to shut down cleanly), so it never took effect → the IP collision persisted → the box lost networking → **phase D reported a false design veto against #16**. Trap 9. Also found: `dir` storage makes every clone a full disk copy (#29) | | 7 | 41/49 | the clone-identity fix could not reboot (systemd needs a valid machine-id to shut down cleanly), so it never took effect → the IP collision persisted → the box lost networking → **phase D reported a false design veto against #16**. Trap 9. Also found: `dir` storage makes every clone a full disk copy (#29) |
**The instrument has been less reliable than the thing it measures.** Four of **The instrument was less reliable than the thing it measured.** Of ten runs,
five runs died on drill plumbing, not on claudebox. That is worth stating four died on drill plumbing and three on bugs in claudebox or the host. It still
plainly: if a probe can be answered by hand (see above), answer it by hand and paid for itself many times over — every one of those seven failures was a real
move on rather than paying for another full run. defect, and the audit's headline finding overturned the premise it was written
to confirm. But the lesson stands: **if a probe can be answered by hand, answer
it by hand** rather than paying for another full run.
### The B3 flip — a lesson worth keeping ### The B3 flip — a lesson worth keeping