Merge pull request #31 from claude-hdb/fix/drill-no-verdicts-on-broken-baseline

fix(drill): refuse to judge #16 on a broken baseline

drill/RUNS.md

@@ -83,6 +83,15 @@ Read this before adding a probe. Every one of these cost a run.
    filtering) in place, so setup converges against a moving target. Delete the
    boxes and revert the mutations **first**.
 
+9. **Never render a verdict on a broken baseline.** Run 7's box had no network
+   (a clone/source IP collision), and phase D dutifully reported *"L2 filtering
+   BREAKS the box — design veto"*. It did not; the box was already broken. A
+   measurement taken on a broken instrument is not evidence, and #16 would have
+   been redesigned around a fiction. Phase D is now gated on baseline egress
+   passing, and refuses to judge otherwise. This is the same failure as the B3
+   flip, in a different costume: **check that the thing you are measuring with
+   still works before you trust what it tells you.**
+
 ## Diagnosing a stall
 
 The drill narrates every long step. If it goes quiet, open a second terminal:
@@ -125,7 +134,8 @@ No listener is needed, and none should be started: see trap 3.
 | 3 | 48/49 | trap 4 — `eth0_ip` never matched, so A3 again unprobed. B3 now read *intact*, contradicting run 2 |
 | 4 | hung at C4 | trap 2 again, this time via `claudebox exec` in a command substitution |
 | 5 | stalled in host setup | trap 6 — silence through apt/sudo |
-| 6 | stalled in `setup-host.sh` | trap 8 — cleanup ran *after* setup, so setup reconfigured claudenet's ACLs while run 4's boxes were still attached to it |
+| 6 | stalled in `setup-host.sh` | trap 8 — cleanup ran *after* setup. Recovering the host exposed **two real claudebox bugs**: `setup-host` deadlocks the incus daemon when re-run with boxes up (#26), and clones inherit their source's machine-id → same DHCP lease → **two boxes, one IP** (#27) |
+| 7 | 41/49 | the clone-identity fix could not reboot (systemd needs a valid machine-id to shut down cleanly), so it never took effect → the IP collision persisted → the box lost networking → **phase D reported a false design veto against #16**. Trap 9. Also found: `dir` storage makes every clone a full disk copy (#29) |
 
 **The instrument has been less reliable than the thing it measures.** Four of
 five runs died on drill plumbing, not on claudebox. That is worth stating

drill/drill.sh

@@ -428,9 +428,15 @@ else
 fi
 
 # C1 — public egress (#15 A1; resolving the hostname also proves A5, gateway DNS)
-[ "$(box_curl archive https://api.github.com 20)" = 0 ] \
+BASELINE_OK=1
-  && { ok "box reaches the public internet (and gateway DNS resolves public names)"; aud "A1/A5 egress + public DNS: PASS"; } \
+if [ "$(box_curl archive https://api.github.com 20)" = 0 ]; then
-  || { no "box cannot reach the internet (a box that can't is useless)"; aud "A1/A5 egress: FAIL"; }
+  ok "box reaches the public internet (and gateway DNS resolves public names)"
+  aud "A1/A5 egress + public DNS: PASS"
+else
+  BASELINE_OK=0
+  no "box cannot reach the internet (a box that can't is useless)"
+  aud "A1/A5 egress: FAIL"
+fi
 
 # C2 — box → host (#15 A2). The host DOES listen on the gateway: dnsmasq is on
 # :53 by design (that carve-out is what makes egress DNS work). So probe a port
@@ -541,13 +547,25 @@ phase "D. Hardening rehearsal — #16's changes, applied live (#15 section B)"
 # ===========================================================================
 # The host is disposable, so rehearse the exact changes #16 proposes and watch
 # what breaks. A FAIL here vetoes a piece of #16's design before it is written.
+#
+# But ONLY if the box was healthy to begin with. On run 7 the baseline was
+# broken (a clone/source IP collision took the box's networking down), and phase
+# D dutifully reported "L2 filtering BREAKS the box — design veto". It did not.
+# A verdict measured on a broken box is not a verdict, it is a slander; #16
+# would have been redesigned around a fiction. Refuse to judge instead.
+if [ "$BASELINE_OK" -ne 1 ]; then
+  note "SKIPPING phase D — the box could not reach the internet BEFORE any hardening was applied"
+  inf  "a design verdict measured on a broken baseline is worthless; fix the baseline and re-run"
+  aud  "B1/B3/B5: NOT MEASURED — baseline egress was already broken (see A1)"
+fi
+if [ "$BASELINE_OK" -eq 1 ]; then
 
 # D1 — dns.mode=none (#15 B3): must kill sibling resolution, must NOT kill egress.
 # Runs 2 and 3 DISAGREED on the egress half (broken, then intact) — a verdict
 # that flips is a verdict you cannot design on. Setting dns.mode restarts the
 # network's dnsmasq, so a probe fired immediately can catch it mid-restart.
 # Distinguish TRANSIENT (recovers) from BROKEN (still dead after 30s), and say so.
-if incus network set claudenet dns.mode=none 2>/dev/null; then
+if incus network set claudenet dns.mode=none 2>/tmp/dnsmode.err; then
   sleep 2
   [ -n "$(in_box archive getent hosts peer.incus)" ] \
     && { no "dns.mode=none did not stop sibling resolution"; aud "B3 dns.mode=none: does NOT close the leak"; } \
@@ -571,8 +589,8 @@ if incus network set claudenet dns.mode=none 2>/dev/null; then
     fi
   fi
 else
-  no "incus rejected dns.mode=none on claudenet"
+  no "incus rejected dns.mode=none on claudenet: $(head -1 /tmp/dnsmode.err 2>/dev/null)"
-  aud "B3 dns.mode=none: REJECTED by incus — #16 needs another mechanism"
+  aud "B3 dns.mode=none: REJECTED by incus — $(head -1 /tmp/dnsmode.err 2>/dev/null) — #16 needs another mechanism"
 fi
 
 # D2 — L2 filtering (#15 B5): the box must keep working with it on.
@@ -614,6 +632,8 @@ else
   aud "B1 @internal: rejected ⇒ #16 derives the subnet in setup-host.sh (mask the gateway CIDR)"
 fi
 
+fi   # end of the BASELINE_OK guard around phase D
+
 # ===========================================================================
 if [ "$KEEP" = 1 ]; then
   phase "Boxes left up (--keep-boxes)"