CHANGELOG.md

@@ -115,6 +115,45 @@ which records not just what changed but what each drill run proved.
   deletion near-miss and the real duplicate — each with `changelog-armed.sh`
   asserted green on it, which is the whole reason this script exists.
 
+- **An upgrade over a pre-0.7.0 flat `/opt/box` no longer skips host setup**
+  (#115) — found on the first real host the 0.8.0 drill touched. The
+  installer migrates a flat pre-0.7.0 tree into `versions/<v>`, and
+  `had_install` was computed *after* that migration — so it observed a
+  `versions/` directory the migration had just created, concluded the host
+  was already installed, and skipped `host/setup-host.sh`. The result was
+  silent and self-concealing: `box --version` reported 0.8.0 while every
+  host-side artifact stayed as the old release left it, so the very
+  operator who upgraded *for* the #102 `box-firewall` SIGPIPE fix was the
+  one who did not receive it, with the version string asserting otherwise.
+  `had_install` is now computed **before** the migration block, which is
+  the honest question — a tree that needs migrating has by definition never
+  been converged by this version's `setup-host`. Hosts already on the
+  versioned layout are unaffected: they still read `had_install=1`, for the
+  right reason. The consequence is deliberate: an unattended (`BOX_YES=1`)
+  upgrade on a flat-tree host now *runs* `setup-host`, which the #66 note
+  cautions about — accepted, because `setup-host` converges and is
+  idempotent, and shipping a release whose host half is silently missing is
+  the worse failure.
+- **Host setup runs the version it just installed, not whatever `current`
+  points at** (#115) — a second defect in the same block, reachable only
+  once the fix above lets `setup-host` run at all. The `#66` guard holds
+  the default where it is when the host has existing boxes, so on such a
+  host `current` still names the OLD version; running
+  `$DEST/current/host/setup-host.sh` would then converge the host with the
+  *previous* release's host scripts, reinstating exactly the staleness #115
+  is about, in the one case where the operator's live boxes make it
+  costly. It now runs the installed version's own tree directly.
+- **The pre-0.7.0 migration says what it left behind** (#117) — the
+  migration named itself, but not the *lifecycle*: the old tree becomes a
+  first-class `box versions` entry the operator never installed, and which
+  is indistinguishable from one they deliberately kept as a rollback
+  target. The migration line now names both ways out — keep it to roll back
+  (`box use <v>`) or reap it (`box uninstall <v>`) — and the closing `done`
+  summary re-states it, because the migration line itself scrolls past some
+  250 lines before the install ends. Deleting it automatically stays the
+  wrong default: it is the only thing to roll back *to*, at exactly the
+  moment that matters. No behaviour change.
+
 ## 0.8.0 — 2026-07-19
 
 ### Added

install.sh

@@ -143,12 +143,39 @@ flip_current() {
   mv -Tf "$DEST/current.new.$$" "$DEST/current"
 }
 
+# Whether ANY version was installed before this run — computed BEFORE the
+# migration below, which is the whole point. It gates the host-setup offer: a
+# host that already ran box has made that decision (and may have live boxes the
+# stack must not be rebuilt under, #66); after an upgrade, 'box setup-host'
+# re-applies stack changes on purpose.
+#
+# Order is load-bearing (#115). The migration converts a pre-0.7.0 flat tree
+# into versions/<flat_ver>, so computing this AFTER it made the test true by
+# its own doing: a flat host looked "already installed", setup-host was
+# skipped, and every host-side artifact stayed at the old version while
+# 'box --version' reported the new one. A tree that needs migrating has by
+# definition never been converged by THIS version's setup-host, so it must
+# read as had_install=0. A genuinely versioned tree still reads 1 — the
+# directory it is testing predates this run.
+had_install=0
+if [ -d "$DEST/versions" ] && [ -n "$(ls -A "$DEST/versions" 2>/dev/null)" ]; then
+  had_install=1
+fi
+
 # --- migrate a pre-0.7.0 flat install --------------------------------------
 # 0.6.0 and earlier installed the tree FLAT at $DEST (bin/box directly under
 # it). Move such a tree to versions/<its-VERSION> BEFORE anything else, so an
 # upgrade from 0.6.0 is seamless and the version comparison below sees the
 # truth. The move is two renames inside one parent directory — no copying, no
 # window with no install — and the operator's tree is preserved bit for bit.
+#
+# What the migration LEAVES is the operator's to decide (#117): the old tree
+# becomes a first-class 'box versions' entry — a rollback target if the new
+# version misbehaves, garbage otherwise. Deleting it here is the wrong default
+# (it is the only thing to roll back TO, at exactly the moment that matters),
+# so name it instead — and name it AGAIN in the closing summary, because a
+# line ~250 lines of output above 'done' is a line the operator scrolled past.
+migrated_from=""
 if [ -e "$DEST/bin/box" ] && [ ! -d "$DEST/versions" ]; then
   flat_ver="$(cat "$DEST/VERSION" 2>/dev/null || echo 0.0.0-unknown)"
   # The flat tree's VERSION is data from disk, not from this installer — the
@@ -164,15 +191,9 @@ if [ -e "$DEST/bin/box" ] && [ ! -d "$DEST/versions" ]; then
   mkdir -p "$BINDIR"
   ln -sfn "$DEST/current/bin/box" "$BINDIR/box"
   log "migrated: it now lives at $DEST/versions/$flat_ver (still current; your boxes are untouched)"
-fi
-
-# Whether ANY version was installed before this run — read before we add one.
-# It gates the host-setup offer below: a host that already ran box has made
-# that decision (and may have live boxes the stack must not be rebuilt under,
-# #66); after an upgrade, 'box setup-host' re-applies stack changes on purpose.
-had_install=0
-if [ -d "$DEST/versions" ] && [ -n "$(ls -A "$DEST/versions" 2>/dev/null)" ]; then
-  had_install=1
+  log "  it is a normal version entry now — 'box versions' lists it. Keep it as a"
+  log "  rollback target ('box use $flat_ver'), or reap it: box uninstall $flat_ver"
+  migrated_from="$flat_ver"
 fi
 
 # --- temp workspace --------------------------------------------------------
@@ -386,6 +407,15 @@ esac
 # 'box setup-host' re-applies stack changes deliberately, after an upgrade.
 # BOX_SKIP_SETUP_HOST=1 answers "no" without prompting (image builds, a host set
 # up by hand); BOX_YES answers "yes".
+#
+# It runs $VDIR's script, NOT $DEST/current's. They are usually the same tree,
+# but 'current' does not always flip: the #66 guard above keeps the default
+# where it is when the host has existing boxes, so on such a host 'current'
+# still names the OLD version. Going through it would converge the host with
+# the old release's host-side scripts — reinstating exactly the stale artifacts
+# #115 is about, in the one case where the operator's boxes make it costly.
+# $VDIR is unambiguously the version this run installed, which is the version
+# whose host contract we are being asked to satisfy.
 setup_ok=""
 setup_declined=""
 if [ "$had_install" -eq 1 ]; then
@@ -396,7 +426,7 @@ elif [ -n "${BOX_SKIP_SETUP_HOST:-}" ]; then
   setup_declined=1
 elif [ "$(id -u)" -ne 0 ] && ! command -v sudo >/dev/null 2>&1; then
   warn "cannot set up the host: it needs root and sudo was not found."
-  warn "  run this as root to finish: $DEST/current/host/setup-host.sh"
+  warn "  run this as root to finish: $VDIR/host/setup-host.sh"
   setup_declined=1
 elif confirm "Set up this machine as a box host now? (installs Incus + the isolation stack; needs sudo)"; then
   # </dev/null because under 'curl … | bash' this script IS stdin: a child that
@@ -404,7 +434,7 @@ elif confirm "Set up this machine as a box host now? (installs Incus + the isola
   # it prompts on /dev/tty, so an interactive host can still authenticate.
   # setup-host re-execs itself under sg incus-admin if it must add you to the
   # group; that re-exec is a child here and completes the whole setup in one go.
-  if bash "$DEST/current/host/setup-host.sh" </dev/null; then
+  if bash "$VDIR/host/setup-host.sh" </dev/null; then
     setup_ok=1
   else
     warn "host setup did not complete — box is installed, the host is not ready."
@@ -422,3 +452,12 @@ elif [ -n "$setup_declined" ]; then
 else
   log "done ($SRCDESC, version $new_ver) — finish with 'box setup-host', then: box new --name test"
 fi
+
+# Re-state the migration where the operator is actually looking (#117). The
+# line at the migration itself is true but ~250 lines of output ago; this is
+# the last thing printed, and it is where an unexplained 'box versions' entry
+# stops being a surprise.
+if [ -n "$migrated_from" ]; then
+  log "note: your pre-0.7.0 install was migrated to versions/$migrated_from and 'box versions' now lists it."
+  log "  keep it to roll back ('box use $migrated_from'), or remove it: box uninstall $migrated_from"
+fi

test/cli.sh

@@ -1695,9 +1695,17 @@ chmod +x "$ISHIM/incus"
 
 # A fabricated "newer release": the same CLI, a different VERSION — what an
 # upgrade actually is, from the installer's point of view.
-SRC9="$WORK/src-9.9.9"; mkdir -p "$SRC9/bin"
+SRC9="$WORK/src-9.9.9"; mkdir -p "$SRC9/bin" "$SRC9/host"
 cp "$ROOT/bin/box" "$SRC9/bin/box"; chmod +x "$SRC9/bin/box"
 echo "9.9.9-drill" > "$SRC9/VERSION"
+# A stub host/setup-host.sh that only announces itself: enough to prove WHETHER
+# the installer ran host setup, and from WHICH version's tree, with no Incus and
+# no root. The real script builds the isolation stack; this one echoes (#115).
+cat > "$SRC9/host/setup-host.sh" <<'STUB'
+#!/usr/bin/env bash
+echo "SETUP-HOST-RAN-FROM 9.9.9-drill"
+STUB
+chmod +x "$SRC9/host/setup-host.sh"
 SRC8="$WORK/src-8.8.8"; mkdir -p "$SRC8/bin"
 cp "$ROOT/bin/box" "$SRC8/bin/box"; chmod +x "$SRC8/bin/box"
 echo "8.8.8-drill" > "$SRC8/VERSION"
@@ -1708,6 +1716,15 @@ inst() {  # inst <box_home> <box_bin> [VAR=val ...] — run install.sh for real
       BOX_HOME="$h" BOX_BIN="$b" BOX_YES=1 BOX_SKIP_SETUP_HOST=1 \
       BOX_INSTALL_SOURCE="$ROOT" "$@" bash "$ROOT/install.sh"
 }
+inst_setup() {  # like inst, but WITHOUT BOX_SKIP_SETUP_HOST — host setup is the
+  # thing under test, so the switch that suppresses it has to come off. Safe
+  # offline: the only setup-host on these fabricated sources is the echo stub
+  # above, and BOX_YES=1 answers its prompt.
+  local h="$1" b="$2"; shift 2
+  env HOME="$FAKEHOME" PATH="$ISHIM:$PATH" FAKE_BOXES= \
+      BOX_HOME="$h" BOX_BIN="$b" BOX_YES=1 \
+      BOX_INSTALL_SOURCE="$ROOT" "$@" bash "$ROOT/install.sh"
+}
 ibox() {  # ibox [VAR=val ...] <cmd...> — run an installed box under the shim
   env HOME="$FAKEHOME" PATH="$ISHIM:$PATH" FAKE_BOXES= "$@"
 }
@@ -1783,6 +1800,32 @@ check "migrate: current points at the migrated version" 0 "versions/$VER" readli
 check "migrate: the PATH symlink was re-pointed through current" 0 "$H3/current/bin/box" readlink "$B3/box"
 check "migrate: the migrated install answers --version" 0 "box $VER" ibox "$B3/box" --version
 
+# #117: the migration is not silent about the entry it manufactured. The old
+# tree is now a first-class 'box versions' row the operator never installed —
+# so the output has to name the way back out (uninstall) and the reason to
+# keep it (rollback), at the migration AND again in the closing summary, which
+# is the half an operator scrolling ~250 lines of install output actually sees.
+H3B="$WORK/h3b"; B3B="$WORK/b3b"; mkdir -p "$H3B/bin" "$B3B"
+cp "$ROOT/bin/box" "$H3B/bin/box"; chmod +x "$H3B/bin/box"
+cp "$ROOT/VERSION" "$H3B/VERSION"
+ln -s "$H3B/bin/box" "$B3B/box"
+mig_out="$WORK/mig-out.txt"
+inst "$H3B" "$B3B" BOX_INSTALL_SOURCE="$SRC9" >"$mig_out" 2>&1 || true
+check "migrate: the output points at the reap command (#117)" 0 "box uninstall $VER" \
+  cat "$mig_out"
+check "migrate: ...and names keeping it as a rollback target (#117)" 0 "keep it to roll back" \
+  cat "$mig_out"
+check "migrate: ...and the closing summary re-states it (#117)" 0 "was migrated to versions/$VER" \
+  cat "$mig_out"
+# ...and the note is conditional: an install with nothing to migrate must not
+# mention a migration at all. grep exits 1 when the string is absent, which is
+# the pass here.
+nomig_out="$WORK/nomig-out.txt"
+H3C="$WORK/h3c"; B3C="$WORK/b3c"
+inst "$H3C" "$B3C" >"$nomig_out" 2>&1 || true
+check "migrate: a NON-migrating install stays silent about migration (#117)" 1 "" \
+  grep -qF "was migrated to versions/" "$nomig_out"
+
 # ...and the seamless 0.6.0 → 0.7.0 upgrade: flat tree in, new version beside it.
 H4="$WORK/h4"; B4="$WORK/b4"; mkdir -p "$H4/bin" "$B4"
 cp "$ROOT/bin/box" "$H4/bin/box"; chmod +x "$H4/bin/box"
@@ -1795,6 +1838,44 @@ check "migrate+upgrade: both versions present" 0 "" \
 check "migrate+upgrade: no boxes → the new version is the default" 0 "box 9.9.9-drill" \
   ibox "$B4/box" --version
 
+# #115, end to end and fully offline: a flat pre-0.7.0 tree must still count as
+# "no install yet" and RUN host setup. The migration converts the flat tree into
+# versions/<v>, which is precisely what used to make had_install read 1 — the
+# host then skipped setup-host while 'box --version' reported the new release,
+# leaving every host-side artifact (box-firewall, #102) at the old one. The stub
+# setup-host echoes a marker, so the marker IS the proof it ran.
+H4B="$WORK/h4b"; B4B="$WORK/b4b"; mkdir -p "$H4B/bin" "$B4B"
+cp "$ROOT/bin/box" "$H4B/bin/box"; chmod +x "$H4B/bin/box"
+cp "$ROOT/VERSION" "$H4B/VERSION"
+ln -s "$H4B/bin/box" "$B4B/box"
+check "flat upgrade: host setup RUNS over a migrated flat tree (#115)" 0 "SETUP-HOST-RAN-FROM 9.9.9-drill" \
+  inst_setup "$H4B" "$B4B" BOX_INSTALL_SOURCE="$SRC9"
+
+# The converse, so the gate is proven to still GATE: H4B is now a genuinely
+# versioned tree, which HAS already made the host-setup decision — a re-run must
+# not redo it. Without this, "fix" and "run setup-host unconditionally" would be
+# indistinguishable.
+vers_out="$WORK/versioned-upgrade-out.txt"
+inst_setup "$H4B" "$B4B" BOX_INSTALL_SOURCE="$SRC8" >"$vers_out" 2>&1 || true
+check "versioned upgrade: an existing versioned install still SKIPS host setup" 0 "already had a box install" \
+  cat "$vers_out"
+check "versioned upgrade: ...and the stub did NOT run" 1 "" \
+  grep -qF "SETUP-HOST-RAN-FROM" "$vers_out"
+
+# 'current' does not always flip: the #66 guard holds the default under existing
+# boxes. Host setup must still come from the version just installed, or the
+# upgrade converges the host with the OLD release's host scripts — reinstating
+# the very staleness #115 is about. The flat fixture carries no host/ dir at all,
+# so going through 'current' could not even find a script to run.
+H10="$WORK/h10"; B10="$WORK/b10"; mkdir -p "$H10/bin" "$B10"
+cp "$ROOT/bin/box" "$H10/bin/box"; chmod +x "$H10/bin/box"
+cp "$ROOT/VERSION" "$H10/VERSION"
+ln -s "$H10/bin/box" "$B10/box"
+check "flat upgrade under boxes: setup-host runs the NEW version's script" 0 "SETUP-HOST-RAN-FROM 9.9.9-drill" \
+  inst_setup "$H10" "$B10" BOX_INSTALL_SOURCE="$SRC9" FAKE_BOXES=work
+check "flat upgrade under boxes: ...while the default correctly stayed put (#66)" 0 "box $VER" \
+  ibox "$B10/box" --version
+
 # A broken current must halt the single-version path BEFORE any decision: the
 # CURRENT guard keys off what current resolves to, and a dangling link makes
 # that answer a lie. Drive the version tree's own binary — the current chain