feat: snapshot 'pristine' at mint, before the rig bootstrap hook #128
5 changed files with 419 additions and 0 deletions
38
CHANGELOG.md
38
CHANGELOG.md
|
|
@ -74,6 +74,44 @@ which records not just what changed but what each drill run proved.
|
||||||
cpu/memory/disk/tier out of the namespace and the lineage keys out of the
|
cpu/memory/disk/tier out of the namespace and the lineage keys out of the
|
||||||
clone's re-stamp.
|
clone's re-stamp.
|
||||||
|
|
||||||
|
- **Every fresh mint marks a `pristine` snapshot, before rig converges
|
||||||
|
anything** (#104, child of heavy-duty/rig#62) — `cmd_new` runs launch →
|
||||||
|
`wait_agent` → `cloud-init status --wait` → the rig bootstrap hook, and in
|
||||||
|
the gap between the last two the guest **is** pristine Debian plus box's
|
||||||
|
thin seed. That is exactly the state rig#62's "back to pristine Debian"
|
||||||
|
names; it existed for a few seconds on every mint box has ever taken, and
|
||||||
|
nothing captured it. Now `box restore <box> pristine` is the whole answer
|
||||||
|
for every creds-free tenant role, because everything `rig bootstrap
|
||||||
|
claude|codex|grok|staging` does — docker, node, the agent CLI, the
|
||||||
|
agent-context file, the role marker — is box-local and file-shaped, so a
|
||||||
|
filesystem rollback reaches all of it without paying a ~10-minute re-mint.
|
||||||
|
Default on and unconditional within a fresh mint (a blank box has no rig
|
||||||
|
hook but has the same moment, and the label must mean one thing on every
|
||||||
|
box), never fatal (a mint that worked is not failed by a checkpoint that
|
||||||
|
didn't), and **never taken on a `--from` clone**: a clone skips cloud-init
|
||||||
|
and rig entirely, so it has no pristine moment, and a mark taken there
|
||||||
|
would be the source's worked-in state wearing a label that promises
|
||||||
|
pristine Debian. A clone inherits its source's snapshots (copying a box) or
|
||||||
|
starts with none (copying a snapshot), and `box new` says which. On a host
|
||||||
|
whose pool uses the `dir` driver — the documented fallback in
|
||||||
|
`host/setup-host.sh`, where there is no copy-on-write and the mark would be
|
||||||
|
a full multi-GB copy of the root on every mint — the mint skips it loudly
|
||||||
|
and names the by-hand command, rather than silently doubling every mint's
|
||||||
|
disk cost; the driver is read back from the `box-net` profile's own root
|
||||||
|
pool, so it is the pool that actually placed the instance. Snapshots do not
|
||||||
|
outlive their box (`box rm` takes every one with it), so the help text,
|
||||||
|
README and design doc say plainly that this is an **undo, not a backup** —
|
||||||
|
`box export` remains the durable path — and that it cannot reach off-box
|
||||||
|
state such as a tailnet join or a runner registration. `BOX_SNAPSHOT_PRISTINE=0`
|
||||||
|
opts out, in the `BOX_LAUNCH_TIMEOUT` shape rather than a new flag.
|
||||||
|
The never-fatal contract is now structural rather than incidental: all three
|
||||||
|
storage probes carry `|| true`, so a restricted tier's refusal stays an
|
||||||
|
answer even if `inherit_errexit` is ever switched on in `bin/box` (#107's
|
||||||
|
class), and the clone's inheritance read captures before it greps rather
|
||||||
|
than piping a multi-line `incus` writer into an early-exit reader (#124's
|
||||||
|
class), which under `pipefail` could have narrated the wrong inheritance
|
||||||
|
shape on a clone that does carry a `pristine`.
|
||||||
|
|
||||||
### Changed
|
### Changed
|
||||||
|
|
||||||
- **`state:needs-human` no longer waits on the cron to become true** (#141)
|
- **`state:needs-human` no longer waits on the cron to become true** (#141)
|
||||||
|
|
|
||||||
44
README.md
44
README.md
|
|
@ -332,6 +332,48 @@ refuses rather than assuming yes).
|
||||||
Forgotten what you called a checkpoint? `box info work` prints the box's
|
Forgotten what you called a checkpoint? `box info work` prints the box's
|
||||||
snapshot labels and the `--from` line to clone one.
|
snapshot labels and the `--from` line to clone one.
|
||||||
|
|
||||||
|
### `pristine` — the one checkpoint box takes for you
|
||||||
|
|
||||||
|
Every fresh mint marks a snapshot called `pristine`
|
||||||
|
([#104](https://github.com/heavy-duty/box/issues/104)) at the one moment it
|
||||||
|
is true: **after cloud-init, before `rig bootstrap` converges the tenant
|
||||||
|
role.** At that instant the guest is pristine Debian plus box's thin seed
|
||||||
|
(the user, tmux, rig) and nothing else — the state
|
||||||
|
[heavy-duty/rig#62](https://github.com/heavy-duty/rig/issues/62) calls "back
|
||||||
|
to pristine Debian". It exists for a few seconds on every mint, so box
|
||||||
|
captures it rather than asking you to be quick.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
box restore work pristine # undo the tenant role and everything since
|
||||||
|
```
|
||||||
|
|
||||||
|
That is a complete undo for every tenant role: everything `rig bootstrap
|
||||||
|
claude|codex|grok|staging` does — docker, node, the agent CLI, the
|
||||||
|
agent-context file, the role marker — is box-local and file-shaped, so a
|
||||||
|
filesystem rollback reaches all of it, without paying a ~10-minute re-mint.
|
||||||
|
|
||||||
|
Three things it deliberately does not do:
|
||||||
|
|
||||||
|
- **It is an undo, not a backup.** Snapshots die with their box: `box rm`
|
||||||
|
deletes a box _and_ every snapshot it has. `box export` is the only state
|
||||||
|
that outlives the box — see below.
|
||||||
|
- **It cannot reach off-box state.** A tailnet join, a GitHub runner
|
||||||
|
registration, a pushed commit: those are records held somewhere else, and
|
||||||
|
no filesystem rollback undoes them (rig#62 covers those separately).
|
||||||
|
- **A `--from` clone gets no `pristine` of its own.** A clone skips
|
||||||
|
cloud-init and rig entirely, so it has no pristine moment to capture, and
|
||||||
|
box will not label a source's worked-in state as one. Cloning a _box_
|
||||||
|
inherits the source's snapshots (a real `pristine` among them, if the
|
||||||
|
source had one); cloning a _snapshot_ starts with none. `box new` says
|
||||||
|
which of the two you got.
|
||||||
|
|
||||||
|
On a host whose storage pool uses the `dir` driver, a snapshot is a full
|
||||||
|
multi-GB copy rather than a near-free copy-on-write mark, so the mint
|
||||||
|
**skips** `pristine` and says so loudly — take it by hand with `box snapshot
|
||||||
|
<box> pristine` if you want it anyway. btrfs is what `box setup-host`
|
||||||
|
installs by default precisely so snapshots are cheap. `BOX_SNAPSHOT_PRISTINE=0`
|
||||||
|
skips the mark on any host.
|
||||||
|
|
||||||
## Survive the host: `box export` / `box import`
|
## Survive the host: `box export` / `box import`
|
||||||
|
|
||||||
Snapshots live _inside_ a box, and `box rm` deletes the box **and** its
|
Snapshots live _inside_ a box, and `box rm` deletes the box **and** its
|
||||||
|
|
@ -397,6 +439,8 @@ box tmux <box> [session] # attach/create a tmux session — survives disconn
|
||||||
box snapshot <box> [label] # checkpoint (label defaults to manual-<epoch>)
|
box snapshot <box> [label] # checkpoint (label defaults to manual-<epoch>)
|
||||||
box restore <box> <snap> [--force]
|
box restore <box> <snap> [--force]
|
||||||
# roll back to a snapshot — destructive, asks first
|
# roll back to a snapshot — destructive, asks first
|
||||||
|
# 'pristine' is auto-marked at mint: back to
|
||||||
|
# pristine Debian + box's seed, before rig ran
|
||||||
box export <box> [<file>] [--instance-only]
|
box export <box> [<file>] [--instance-only]
|
||||||
# one portable file (snapshots incl.) — survives rm & host
|
# one portable file (snapshots incl.) — survives rm & host
|
||||||
box import <file> [--name <box>]
|
box import <file> [--name <box>]
|
||||||
|
|
|
||||||
149
bin/box
149
bin/box
|
|
@ -296,6 +296,21 @@ there is no flag for a network or a security key, on purpose.
|
||||||
BOX_LAUNCH_TIMEOUT=<seconds> (default 600) bounds the 'incus launch' call —
|
BOX_LAUNCH_TIMEOUT=<seconds> (default 600) bounds the 'incus launch' call —
|
||||||
a launch that overruns it fails loudly instead of hanging forever (#93).
|
a launch that overruns it fails loudly instead of hanging forever (#93).
|
||||||
|
|
||||||
|
Every fresh mint marks a snapshot called 'pristine' (#104) at the one moment
|
||||||
|
it is true: after cloud-init, before rig converges anything. That is the
|
||||||
|
guest as pristine Debian plus box's thin seed, and it is what
|
||||||
|
|
||||||
|
box restore <box> pristine
|
||||||
|
|
||||||
|
rolls back to — undoing the tenant role and everything done since, without
|
||||||
|
paying a re-mint. It is an UNDO, not a backup: 'box rm' deletes a box and
|
||||||
|
every snapshot it has, so 'box export' remains the only state that outlives
|
||||||
|
the box. A --from clone takes no 'pristine' of its own — it has no pristine
|
||||||
|
moment; it inherits the source's snapshots (copying a box) or starts with
|
||||||
|
none (copying a snapshot). On a 'dir'-backend host, where a snapshot is a
|
||||||
|
full copy rather than a CoW mark, the mint skips it and says so.
|
||||||
|
BOX_SNAPSHOT_PRISTINE=0 skips it anywhere.
|
||||||
|
|
||||||
box new --name scratch # blank, the default
|
box new --name scratch # blank, the default
|
||||||
box new --name work --template claude-box
|
box new --name work --template claude-box
|
||||||
box new --name lean --template claude-box --cpu 2 --memory 3GiB
|
box new --name lean --template claude-box --cpu 2 --memory 3GiB
|
||||||
|
|
@ -377,6 +392,13 @@ label defaults to manual-<epoch>; 'box info <box>' shows the labels you
|
||||||
have.
|
have.
|
||||||
|
|
||||||
box snapshot work authed
|
box snapshot work authed
|
||||||
|
|
||||||
|
One label is taken for you: every fresh mint marks 'pristine' after
|
||||||
|
cloud-init and before rig's tenant role runs (#104) — see 'box help restore'.
|
||||||
|
|
||||||
|
Snapshots do not outlive their box: 'box rm' deletes a box and every
|
||||||
|
snapshot it has. They are an undo, not a backup — 'box export' is the
|
||||||
|
durable path.
|
||||||
EOF
|
EOF
|
||||||
;;
|
;;
|
||||||
restore) cat <<'EOF'
|
restore) cat <<'EOF'
|
||||||
|
|
@ -385,6 +407,22 @@ that snapshot is lost. 'box info <box>' lists the labels.
|
||||||
|
|
||||||
box restore work authed
|
box restore work authed
|
||||||
box restore work authed --force
|
box restore work authed --force
|
||||||
|
box restore work pristine
|
||||||
|
|
||||||
|
'pristine' is the one label box takes for you (#104): every fresh mint marks
|
||||||
|
it after cloud-init and before rig converges the tenant role, so it is the
|
||||||
|
guest as pristine Debian plus box's thin seed. Restoring it is the answer to
|
||||||
|
"I broke this box" for every tenant role — everything 'rig bootstrap
|
||||||
|
<tenant>' does is box-local and file-shaped, so a filesystem rollback undoes
|
||||||
|
all of it, without a ~10-minute re-mint.
|
||||||
|
|
||||||
|
What it does NOT undo is anything that left the box: a tailnet join, a
|
||||||
|
GitHub runner registration, a pushed commit. Those are off-box records a
|
||||||
|
filesystem rollback cannot reach (heavy-duty/rig#62). And 'pristine' is not
|
||||||
|
a backup — it dies with the box on 'box rm'; 'box export' is what survives.
|
||||||
|
|
||||||
|
A box minted with --from has no 'pristine' of its own unless it inherited
|
||||||
|
one from its source, because a clone has no pristine moment to capture.
|
||||||
|
|
||||||
Destructive, so it asks first — naming the snapshot it is rolling back to,
|
Destructive, so it asks first — naming the snapshot it is rolling back to,
|
||||||
because the whole risk is picking the wrong label. --force (-f) skips the
|
because the whole risk is picking the wrong label. --force (-f) skips the
|
||||||
|
|
@ -1052,6 +1090,77 @@ cmd_templates() {
|
||||||
# thing on every host that reads it back.
|
# thing on every host that reads it back.
|
||||||
mint_time() { date -u +%Y-%m-%dT%H:%M:%SZ; }
|
mint_time() { date -u +%Y-%m-%dT%H:%M:%SZ; }
|
||||||
|
|
||||||
|
# Which storage driver is under this box? The placement contract answers it:
|
||||||
|
# every box launches with --profile box-net, and that profile hardcodes the
|
||||||
|
# root disk's pool (profiles/box-net.yaml), so the pool is not a guess — it is
|
||||||
|
# read back from the profile that actually placed the instance, and only falls
|
||||||
|
# back to the name setup-host.sh creates. Two probes for the driver because
|
||||||
|
# the tiers see storage differently: 'incus storage show' is the admin read
|
||||||
|
# (the same line setup-host.sh:297 prints), 'incus storage list --format csv'
|
||||||
|
# is the one a restricted user's incus-user proxy is likelier to answer.
|
||||||
|
# Prints the driver, or nothing when neither probe answers.
|
||||||
|
storage_driver() {
|
||||||
|
local pool driver
|
||||||
|
pool="$(incus profile device get box-net root pool 2>/dev/null || true)"
|
||||||
|
[ -n "$pool" ] || pool=default
|
||||||
|
# '|| true' on all three probes, not just the first. A refusal here is an
|
||||||
|
# ANSWER ("this tier cannot read the pool"), never an error — this function
|
||||||
|
# must never be the thing that kills a mint. Today the two pipelines are
|
||||||
|
# safe only by accident: command substitution strips errexit, so a failing
|
||||||
|
# 'incus storage show' under pipefail falls through to the fallback instead
|
||||||
|
# of aborting. Add 'shopt -s inherit_errexit' to this file — exactly the
|
||||||
|
# robustness tweak #107 describes sailing through review — and that accident
|
||||||
|
# reverses into a fatal abort mid-mint, inside the function whose contract
|
||||||
|
# is NEVER fatal. Make the three probes read alike and depend on nothing.
|
||||||
|
driver="$(incus storage show "$pool" 2>/dev/null | awk '/^driver:/ {print $2; exit}' || true)"
|
||||||
|
[ -n "$driver" ] || driver="$(incus storage list --format csv 2>/dev/null \
|
||||||
|
| awk -F, -v p="$pool" '$1 == p {print $2; exit}' || true)"
|
||||||
|
printf '%s' "$driver"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Take the 'pristine' snapshot, or say loudly why not. Default ON: the value
|
||||||
|
# of this mark only exists if it is already there on the bad day, and nobody
|
||||||
|
# takes it by hand at the one moment it is true. The escape hatch is the
|
||||||
|
# BOX_LAUNCH_TIMEOUT shape — an environment knob, not another flag on 'new'.
|
||||||
|
snapshot_pristine() {
|
||||||
|
local instance="$1" name="$2" driver
|
||||||
|
if [ "${BOX_SNAPSHOT_PRISTINE:-1}" = 0 ]; then
|
||||||
|
echo "box: skipping the 'pristine' snapshot (BOX_SNAPSHOT_PRISTINE=0)."
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
driver="$(storage_driver)"
|
||||||
|
# On btrfs (the deliberate default, host/setup-host.sh:261-298) this costs
|
||||||
|
# kilobytes until the guest diverges. On the documented 'dir' fallback
|
||||||
|
# (:294) there is no CoW at all: the snapshot is a full copy of the root,
|
||||||
|
# several GB and minutes, on EVERY mint. Skip it there — but loudly, and
|
||||||
|
# naming the by-hand command, because a silent skip teaches an operator to
|
||||||
|
# expect a mark that will not be there.
|
||||||
|
if [ "$driver" = dir ]; then
|
||||||
|
echo "box: NOT taking the 'pristine' snapshot — this host's storage pool is the 'dir'" >&2
|
||||||
|
echo "box: driver, which has no copy-on-write, so the mark would be a full multi-GB copy" >&2
|
||||||
|
echo "box: of the root disk on every mint rather than the near-free btrfs one." >&2
|
||||||
|
echo "box: take it by hand if you want it here: box snapshot $name pristine" >&2
|
||||||
|
echo "box: or fix the host (btrfs is what 'box setup-host' installs by default)." >&2
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
# Neither probe answered — an unusual host, or a tier that cannot read the
|
||||||
|
# pool. Proceed, and say what was assumed. The two mistakes are not
|
||||||
|
# symmetric: a snapshot taken on a 'dir' host wastes disk the operator can
|
||||||
|
# see in 'box info' and delete, while a snapshot NOT taken is the pristine
|
||||||
|
# moment gone for good, which is the whole thing this exists to prevent.
|
||||||
|
[ -n "$driver" ] || echo "box: could not read the storage driver — taking 'pristine' anyway," \
|
||||||
|
"assuming copy-on-write (BOX_SNAPSHOT_PRISTINE=0 to skip)." >&2
|
||||||
|
echo "box: marking 'pristine' — the guest after cloud-init, before rig converges anything."
|
||||||
|
if ! incus snapshot create "$instance" pristine </dev/null; then
|
||||||
|
# An undo is not the mint's product: a mint that worked must not be
|
||||||
|
# failed by a checkpoint that didn't.
|
||||||
|
echo "box: WARNING — could not take the 'pristine' snapshot (incus said why above)." >&2
|
||||||
|
echo "box: the box is fine; it just has no pristine mark. The mint continues." >&2
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
echo "box: roll back to it any time with: box restore $name pristine"
|
||||||
|
}
|
||||||
|
|
||||||
cmd_new() {
|
cmd_new() {
|
||||||
[ -n "$name" ] || usage_error "usage: $(synopsis_of new)"
|
[ -n "$name" ] || usage_error "usage: $(synopsis_of new)"
|
||||||
require_stack
|
require_stack
|
||||||
|
|
@ -1108,6 +1217,33 @@ cmd_new() {
|
||||||
wait_agent "$instance"
|
wait_agent "$instance"
|
||||||
reset_identity "$instance"
|
reset_identity "$instance"
|
||||||
echo "box: cloned $srcref — isolation and auth state carry over from the source."
|
echo "box: cloned $srcref — isolation and auth state carry over from the source."
|
||||||
|
# A clone gets NO 'pristine' of its own (#104). It skips cloud-init and
|
||||||
|
# the rig hook entirely, so the pristine MOMENT never happens here —
|
||||||
|
# anything box marked at clone time would be "whatever the source was",
|
||||||
|
# converged and worked-in, wearing a label that promises pristine Debian.
|
||||||
|
# That is worse than no mark at all, so the mint-path snapshot lives
|
||||||
|
# strictly in the fresh-mint branch below and never runs here.
|
||||||
|
#
|
||||||
|
# What a clone gets instead is INHERITANCE, and it is already correct:
|
||||||
|
# 'incus copy <box>' carries the source's snapshots, so a source that has
|
||||||
|
# a 'pristine' hands the clone a real one — the true pristine state of
|
||||||
|
# the disk this clone descends from. Copying a SNAPSHOT ('--from box/snap')
|
||||||
|
# carries no snapshot list, so that clone starts with none. Both are
|
||||||
|
# honest; neither is invented. Say which one this is rather than leaving
|
||||||
|
# the operator to run 'box info' to find out.
|
||||||
|
# Capture first, THEN read (#124's class). Piping a multi-line incus
|
||||||
|
# writer straight into an early-exit reader lets grep close the pipe on
|
||||||
|
# the first match, SIGPIPE incus, and hand pipefail a 141 — which here
|
||||||
|
# would read as "no pristine" and narrate the WRONG inheritance shape on a
|
||||||
|
# clone that actually has one. Un-racy today at this writer's size, but
|
||||||
|
# the assignment costs nothing and does not depend on that staying true.
|
||||||
|
local snaps; snaps="$(incus snapshot list "$instance" --format csv 2>/dev/null || true)"
|
||||||
|
if printf '%s\n' "$snaps" | grep -q '^pristine,'; then
|
||||||
|
echo "box: it inherited the source's snapshots, 'pristine' among them (box restore $name pristine)."
|
||||||
|
else
|
||||||
|
echo "box: no 'pristine' mark here — a clone has no pristine moment to capture, and box"
|
||||||
|
echo "box: will not label the source's state as one. 'box info $name' lists what it did inherit."
|
||||||
|
fi
|
||||||
else
|
else
|
||||||
local t="${template:-blank}" m extra=()
|
local t="${template:-blank}" m extra=()
|
||||||
load_template "$t"
|
load_template "$t"
|
||||||
|
|
@ -1287,6 +1423,19 @@ cmd_new() {
|
||||||
echo " not serving, a VPN resolver the box inherits). Diagnose it: box doctor" >&2
|
echo " not serving, a VPN resolver the box inherits). Diagnose it: box doctor" >&2
|
||||||
die "cloud-init failed — the box is incomplete, so refusing to hand it over"
|
die "cloud-init failed — the box is incomplete, so refusing to hand it over"
|
||||||
fi
|
fi
|
||||||
|
# The pristine mark (#104, child of rig#62). RIGHT HERE — cloud-init has
|
||||||
|
# finished and rig has not started — the guest is pristine Debian plus
|
||||||
|
# box's seed, and nothing else. That state exists for a few seconds on
|
||||||
|
# every mint and has never been captured; rig#62's "back to pristine
|
||||||
|
# Debian" is exactly this state, so take it while it is true.
|
||||||
|
#
|
||||||
|
# Unconditional within a fresh mint, not gated on T_BOOTSTRAP_ROLE: a
|
||||||
|
# blank box has no rig hook but it does have the same pristine moment,
|
||||||
|
# and 'box restore <box> pristine' should mean one thing on every box.
|
||||||
|
#
|
||||||
|
# NEVER fatal. This is an undo, not the mint's product — a mint that
|
||||||
|
# worked must not be failed by a checkpoint that didn't.
|
||||||
|
snapshot_pristine "$instance" "$name"
|
||||||
# The tenant convergence (#81): the seed above is thin — the user, tmux,
|
# The tenant convergence (#81): the seed above is thin — the user, tmux,
|
||||||
# rig — and what the box BECOMES is rig's job. A template that names a
|
# rig — and what the box BECOMES is rig's job. A template that names a
|
||||||
# bootstrap role gets it auto-run here, as root inside the guest, because
|
# bootstrap role gets it auto-run here, as root inside the guest, because
|
||||||
|
|
|
||||||
|
|
@ -43,6 +43,18 @@ snapshots, not a secrets store:
|
||||||
|
|
||||||
Log in once → snapshot → spin up authed boxes from it.
|
Log in once → snapshot → spin up authed boxes from it.
|
||||||
|
|
||||||
|
One checkpoint box takes itself: **`pristine`**, marked on every fresh mint
|
||||||
|
after cloud-init and before the `rig bootstrap` hook (#104, child of
|
||||||
|
heavy-duty/rig#62). That instant — pristine Debian plus box's thin seed, rig
|
||||||
|
installed but not yet run — is the state "back to pristine Debian" names, it
|
||||||
|
lasts a few seconds, and nobody is standing there to snapshot it. `box
|
||||||
|
restore <n> pristine` undoes a tenant role wholesale, because everything the
|
||||||
|
creds-free roles do is box-local and file-shaped. A `--from` clone takes no
|
||||||
|
`pristine`: it never has the moment, so it inherits its source's snapshots or
|
||||||
|
has none, and box refuses to label a worked-in state as pristine. On a `dir`
|
||||||
|
pool the mark would be a full copy rather than a CoW one, so the mint skips
|
||||||
|
it loudly.
|
||||||
|
|
||||||
Snapshots are in-box state: `box rm` deletes a box *and* its snapshots, and a
|
Snapshots are in-box state: `box rm` deletes a box *and* its snapshots, and a
|
||||||
clone still lives on the same host. The off-host mechanism is `box export` /
|
clone still lives on the same host. The off-host mechanism is `box export` /
|
||||||
`box import` (#70) — one portable backup tarball, snapshots included by
|
`box import` (#70) — one portable backup tarball, snapshots included by
|
||||||
|
|
|
||||||
176
test/cli.sh
176
test/cli.sh
|
|
@ -454,6 +454,182 @@ check "new: the workload join is printed, never exec'd" 1 "" bash -c '
|
||||||
check "templates: no template names a creds-holding role" 1 "" bash -c '
|
check "templates: no template names a creds-holding role" 1 "" bash -c '
|
||||||
grep -h "^BOX_BOOTSTRAP_ROLE=" "'"$ROOT"'"/templates/*/box.env | grep -qE "workload|host|custom"'
|
grep -h "^BOX_BOOTSTRAP_ROLE=" "'"$ROOT"'"/templates/*/box.env | grep -qE "workload|host|custom"'
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# The 'pristine' mark (#104, child of heavy-duty/rig#62). The whole feature is
|
||||||
|
# a MOMENT: the guest after cloud-init and before rig converges anything. Get
|
||||||
|
# the position wrong by one step and the mark is a lie — a 'pristine' taken
|
||||||
|
# after 'rig bootstrap' is a converged box wearing the wrong label, and
|
||||||
|
# nothing at runtime would ever say so. So the position is pinned by line
|
||||||
|
# order, the way the other mint-path guards are (a daemon-free run cannot
|
||||||
|
# mint), and the policy half is DRIVEN against a stubbed incus.
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# shellcheck disable=SC2016 # the $-strings are literals in the target file
|
||||||
|
check "pristine: the mark is taken in the fresh-mint branch" 0 "" bash -c '
|
||||||
|
awk "/^cmd_new\(\) \{/,/^\}/" "'"$ROOT"'/bin/box" | grep -q "snapshot_pristine \"\$instance\""'
|
||||||
|
# AFTER cloud-init: before it, the guest is mid-install and the mark is not
|
||||||
|
# pristine Debian, it is a half-provisioned one.
|
||||||
|
# shellcheck disable=SC2016 # the $-strings are literals in the target file
|
||||||
|
check "pristine: the mark orders AFTER the cloud-init wait" 0 "" bash -c '
|
||||||
|
fn="$(awk "/^cmd_new\(\) \{/,/^\}/" "'"$ROOT"'/bin/box")"
|
||||||
|
wait="$(printf "%s\n" "$fn" | grep -n "cloud-init status --wait" | head -1 | cut -d: -f1)"
|
||||||
|
snap="$(printf "%s\n" "$fn" | grep -n "snapshot_pristine " | head -1 | cut -d: -f1)"
|
||||||
|
[ -n "$wait" ] && [ -n "$snap" ] && [ "$wait" -lt "$snap" ]'
|
||||||
|
# BEFORE the rig hook: this is the assertion the whole issue rests on.
|
||||||
|
# shellcheck disable=SC2016 # the $-strings are literals in the target file
|
||||||
|
check "pristine: the mark orders BEFORE the rig bootstrap hook (the moment)" 0 "" bash -c '
|
||||||
|
fn="$(awk "/^cmd_new\(\) \{/,/^\}/" "'"$ROOT"'/bin/box")"
|
||||||
|
snap="$(printf "%s\n" "$fn" | grep -n "snapshot_pristine " | head -1 | cut -d: -f1)"
|
||||||
|
run="$(printf "%s\n" "$fn" | grep -n "rig bootstrap \$T_BOOTSTRAP_ROLE" | head -1 | cut -d: -f1)"
|
||||||
|
[ -n "$snap" ] && [ -n "$run" ] && [ "$snap" -lt "$run" ]'
|
||||||
|
# NOT under the T_BOOTSTRAP_ROLE guard: a blank box has no rig hook but has
|
||||||
|
# the same pristine moment, and "box restore <box> pristine" must mean one
|
||||||
|
# thing on every box box mints.
|
||||||
|
# shellcheck disable=SC2016 # the $-strings are literals in the target file
|
||||||
|
check "pristine: the mark is unconditional, not gated on a tenant role" 0 "" bash -c '
|
||||||
|
fn="$(awk "/^cmd_new\(\) \{/,/^\}/" "'"$ROOT"'/bin/box")"
|
||||||
|
snap="$(printf "%s\n" "$fn" | grep -n "snapshot_pristine " | head -1 | cut -d: -f1)"
|
||||||
|
guard="$(printf "%s\n" "$fn" | grep -n "if \[ -n \"\$T_BOOTSTRAP_ROLE\" \]" | head -1 | cut -d: -f1)"
|
||||||
|
[ -n "$snap" ] && [ -n "$guard" ] && [ "$snap" -lt "$guard" ]'
|
||||||
|
|
||||||
|
# THE CLONE TRAP. --from skips cloud-init and the rig hook entirely, so the
|
||||||
|
# pristine moment never happens on that path. A mark taken there would be
|
||||||
|
# "whatever the source was" — converged, worked-in — wearing a label that
|
||||||
|
# promises pristine Debian, which is strictly worse than no mark. Pin the
|
||||||
|
# ABSENCE: extract the clone branch alone (up to its 'else') and assert
|
||||||
|
# nothing in it takes the mark.
|
||||||
|
CLONEBR="$(mktemp)"
|
||||||
|
awk '/if \[ -n "\$from" \]; then/,/^ else$/' "$ROOT/bin/box" > "$CLONEBR"
|
||||||
|
check "pristine: the clone branch extracted from bin/box (guards the awk)" 0 "incus copy" cat "$CLONEBR"
|
||||||
|
check "pristine: a --from clone takes NO mark of its own (the correctness trap)" 1 "" \
|
||||||
|
grep -q "snapshot_pristine" "$CLONEBR"
|
||||||
|
check "pristine: nothing on the clone path creates a snapshot at all" 1 "" \
|
||||||
|
grep -q "incus snapshot create" "$CLONEBR"
|
||||||
|
# Inheritance is the other half of the decision, and it must be SAID: a clone
|
||||||
|
# of a box carries the source's snapshots (a real pristine among them), a
|
||||||
|
# clone of a snapshot carries none. Silence there sends the operator to
|
||||||
|
# 'box info' to find out which world they are in.
|
||||||
|
check "pristine: the clone narrates whether a pristine rode along" 0 "" \
|
||||||
|
grep -q "no 'pristine' mark here" "$CLONEBR"
|
||||||
|
# ...and it reads the snapshot list CAPTURE-FIRST (#124's class). Piping a
|
||||||
|
# multi-line incus writer into an early-exit reader lets the reader close the
|
||||||
|
# pipe, SIGPIPE incus, and hand pipefail a 141 — which on THIS line reads as
|
||||||
|
# "no pristine" and narrates the wrong inheritance shape on a clone that has
|
||||||
|
# one. Pin the shape, not the instance spelling: no 'incus snapshot list'
|
||||||
|
# feeding grep/head/sed/awk/read directly.
|
||||||
|
check "pristine: the clone's inheritance read is capture-first, not a piped early-exit reader" 1 "" \
|
||||||
|
grep -Eq 'incus snapshot list[^|]*\| *(grep|head|sed|awk|read)' "$CLONEBR"
|
||||||
|
rm -f "$CLONEBR"
|
||||||
|
|
||||||
|
# The policy half, DRIVEN not grepped: extract storage_driver +
|
||||||
|
# snapshot_pristine and run them against a stubbed incus, so every branch is
|
||||||
|
# actually executed on a host with no daemon.
|
||||||
|
PRISFN="$(mktemp)"
|
||||||
|
awk '/^storage_driver\(\) \{/,/^\}/;/^snapshot_pristine\(\) \{/,/^\}/' "$ROOT/bin/box" > "$PRISFN"
|
||||||
|
check "pristine: the functions extracted from bin/box (guards the awk)" 0 "BOX_SNAPSHOT_PRISTINE" cat "$PRISFN"
|
||||||
|
check "pristine: the extracted functions are valid bash" 0 "" bash -n "$PRISFN"
|
||||||
|
|
||||||
|
# storage_driver's probes must survive a REFUSAL, and not by accident. Today
|
||||||
|
# command substitution strips errexit, so a failing probe falls through to the
|
||||||
|
# fallback; add 'shopt -s inherit_errexit' to bin/box — the robustness tweak
|
||||||
|
# #107 describes sailing through review — and under pipefail that same refusal
|
||||||
|
# becomes a fatal abort mid-mint, inside the function whose contract is NEVER
|
||||||
|
# fatal. Drive it with inherit_errexit ON and a tier that refuses both probes:
|
||||||
|
# the function must return empty (the unreadable-pool case) and the caller
|
||||||
|
# must still be alive afterwards.
|
||||||
|
driver_under_inherit_errexit() {
|
||||||
|
# shellcheck disable=SC2016 # the body is the stub's source, expanded by the
|
||||||
|
# inner bash, never by this shell.
|
||||||
|
env PRISFN="$PRISFN" bash -c '
|
||||||
|
set -euo pipefail
|
||||||
|
shopt -s inherit_errexit
|
||||||
|
incus() {
|
||||||
|
case "$*" in
|
||||||
|
"profile device get box-net root pool") printf "boxpool\n" ;;
|
||||||
|
*) printf "incus: not authorized\n" >&2; return 1 ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
. "$PRISFN"
|
||||||
|
d="$(storage_driver)"
|
||||||
|
printf "SURVIVED driver=[%s]\n" "$d"
|
||||||
|
' 2>&1
|
||||||
|
}
|
||||||
|
check "pristine: a refused storage probe is an answer, not a fatal (survives inherit_errexit)" 0 "SURVIVED driver=[]" \
|
||||||
|
driver_under_inherit_errexit
|
||||||
|
|
||||||
|
# pris <driver> [env...] — drive snapshot_pristine against a fake pool of
|
||||||
|
# <driver>. 'none' makes both probes answer nothing (the unreadable-pool
|
||||||
|
# case). Every incus call the function can make is stubbed and echoed, so the
|
||||||
|
# assertions read the real control flow, not a mock's opinion of it.
|
||||||
|
pris() { # pris <driver> <instance> [VAR=VAL...]
|
||||||
|
local driver="$1" instance="$2"; shift 2
|
||||||
|
# shellcheck disable=SC2016 # the body is the stub's source, expanded by the
|
||||||
|
# inner bash from the environment 'env' sets up — never by this shell.
|
||||||
|
env "$@" DRIVER="$driver" INSTANCE="$instance" PRISFN="$PRISFN" bash -c '
|
||||||
|
incus() {
|
||||||
|
case "$*" in
|
||||||
|
"profile device get box-net root pool") printf "boxpool\n" ;;
|
||||||
|
"storage show boxpool")
|
||||||
|
[ "$DRIVER" = none ] && return 1
|
||||||
|
printf "name: boxpool\ndriver: %s\n" "$DRIVER" ;;
|
||||||
|
"storage list --format csv")
|
||||||
|
[ "$DRIVER" = none ] && return 1
|
||||||
|
printf "boxpool,%s,,0,CREATED\n" "$DRIVER" ;;
|
||||||
|
"snapshot create inst-x pristine") printf "STUB: snapshot created\n" ;;
|
||||||
|
"snapshot create fail-x pristine") printf "STUB: incus refused\n" >&2; return 1 ;;
|
||||||
|
*) printf "STUB: unexpected incus call: %s\n" "$*" >&2; return 1 ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
. "$PRISFN"
|
||||||
|
snapshot_pristine "$INSTANCE" boxname
|
||||||
|
' 2>&1
|
||||||
|
}
|
||||||
|
# The absence assertions need a command 'check' can run, not a pipeline.
|
||||||
|
pris_took_mark() { pris "$@" | grep -q "STUB: snapshot created"; }
|
||||||
|
check "pristine: btrfs (the designed backend) takes the mark" 0 "STUB: snapshot created" \
|
||||||
|
pris btrfs inst-x
|
||||||
|
check "pristine: btrfs names the restore command for the operator" 0 "box restore boxname pristine" \
|
||||||
|
pris btrfs inst-x
|
||||||
|
# The 'dir' fallback (host/setup-host.sh:294) has no CoW: the mark would be a
|
||||||
|
# full multi-GB copy of the root on EVERY mint. Skip — and LOUDLY, naming the
|
||||||
|
# by-hand command, because a silent skip teaches an operator to expect a mark
|
||||||
|
# that is not there.
|
||||||
|
check "pristine: a 'dir' pool SKIPS the mark (no CoW — it would be a full copy)" 0 "NOT taking" \
|
||||||
|
pris dir inst-x
|
||||||
|
check "pristine: the dir skip is loud and names the by-hand command" 0 "box snapshot boxname pristine" \
|
||||||
|
pris dir inst-x
|
||||||
|
check "pristine: the dir skip never reaches incus snapshot create" 1 "" \
|
||||||
|
pris_took_mark dir inst-x
|
||||||
|
# Neither probe answers (an unusual host, or a tier that cannot read the
|
||||||
|
# pool). The two mistakes are not symmetric — a mark taken on 'dir' wastes
|
||||||
|
# disk the operator can see and delete, a mark NOT taken is the moment gone
|
||||||
|
# for good. So proceed, and say what was assumed.
|
||||||
|
check "pristine: an unreadable pool takes the mark anyway (the asymmetry)" 0 "STUB: snapshot created" \
|
||||||
|
pris none inst-x
|
||||||
|
check "pristine: ...and says what it assumed rather than pretending it knew" 0 "could not read the storage driver" \
|
||||||
|
pris none inst-x
|
||||||
|
# The escape hatch is an environment knob (the BOX_LAUNCH_TIMEOUT shape), not
|
||||||
|
# another flag on 'new'.
|
||||||
|
check "pristine: BOX_SNAPSHOT_PRISTINE=0 skips it anywhere" 0 "BOX_SNAPSHOT_PRISTINE=0" \
|
||||||
|
pris btrfs inst-x BOX_SNAPSHOT_PRISTINE=0
|
||||||
|
check "pristine: the opt-out never reaches incus snapshot create" 1 "" \
|
||||||
|
pris_took_mark btrfs inst-x BOX_SNAPSHOT_PRISTINE=0
|
||||||
|
# A failed snapshot must NOT fail the mint. The mark is an undo, not the
|
||||||
|
# mint's product: a mint that worked must not be failed by a checkpoint that
|
||||||
|
# didn't.
|
||||||
|
check "pristine: a failed snapshot warns and returns 0 (never fails a good mint)" 0 "WARNING" \
|
||||||
|
pris btrfs fail-x
|
||||||
|
rm -f "$PRISFN"
|
||||||
|
|
||||||
|
# The durability caveat, pinned in the help text: a snapshot dies with its
|
||||||
|
# box, so nothing box says may let anyone read 'pristine' as a backup (#104's
|
||||||
|
# closing note; 'box export' is the durable path).
|
||||||
|
check "pristine: 'box help snapshot' refuses to sell snapshots as backups" 0 "not a backup" \
|
||||||
|
bash -c '"'"$ROOT"'/bin/box" help snapshot'
|
||||||
|
check "pristine: 'box help restore' documents the mark and its off-box blind spot" 0 "off-box" \
|
||||||
|
bash -c '"'"$ROOT"'/bin/box" help restore'
|
||||||
|
check "pristine: 'box help new' documents the mark and the opt-out" 0 "BOX_SNAPSHOT_PRISTINE" \
|
||||||
|
bash -c '"'"$ROOT"'/bin/box" help new'
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# The restricted tier (#74). box_tier() is the decision the whole tier hangs
|
# The restricted tier (#74). box_tier() is the decision the whole tier hangs
|
||||||
# on, so it is DRIVEN, not grepped: extracted from bin/box, sourced, and run
|
# on, so it is DRIVEN, not grepped: extracted from bin/box, sourced, and run
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue