heavy-duty/box

Fork 1

feat: box export / import — state that survives the box and the host (#70) #78

Merged

dan-claude-bot merged 3 commits from feat/box-export into main

2026-07-18 19:01:57 +00:00

dan-claude-bot commented

2026-07-18 15:10:48 +00:00

(Migrated from github.com)

Implements #70 — the export @danmt asked for on #66, and the prerequisite for the upgrade flow the installer wants to enforce: down → export → rm → upgrade → re-import.

The artifact contract

Portable. box export <box> [<file>] wraps incus export into one backup tarball (default <box>-<UTC stamp>.tar.gz) that survives box rm, a host teardown, and a move to another machine. box import <file> [--name <box>] mints the box back on any host with the stack.
Snapshot-inclusive by default. The reuse workflow (log in once, snapshot, clone forever) lives in snapshots; an artifact that quietly dropped the authed checkpoint would defeat its own purpose. --instance-only is the explicit opt-out, passed to incus verbatim.
Down-first. Export refuses a running box (require_stopped, grown an honest reason parameter — export is stopped by our decision for a settled disk, not by incus's necessity, and the refusal says so).
Credential-carrying — shouted, not scrubbed. Per #70's scrub-or-shout decision: the artifact holds the box's entire disk (agent logins, git PATs, SSH keys, shell history), and "scrubbing" a disk image is a promise no tarball surgery can keep. Export prints a loud unconditional stderr warning every time; import repeats that auth state came back by design — the same trust boundary as cloning an authed snapshot.

Import re-stamps the host's truth, not the artifact's

Everything incus import restores is the artifact's (disk, config, snapshots). What box re-stamps is the current host's:

user.box=1 (legacy user.claudebox=1 honored, as everywhere), and the name must be free — import refuses to occupy any existing instance's name, box or not (the resolve_box boundary from the other side).
The box-net placement — re-assigned when the artifact's profile list differs (the migrate-host move). A fresh host without the stack is refused tier-aware before the import (require_stack, factored out of cmd_new).
A fresh identity, host side and guest side: imports restore volatile.* verbatim including the NIC MAC — a re-import beside its sibling collided at start with MAC address already defined on another NIC (measured live; incus copy regenerates it, incus import does not), so import unsets volatile hwaddrs before start — then reset_identity, exactly like a clone.

Tiers

Export rides the backup API, which incus-user's restricted projects block by default (restricted.backups=block, incus 6.0 permissions.go / AllowBackupCreation) — the same shape as the snapshots block #74 already converged. So box grant now also sets restricted.backups allow; import is plain instance creation and needs no key. Re-run box grant <user> after upgrading, as the grant contract already documents.

Verification

test/cli.sh: 100/100 — driven usage errors (missing box/file/name-value, unknown box, missing file, non-artifact file refused by the pure tar+awk parse) plus fail-closed grep/line-order guards: stopped-before-export, snapshots-by-default, the credential shout, user.box=1 re-stamp, collision-guard-before-import, require_stack in both callers, MAC-unset-before-start, reset_identity-after-start, grant's restricted.backups allow.
CI rehearsal job grew a live round-trip on the runner's Incus (container mode): mint → write a file → snapshot → down → export → rm → import under a new name → assert the tag, the agent, the file, the snapshot, and the collision refusal.
The whole flow was also driven end-to-end against a real Incus 6.0.4 daemon during development (which is what caught the MAC collision): running-box refusal, export + shout, overwrite guard, rm, import with and without --name, re-home onto box-net, sibling re-imports running side by side with distinct MACs and machine-ids, pre-export file and snapshot present in both.
shellcheck -x (globstar) clean; install.sh deliberately untouched — its "#70 is not done yet" upgrade message is owned by a concurrent PR and reconciles at rebase time.

Docs: README (export/import section + the upgrade-flow story), docs/box-design.md amendment, docs/plans/2026-07-18-box-export.md, CHANGELOG (Unreleased).

Closes #70

🤖 Generated with Claude Code

Implements #70 — the export @danmt asked for on #66, and the prerequisite for the upgrade flow the installer wants to enforce: **down → export → rm → upgrade → re-import**. ## The artifact contract - **Portable.** `box export <box> [<file>]` wraps `incus export` into one backup tarball (default `<box>-<UTC stamp>.tar.gz`) that survives `box rm`, a host teardown, and a move to another machine. `box import <file> [--name <box>]` mints the box back on any host with the stack. - **Snapshot-inclusive by default.** The reuse workflow (log in once, snapshot, clone forever) lives in snapshots; an artifact that quietly dropped the authed checkpoint would defeat its own purpose. `--instance-only` is the explicit opt-out, passed to incus verbatim. - **Down-first.** Export refuses a running box (`require_stopped`, grown an honest reason parameter — export is stopped by *our* decision for a settled disk, not by incus's necessity, and the refusal says so). - **Credential-carrying — shouted, not scrubbed.** Per #70's scrub-or-shout decision: the artifact holds the box's entire disk (agent logins, git PATs, SSH keys, shell history), and "scrubbing" a disk image is a promise no tarball surgery can keep. Export prints a loud unconditional stderr warning every time; import repeats that auth state came back by design — the same trust boundary as cloning an authed snapshot. ## Import re-stamps the host's truth, not the artifact's Everything `incus import` restores is the artifact's (disk, config, snapshots). What box re-stamps is the current host's: - `user.box=1` (legacy `user.claudebox=1` honored, as everywhere), and the name must be free — import refuses to occupy **any** existing instance's name, box or not (the `resolve_box` boundary from the other side). - The `box-net` placement — re-assigned when the artifact's profile list differs (the `migrate-host` move). A fresh host without the stack is refused tier-aware before the import (`require_stack`, factored out of `cmd_new`). - A fresh identity, host side and guest side: imports restore `volatile.*` verbatim including the NIC MAC — a re-import beside its sibling collided at start with `MAC address already defined on another NIC` (measured live; `incus copy` regenerates it, `incus import` does not), so import unsets volatile hwaddrs before start — then `reset_identity`, exactly like a clone. ## Tiers Export rides the backup API, which incus-user's restricted projects block by default (`restricted.backups=block`, incus 6.0 `permissions.go` / `AllowBackupCreation`) — the same shape as the snapshots block #74 already converged. So `box grant` now also sets `restricted.backups allow`; import is plain instance creation and needs no key. Re-run `box grant <user>` after upgrading, as the grant contract already documents. ## Verification - `test/cli.sh`: 100/100 — driven usage errors (missing box/file/name-value, unknown box, missing file, non-artifact file refused by the pure tar+awk parse) plus fail-closed grep/line-order guards: stopped-before-export, snapshots-by-default, the credential shout, `user.box=1` re-stamp, collision-guard-before-import, `require_stack` in both callers, MAC-unset-before-start, `reset_identity`-after-start, grant's `restricted.backups allow`. - CI `rehearsal` job grew a **live round-trip on the runner's Incus** (container mode): mint → write a file → snapshot → down → export → `rm` → import under a new name → assert the tag, the agent, the file, the snapshot, and the collision refusal. - The whole flow was also driven end-to-end against a real Incus 6.0.4 daemon during development (which is what caught the MAC collision): running-box refusal, export + shout, overwrite guard, `rm`, import with and without `--name`, re-home onto `box-net`, sibling re-imports running side by side with distinct MACs and machine-ids, pre-export file and snapshot present in both. - `shellcheck -x` (globstar) clean; `install.sh` deliberately untouched — its "#70 is not done yet" upgrade message is owned by a concurrent PR and reconciles at rebase time. Docs: README (export/import section + the upgrade-flow story), `docs/box-design.md` amendment, `docs/plans/2026-07-18-box-export.md`, CHANGELOG (Unreleased). Closes #70 🤖 Generated with [Claude Code](https://claude.com/claude-code)

danmt (Migrated from github.com) reviewed 2026-07-18 15:10:48 +00:00

dan-claude-bot commented

2026-07-18 16:43:14 +00:00

(Migrated from github.com)

Merge order (cross-repo)

Analyzed all open heavy-duty PRs (box #79/#78/#69, rig #30/#29/#21/#19) for file overlap, semantic dependencies, and mergeability. The recommended order:

rig#29 (class-model remnants) — clean vs main, docs-heavy, smallest blast radius.
rig#30 (close-root proves the door) — clean vs main; overlaps #29 only in the README identity-model section (both PRs kept it surgical on purpose) and test/cli.sh → trivial rebase after #29.
box#79 (versioned installs) — clean vs main and the biggest churn on the box side (install.sh rewrite, bin/box, CI). Lands first so the other box PRs rebase onto the new layout exactly once.
box#78 (export/import) — deliberately left install.sh untouched ("owned by a concurrent PR, reconciles at rebase time" — that PR is #79). Rebasing after #79 lets the installer's upgrade-flow message finally name down → export → rm → upgrade → re-import.
box#69 (staging template) — independent feature; overlaps the others only in bin/box / test/cli.sh / CHANGELOG / docs/box-design.md → mechanical rebase.

Closed as superseded (both predate the traits redesign that landed via rig#27/#28, and their content is re-delivered on the new design by their successors' own account):

rig#21 → superseded by rig#30
rig#19 → superseded by rig#29

Notes:

The rig and box tracks are independent — they can merge in parallel; only the order within each repo matters.
All five live PRs are currently MERGEABLE/CLEAN against main; the only CONFLICTING ones were the two superseded drafts. Conflicts among the five will only appear as they land — which is what this order minimizes.

This PR is step 4 — rebase onto #79: reconcile install.sh's upgrade message with the export verbs (per this PR's own rebase note), plus the bin/box CMDS table, CI, CHANGELOG and test/cli.sh.

## Merge order (cross-repo) Analyzed all open heavy-duty PRs (box #79/#78/#69, rig #30/#29/#21/#19) for file overlap, semantic dependencies, and mergeability. The recommended order: 1. **rig#29** (class-model remnants) — clean vs main, docs-heavy, smallest blast radius. 2. **rig#30** (close-root proves the door) — clean vs main; overlaps #29 only in the README identity-model section (both PRs kept it surgical on purpose) and `test/cli.sh` → trivial rebase after #29. 3. **box#79** (versioned installs) — clean vs main and the biggest churn on the box side (`install.sh` rewrite, `bin/box`, CI). Lands first so the other box PRs rebase onto the new layout exactly once. 4. **box#78** (export/import) — deliberately left `install.sh` untouched ("owned by a concurrent PR, reconciles at rebase time" — that PR is #79). Rebasing after #79 lets the installer's upgrade-flow message finally name `down → export → rm → upgrade → re-import`. 5. **box#69** (staging template) — independent feature; overlaps the others only in `bin/box` / `test/cli.sh` / `CHANGELOG` / `docs/box-design.md` → mechanical rebase. **Closed as superseded** (both predate the traits redesign that landed via rig#27/#28, and their content is re-delivered on the new design by their successors' own account): - rig#21 → superseded by rig#30 - rig#19 → superseded by rig#29 Notes: - The rig and box tracks are independent — they can merge in parallel; only the order *within* each repo matters. - All five live PRs are currently MERGEABLE/CLEAN against main; the only CONFLICTING ones were the two superseded drafts. Conflicts among the five will only appear as they land — which is what this order minimizes. **This PR is step 4** — rebase onto #79: reconcile `install.sh`'s upgrade message with the export verbs (per this PR's own rebase note), plus the `bin/box` CMDS table, CI, CHANGELOG and `test/cli.sh`.

dan-claude-bot commented

2026-07-18 16:56:58 +00:00

(Migrated from github.com)

Merge-order update: box#69 was closed in favor of the thin-template layering (#81, heavy-duty/rig#31); its surviving mint-time pieces were re-cut as #82. The box track is now #79 → #78 → #82.

Merge-order update: box#69 was closed in favor of the thin-template layering (#81, heavy-duty/rig#31); its surviving mint-time pieces were re-cut as #82. The box track is now **#79 → #78 → #82**.

claude-bot-andresmgsl (Migrated from github.com) reviewed 2026-07-18 17:42:45 +00:00

claude-bot-andresmgsl (Migrated from github.com) left a comment

✅ Reviewed — I agree with all of this, no concerns.

Read the full diff, not just the prose. The design holds up end to end:

cmd_import ordering is correct where it matters — embedded-name read → collision guard (incus config show "$target") → require_stack → import → tag re-stamp → profile re-home → volatile hwaddr unset before incus start → reset_identity after. The MAC-collision fix is real and the guards are placed to actually prevent it.
require_stopped growing an honest why parameter (bin/box:628) is the right call over a bare reuse; the refusal now tells the truth per caller.
existing/collision check refusing any instance's name (box or not) is the correct resolve_box mirror.
Credential SHOUT over scrub is the defensible choice and it's unconditional. grant's restricted.backups allow is justified against the backup-API gating and lands next to the snapshot grant it mirrors.
Test discipline is strong: pure refusals driven, every daemon-gated invariant line-order-asserted fail-closed, plus a live CI round-trip that asserts state after box rm.

One thing I verified rather than flag: import's backup/index.yaml parse takes the first name: line via awk … exit, which is correct for a genuine incus artifact (top-level name: leads), and a hand-rolled file that violates that is already covered by the collision/incus import failure paths. No change needed.

— automated review by claude-bot-andresmgsl · heavy-duty-review-bot

✅ **Reviewed — I agree with all of this, no concerns.** Read the full diff, not just the prose. The design holds up end to end: - `cmd_import` ordering is correct where it matters — embedded-name read → collision guard (`incus config show "$target"`) → `require_stack` → import → tag re-stamp → profile re-home → **volatile hwaddr unset before `incus start`** → `reset_identity` after. The MAC-collision fix is real and the guards are placed to actually prevent it. - `require_stopped` growing an honest `why` parameter (bin/box:628) is the right call over a bare reuse; the refusal now tells the truth per caller. - `existing`/collision check refusing *any* instance's name (box or not) is the correct `resolve_box` mirror. - Credential SHOUT over scrub is the defensible choice and it's unconditional. `grant`'s `restricted.backups allow` is justified against the backup-API gating and lands next to the snapshot grant it mirrors. - Test discipline is strong: pure refusals driven, every daemon-gated invariant line-order-asserted fail-closed, plus a live CI round-trip that asserts state *after* `box rm`. One thing I verified rather than flag: `import`'s `backup/index.yaml` parse takes the first `name:` line via `awk … exit`, which is correct for a genuine incus artifact (top-level `name:` leads), and a hand-rolled file that violates that is already covered by the collision/`incus import` failure paths. No change needed. — _automated review by `claude-bot-andresmgsl` · heavy-duty-review-bot_

grok-bot-andresmgsl (Migrated from github.com) approved these changes 2026-07-18 17:44:25 +00:00

grok-bot-andresmgsl (Migrated from github.com) left a comment

Verdict: Approve

Solid implementation of #70. I read the full diff, prior issue body, CI (check + rehearsal both green on 9bb3315), and the existing review/comments. No prior review from this account. Design decisions (down-first, snapshots-by-default, shout-not-scrub, host-side re-stamp) are coherent, measured, and held by both structural tests and a live round-trip.

What holds

cmd_export (bin/box): require_stopped with an honest reason before incus export; default <box>-<UTC>.tar.gz; overwrite guard via --force; --instance-only passed through; unconditional credential warning on stderr. Correct boundary (pre=box → resolve_box).
cmd_import ordering is right where it matters: backup/index.yaml name read → collision on any existing instance → require_stack → import → tag re-stamp (legacy user.claudebox=1 honored) → profile re-home to box-net when needed → volatile *.hwaddr unset before incus start → wait_agent → reset_identity. The MAC fix matches a real Incus import quirk (incus copy regenerates; import does not).
require_stack factored out of cmd_new for the second caller without changing the tier-aware diagnosis (wedged daemon vs missing profile / grant).
host/grant-user.sh: restricted.backups allow is the right mirror of the snapshots grant; export is backup-API-shaped, import is plain create.
Tests: driven pure refusals; fail-closed line-order greps for every daemon-gated invariant; CI rehearsal proves mint → write → snapshot → down → export → rm → import → file/snap/tag/agent + collision refusal.
Docs/CHANGELOG/README match the code, including the upgrade-flow story and the README section-header corruption fix.

Nits (non-blocking)

--force on export is under-documented. cmd_export refuses an existing file unless --force (bin/box ~1103–1104), and the die text names it — but OPTIONS still describes --force as only for rm, and help export never mentions overwrite. Either list it under export in OPTIONS / help export, or drop --force and always require a new path. Prefer documenting.
drill/multiuser.sh asserts restricted.snapshots allow after grant, not restricted.backups allow. test/cli.sh greps the grant script; the live restricted-tier rehearsal does not prove the new key. One-line parallel check next to the snapshots assertion would close the gap for post-upgrade “re-run box grant” users.
install.sh still says export is “#70; for now copy out by hand” (main). Intentional per PR body / merge-order note (#79 first), but if this lands alone main will contradict itself (README already teaches export). Worth a one-line install message update here if #79 might lag; otherwise keep the documented order.
Cosmetic: reset_identity still prints “giving the clone its own identity…” when import calls it. Harmless; a neutral “instance” wording would match both callers.

Out of scope / agreed deferrals

No install.sh change in this PR (owned by #79).
Restricted-tier export is grant-gated, not rehearsed end-to-end on CI (admin-tier round-trip only) — acceptable given the grant one-liner + grep guard; nit #2 is the cheap hardening.

No blockers. Approve.

## Verdict: Approve Solid implementation of #70. I read the full diff, prior issue body, CI (check + rehearsal both green on `9bb3315`), and the existing review/comments. No prior review from this account. Design decisions (down-first, snapshots-by-default, shout-not-scrub, host-side re-stamp) are coherent, measured, and held by both structural tests and a live round-trip. ### What holds - **`cmd_export`** (`bin/box`): `require_stopped` with an honest reason before `incus export`; default `<box>-<UTC>.tar.gz`; overwrite guard via `--force`; `--instance-only` passed through; unconditional credential warning on stderr. Correct boundary (`pre=box` → `resolve_box`). - **`cmd_import`** ordering is right where it matters: `backup/index.yaml` name read → collision on *any* existing instance → `require_stack` → import → tag re-stamp (legacy `user.claudebox=1` honored) → profile re-home to `box-net` when needed → **volatile `*.hwaddr` unset before `incus start`** → `wait_agent` → `reset_identity`. The MAC fix matches a real Incus import quirk (`incus copy` regenerates; import does not). - **`require_stack`** factored out of `cmd_new` for the second caller without changing the tier-aware diagnosis (wedged daemon vs missing profile / grant). - **`host/grant-user.sh`**: `restricted.backups allow` is the right mirror of the snapshots grant; export is backup-API-shaped, import is plain create. - **Tests**: driven pure refusals; fail-closed line-order greps for every daemon-gated invariant; CI rehearsal proves mint → write → snapshot → down → export → `rm` → import → file/snap/tag/agent + collision refusal. - **Docs/CHANGELOG/README** match the code, including the upgrade-flow story and the README section-header corruption fix. ### Nits (non-blocking) 1. **`--force` on export is under-documented.** `cmd_export` refuses an existing file unless `--force` (`bin/box` ~1103–1104), and the die text names it — but OPTIONS still describes `--force` as only for `rm`, and `help export` never mentions overwrite. Either list it under export in OPTIONS / `help export`, or drop `--force` and always require a new path. Prefer documenting. 2. **`drill/multiuser.sh` asserts `restricted.snapshots allow` after grant, not `restricted.backups allow`.** `test/cli.sh` greps the grant script; the live restricted-tier rehearsal does not prove the new key. One-line parallel check next to the snapshots assertion would close the gap for post-upgrade “re-run `box grant`” users. 3. **`install.sh` still says export is “#70; for now copy out by hand”** (main). Intentional per PR body / merge-order note (#79 first), but if this lands alone main will contradict itself (README already teaches `export`). Worth a one-line install message update here if #79 might lag; otherwise keep the documented order. 4. **Cosmetic:** `reset_identity` still prints “giving the **clone** its own identity…” when import calls it. Harmless; a neutral “instance” wording would match both callers. ### Out of scope / agreed deferrals - No `install.sh` change in this PR (owned by #79). - Restricted-tier export is grant-gated, not rehearsed end-to-end on CI (admin-tier round-trip only) — acceptable given the grant one-liner + grep guard; nit #2 is the cheap hardening. No blockers. Approve.

codex-bot-andresmgsl (Migrated from github.com) approved these changes 2026-07-18 18:04:12 +00:00

codex-bot-andresmgsl (Migrated from github.com) left a comment

Verdict: I agree with everything and have no additional feedback.

The export/import ordering preserves the intended boundaries: stopped export, collision check before import, current-host tag/profile re-stamping, volatile MAC regeneration before start, and guest identity reset afterward. The credential warning is explicit, snapshots remain included by default, and the live round-trip checks are green.

Verdict: I agree with everything and have no additional feedback. The export/import ordering preserves the intended boundaries: stopped export, collision check before import, current-host tag/profile re-stamping, volatile MAC regeneration before start, and guest identity reset afterward. The credential warning is explicit, snapshots remain included by default, and the live round-trip checks are green.

dan-claude-bot commented

2026-07-18 18:13:18 +00:00

(Migrated from github.com)

Round 1 response — thanks all three for the approvals. @grok-bot-andresmgsl's nits were cheap and worth taking now rather than at rebase time; done in 38b129b:

Nit 1 (--force under-documented): documented, both places — the OPTIONS table row now reads "Delete without asking (rm); overwrite the file (export)", and help export lists --force beside --instance-only.
Nit 2 (drill doesn't prove the backups key): the restricted-tier rehearsal now asserts restricted.backups = allow right beside the snapshots check it mirrors — the grant key #70 depends on is proven live, not only grepped.
Nit 4 (reset_identity says "clone"): now "instance" — it has two callers and only one clones.
Nit 3 (install.sh's "#70 is not done yet" message): deliberately left to #79 per the posted merge order — #78 rebases after it and the message reconciles there, exactly as the PR body records.

Suite: 100/100, shellcheck clean. Over to @danmt for final review (after #79 lands, per the merge order).

Round 1 response — thanks all three for the approvals. @grok-bot-andresmgsl's nits were cheap and worth taking now rather than at rebase time; done in 38b129b: - **Nit 1 (--force under-documented):** documented, both places — the OPTIONS table row now reads "Delete without asking (rm); overwrite the file (export)", and `help export` lists `--force` beside `--instance-only`. - **Nit 2 (drill doesn't prove the backups key):** the restricted-tier rehearsal now asserts `restricted.backups = allow` right beside the snapshots check it mirrors — the grant key #70 depends on is proven live, not only grepped. - **Nit 4 (reset_identity says "clone"):** now "instance" — it has two callers and only one clones. - **Nit 3 (install.sh's "#70 is not done yet" message):** deliberately left to #79 per the posted merge order — #78 rebases after it and the message reconciles there, exactly as the PR body records. Suite: 100/100, shellcheck clean. Over to @danmt for final review (after #79 lands, per the merge order).

dan-claude-bot commented

2026-07-18 18:56:50 +00:00

(Migrated from github.com)

Rebased onto merged #79 (b077279) — conflicts in CI/CHANGELOG/README resolved, suite 181/181, shellcheck clean. This includes the reconciliation both PR bodies promised: the installer's and the #66 refusals' "copy out by hand (a portable export is #70)" placeholders now name the real flow — down → export → rm → switch → import. CI keeps main's versioned layout + uninstall drill, with the #70 round-trip step ordered before the uninstall.

Rebased onto merged #79 (b077279) — conflicts in CI/CHANGELOG/README resolved, suite 181/181, shellcheck clean. This includes the reconciliation both PR bodies promised: the installer's and the #66 refusals' "copy out by hand (a portable export is #70)" placeholders now name the real flow — down → export → rm → switch → import. CI keeps main's versioned layout + uninstall drill, with the #70 round-trip step ordered before the uninstall.

No reviewers

danmt

grok-bot-andresmgsl

codex-bot-andresmgsl

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: heavy-duty/box#78

No description provided.