setup-host takes 'dir' storage — every clone is a full disk copy, and cloning is the whole point #29

New issue

Closed

opened 2026-07-13 23:43:44 +00:00 by dan-claude-bot · 0 comments

dan-claude-bot commented

2026-07-13 23:43:44 +00:00

(Migrated from github.com)

host/setup-host.sh bootstraps the storage pool with:

incus storage show default >/dev/null 2>&1 || incus admin init --minimal

--minimal picks the dir backend. Confirmed on a live host: driver=dir pool=default.

dir has no copy-on-write. So incus copy — which is what claudebox new --from runs — physically copies the VM's root.img: several GB once Docker, Node and Claude Code are installed. On the drill host a single clone took minutes, and every clone pays the same cost again, in both time and disk.

Why this cuts at the heart of the tool

From the README, the reuse story is the product:

log in once, snapshot, clone forever

claudebox snapshot work authed
claudebox new --name feature --from work/authed

On a CoW backend (btrfs, zfs, lvm-thin), that clone is near-instant and near-free: it shares blocks with its source and only diverges on write. On dir it is a full disk copy every single time. The feature still works — it is just slow enough and expensive enough that "clone forever" stops being an attractive thing to do, which is the one workflow claudebox exists to make attractive.

The disk cost compounds: N clones of one snapshot = N full copies on dir, ≈ 1 copy plus deltas on CoW.

Proposal

setup-host.sh should create the pool deliberately rather than take --minimal's default:

btrfs if the tooling is present (or installable) — the safest default: works on a file-backed loop device, no kernel module beyond mainline, instant snapshots.
zfs if it is already set up on the host — best-in-class, but a bigger ask (DKMS module, licensing friction on some distros).
dir only as an explicit fallback, and when it is chosen, say so: print that clones will be full copies, because the user's mental model of the tool ("cloning is cheap") is now wrong and nothing tells them.

Sketch:

if ! incus storage show default >/dev/null 2>&1; then
  if command -v mkfs.btrfs >/dev/null || sudo apt-get install -y btrfs-progs; then
    incus storage create default btrfs size=100GiB      # loop-backed; CoW clones
  else
    incus storage create default dir
    echo "NOTE: 'dir' storage — clones are FULL disk copies (minutes, GBs each)." >&2
    echo "      install btrfs-progs and recreate the pool for instant, cheap clones." >&2
  fi
  incus profile device add default root disk pool=default path=/
fi

(Exact wiring needs care: incus admin init --minimal also sets up the default profile's root device, so replacing it means doing that explicitly. Sizing the loop file is a judgement call — the VM root device is pinned at 60GiB in bin/claudebox, though sparse.)

Migration

Existing hosts are on dir and cannot be converted in place. Either leave them (it works, it is just slow) or document the rebuild: teardown-host.sh → recreate the pool → re-mint. Not urgent; this is a performance and disk-cost bug, not a correctness one.

Provenance

Found while running the drill for the #15 audit — the clone step sat for minutes and the storage driver explained why. Same host, same session, that surfaced the daemon wedge (#26) and the clone-identity collision (#27).

`host/setup-host.sh` bootstraps the storage pool with: ```sh incus storage show default >/dev/null 2>&1 || incus admin init --minimal ``` `--minimal` picks the **`dir`** backend. Confirmed on a live host: `driver=dir pool=default`. `dir` has **no copy-on-write**. So `incus copy` — which is what `claudebox new --from` runs — **physically copies the VM's `root.img`**: several GB once Docker, Node and Claude Code are installed. On the drill host a single clone took **minutes**, and every clone pays the same cost again, in both time and disk. ## Why this cuts at the heart of the tool From the README, the reuse story *is* the product: > log in once, snapshot, clone forever - `claudebox snapshot work authed` - `claudebox new --name feature --from work/authed` On a CoW backend (btrfs, zfs, lvm-thin), that clone is **near-instant and near-free**: it shares blocks with its source and only diverges on write. On `dir` it is a **full disk copy every single time**. The feature still works — it is just slow enough and expensive enough that "clone forever" stops being an attractive thing to do, which is the one workflow claudebox exists to make attractive. The disk cost compounds: N clones of one snapshot = N full copies on `dir`, ≈ 1 copy plus deltas on CoW. ## Proposal `setup-host.sh` should create the pool deliberately rather than take `--minimal`'s default: 1. **btrfs** if the tooling is present (or installable) — the safest default: works on a file-backed loop device, no kernel module beyond mainline, instant snapshots. 2. **zfs** if it is already set up on the host — best-in-class, but a bigger ask (DKMS module, licensing friction on some distros). 3. **`dir` only as an explicit fallback**, and when it is chosen, **say so**: print that clones will be full copies, because the user's mental model of the tool ("cloning is cheap") is now wrong and nothing tells them. Sketch: ```sh if ! incus storage show default >/dev/null 2>&1; then if command -v mkfs.btrfs >/dev/null || sudo apt-get install -y btrfs-progs; then incus storage create default btrfs size=100GiB # loop-backed; CoW clones else incus storage create default dir echo "NOTE: 'dir' storage — clones are FULL disk copies (minutes, GBs each)." >&2 echo " install btrfs-progs and recreate the pool for instant, cheap clones." >&2 fi incus profile device add default root disk pool=default path=/ fi ``` (Exact wiring needs care: `incus admin init --minimal` also sets up the default profile's root device, so replacing it means doing that explicitly. Sizing the loop file is a judgement call — the VM root device is pinned at **60GiB** in `bin/claudebox`, though sparse.) ## Migration Existing hosts are on `dir` and cannot be converted in place. Either leave them (it works, it is just slow) or document the rebuild: `teardown-host.sh` → recreate the pool → re-mint. Not urgent; this is a **performance and disk-cost** bug, not a correctness one. ## Provenance Found while running the drill for the [#15](https://github.com/heavy-duty/claudebox/issues/15) audit — the clone step sat for minutes and the storage driver explained why. Same host, same session, that surfaced the daemon wedge ([#26](https://github.com/heavy-duty/claudebox/issues/26)) and the clone-identity collision ([#27](https://github.com/heavy-duty/claudebox/pull/27)).

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: heavy-duty/box#29

No description provided.