Restricted incus tier: redesign around incus-user's per-user bridge (Task-0 findings) #74

New issue

Closed

opened 2026-07-18 00:02:49 +00:00 by dan-claude-bot · 0 comments

dan-claude-bot commented

2026-07-18 00:02:49 +00:00

(Migrated from github.com)

Context

Splitting the restricted-tier work out of #72 / PR #73: a Task-0 rehearsal on real hardware (Debian 13, Incus 6.0.4) vetoed #72's design. This issue is the redesign, built on what the rehearsal actually measured. The original #72 code is preserved on the branch dan-claude-bot:feat/restricted-tier-wip for reference.

What #72 assumed vs. what Incus 6.0.4 does

#72 assumed a restricted (incus-group, not incus-admin) user would share the daemon-global boxnet — that features.networks=false on their project would let them launch onto box's one hardened bridge. Measured reality:

✅ incus-user confinement works: an incus-group user is auto-confined to a restricted project user-<uid>, seeing only their own instances. The core tier idea is sound.
❌ but they cannot reach boxnet. incus-user gives each user a private auto-created bridge incusbr-<uid> and pins restricted.networks.access: incusbr-<uid>. As the user:
- incus network show boxnet → Error: Network not found
- incus launch --profile box-net / ensure_boxnet_profile → fails (the profile references a network they may not use)

So box's entire isolation stack lives on boxnet, which restricted users never touch — they'd land on a stock incusbr-<uid> with none of box's hardening (no box-isolate ACL, no dns.mode=none, no resolver pin, no security.port_isolation, no nft box-to-box drop). box new simply does not work for them as #72 was written.

The fix is a redesign (needs a decision)

An admin can bridge the two worlds, but not the way #72 assumed:

incus project set user-<uid> restricted.networks.access boxnet,incusbr-<uid> — must list both; boxnet alone conflicts with the auto default profile's eth0, still pinned to incusbr-<uid> (confirmed). Then install the box-net profile into that project (admin, per user).
Both are per-project admin actions, and the restricted project does not exist until the user first touches incus — so setup-host cannot pre-seed them.

Two directions to decide between:

Force everyone onto the shared boxnet — an admin convergence hook (box grant <user> / rig users apply doing it / an incus-user project-template config) that, per user, widens restricted.networks.access to include boxnet and installs the box-net profile. One hardened network for all; project confinement for management; existing box↔box isolation (nft drop + port_isolation + dns.mode=none) already prevents cross-user reachability.
Harden the per-user incusbr-<uid> — accept incus-user's per-user bridge and apply box's ACL/DNS/isolation to each one (admin, after lazy creation). More moving parts; the profile becomes per-user (references incusbr-<uid>, not a shipped static YAML).

Open sub-questions: can incus-user's project template be configured at the daemon level (avoiding a per-user hook)? Does widening restricted.networks.access survive an incus-user re-sync? Who owns the convergence — box (box grant), rig (users apply), or incus-user config?

Acceptance

A restricted incus-group user can box new/list/shell/snapshot/rm their own boxes, on a network carrying box's full isolation contract, seeing no one else's.
The convergence is owned by a documented, idempotent path (not manual per-user incus project set).
The drill/multiuser.sh rehearsal (on feat/restricted-tier-wip) passes criteria (a)–(f) on a real multi-user host.
Admin tier unchanged.

Environment for the rehearsal that produced this

Debian 13 (trixie), Incus 6.0.4 (6.0.4-2+deb13u8), incus-user.socket shipped/enabled/active, /dev/kvm + nested virt present. Full write-up in PR #73's plan doc (docs/plans/2026-07-17-multiuser-hosts.md, on feat/restricted-tier-wip) and this PR #73 comment.

Refines #72. Companion: rig#24 (box role), rig#12/#25 (host-class).

## Context Splitting the restricted-tier work out of #72 / [PR #73](https://github.com/heavy-duty/box/pull/73): a **Task-0 rehearsal on real hardware (Debian 13, Incus 6.0.4)** vetoed #72's design. This issue is the redesign, built on what the rehearsal actually measured. The original #72 code is preserved on the branch `dan-claude-bot:feat/restricted-tier-wip` for reference. ## What #72 assumed vs. what Incus 6.0.4 does #72 assumed a restricted (`incus`-group, not `incus-admin`) user would **share** the daemon-global `boxnet` — that `features.networks=false` on their project would let them launch onto box's one hardened bridge. Measured reality: - ✅ **incus-user confinement works**: an `incus`-group user is auto-confined to a restricted project `user-<uid>`, seeing only their own instances. The core tier idea is sound. - ❌ **but they cannot reach `boxnet`**. incus-user gives each user a **private auto-created bridge `incusbr-<uid>`** and pins `restricted.networks.access: incusbr-<uid>`. As the user: - `incus network show boxnet` → `Error: Network not found` - `incus launch --profile box-net` / `ensure_boxnet_profile` → **fails** (the profile references a network they may not use) So box's **entire isolation stack lives on `boxnet`, which restricted users never touch** — they'd land on a stock `incusbr-<uid>` with none of box's hardening (no `box-isolate` ACL, no `dns.mode=none`, no resolver pin, no `security.port_isolation`, no nft box-to-box drop). `box new` simply does not work for them as #72 was written. ## The fix is a redesign (needs a decision) An admin can bridge the two worlds, but not the way #72 assumed: - `incus project set user-<uid> restricted.networks.access boxnet,incusbr-<uid>` — must list **both**; `boxnet` alone conflicts with the auto default profile's `eth0`, still pinned to `incusbr-<uid>` (confirmed). Then install the `box-net` profile **into that project** (admin, per user). - Both are **per-project admin actions**, and the restricted project **does not exist until the user first touches `incus`** — so `setup-host` cannot pre-seed them. Two directions to decide between: 1. **Force everyone onto the shared `boxnet`** — an admin convergence hook (`box grant <user>` / rig `users apply` doing it / an incus-user project-template config) that, per user, widens `restricted.networks.access` to include `boxnet` and installs the `box-net` profile. One hardened network for all; project confinement for management; existing box↔box isolation (nft drop + port_isolation + `dns.mode=none`) already prevents cross-user reachability. 2. **Harden the per-user `incusbr-<uid>`** — accept incus-user's per-user bridge and apply box's ACL/DNS/isolation to each one (admin, after lazy creation). More moving parts; the profile becomes per-user (references `incusbr-<uid>`, not a shipped static YAML). Open sub-questions: can incus-user's project template be configured at the daemon level (avoiding a per-user hook)? Does widening `restricted.networks.access` survive an incus-user re-sync? Who owns the convergence — box (`box grant`), rig (`users apply`), or incus-user config? ## Acceptance - A restricted `incus`-group user can `box new/list/shell/snapshot/rm` **their own** boxes, on a network carrying box's full isolation contract, seeing no one else's. - The convergence is owned by a documented, idempotent path (not manual per-user `incus project set`). - The `drill/multiuser.sh` rehearsal (on `feat/restricted-tier-wip`) passes criteria (a)–(f) on a real multi-user host. - Admin tier unchanged. ## Environment for the rehearsal that produced this Debian 13 (trixie), Incus 6.0.4 (`6.0.4-2+deb13u8`), `incus-user.socket` shipped/enabled/active, `/dev/kvm` + nested virt present. Full write-up in PR #73's plan doc (`docs/plans/2026-07-17-multiuser-hosts.md`, on `feat/restricted-tier-wip`) and [this PR #73 comment](https://github.com/heavy-duty/box/pull/73#issuecomment-5008542760). Refines #72. Companion: rig#24 (`box` role), rig#12/#25 (host-class).

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: heavy-duty/box#74

No description provided.