fix(users): the host= marker gates the box role, not the incus group #60
Labels
No labels
attention
blocked
blocker:ci-red
blocker:conflict
blocker:drill-pending
blocker:unrequested
bug
claimed
documentation
enhancement
epic
merge-next
needs-ruling
needs-triage
offsite
post-merge
ready
release
scope:bootstrap
scope:coolify
scope:db
scope:docs
scope:drill
scope:installer
scope:labels
scope:platform
scope:runner
scope:users
stale
state:addressing
state:bots-reviewing
state:building
state:needs-human
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: heavy-duty/rig#60
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/host-no-bare-socket"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The gap
commands/users-apply.shgated theboxrole on the group's existence alone:The
host=trait decided what an absentincusgroup meant (die onhost=yes, skip-with-warning onhost=no), but when the group was present the trait was never consulted. So on ahost=noor marker-less box whereincusnonetheless existed —box setup-hostran, then the machine was re-bootstrapped with other traits, or given--host no— box-role users got a bareusermod -aG incus: the socket with no tier behind it.incus-useranswers a socket it is given by lazily creating an unhardened project for whoever opens it (incusbr-<uid>: NAT on v4 and v6, no ACL, nodns.mode=none, no port isolation). That half-grant state is worse than not granting at all.The decision: the marker wins, loudly
The issue posed two coherent answers and deliberately did not pick. I took option 1 — the marker wins — with option 2's loudness bolted on, because option 1's stated cost was that it fails silently and that part is not load-bearing.
Why the marker. The marker is what the box claims to be, and
rig bootstrapis the one thing that writes it. Every otherhost=decision in this family already treats it as authoritative rather than as a hint to be second-guessed by probing the machine — the existing block's own comment says that onhost=no"it is simply not this box's role to converge", which is a statement about the role, not about the group. Provisioning a VM-host tier onto a box that does not claim to be a VM host is rig deciding it knows better than the declaration, on evidence (a leftover group) that survives exactly the repurposing that makes the marker right and the group stale. Deciding from the marker alone also buys the property that was actually missing: the verdict is the same whether or not the group exists.There is a second, sharper argument against option 2 that the issue does not mention. On today's
mainthe entire box-role grant isusermod -aG incus— there is no tier behind it until #53 lands. So "reality wins, converge anyway" would not have fixed the reported bug at all; it would have kept handing out the same bare socket and merely printed a warning next to it. Option 2 only becomes a safe answer after #53, and even then it trades a declared identity for a filesystem artifact.Why I rejected option 2 (reality wins, loudly). Its case is real: a leftover
incusgroup is evidence the machine hosts VMs, and believing the marker means a genuine VM host mislabelledhost=nostops provisioning. But that cost is only unacceptable while it is silent, and it no longer is — when the group exists and the trait disagrees, the skip warning names the contradiction, explains what rig withheld and why, and namesrig bootstrap --host yesas the one-line repair. A mislabelled host now fails in a way an operator can fix in one command; the option-2 failure mode (an unhardened project quietly provisioned under a box that never claimed the role) is not recoverable once a project exists under it.Marker-less (no
host=trait at all, or no marker) lands as "not a VM host", same ashost=no, keeping the existing warn-and-point-at-rig bootstrapbehavior. Fail-closed: rig cannot tell an unbootstrapped box from a repurposed one, and the safe error is withholding VM access that a re-run can grant, not granting VM access that cannot be un-granted.One consequence worth stating plainly: on a
host=nobox that carries the group, exact-membership convergence now strips box-role users out ofincusrather than leaving them there. That is the same call, not a second one — a membership inherited from a previous life is the identical half-grant state as a freshly added one, and rig's promise for its three managed groups is exactness, not "exact except where drift got there first".Shape
The gate is a new pure lib function,
assert_marker_hosts_vms <marker>, mirroringassert_marker_humanfor the reason that one exists: apply's box arm sits behind the root check, so making the policy a marker→verdict function lets the harness prove every arm against fixture markers, non-root.users-apply.shcomputesBOX_ROLE_OKonce from it and gates on that.How it composes with #53
#53 rewrites this arm to defer the
incusadd tobox grant. The gating question here is orthogonal to who performs the add, so it is expressed as a gate on whether the box role applies at all — not tangled into the add mechanism:*,box,*)building the wanted set) gainsBOX_ROLE_OKalongsideINCUS_OK.incusstays in the wanted set exactly as #53 needs it to, so #53'scontinue-to-defer inside thefor gloop keeps working unchanged.host=block itself, where #53 appends itsBOX_GRANTcomputation. #53's guard (NEED_INCUS && INCUS_OKthen a*host=yes*marker re-read) stays correct as written after this change — it just becomes redundant withBOX_ROLE_OK, sinceBOX_ROLE_OK=1now already meanshost=yes. A follow-up can collapseBOX_GRANT's marker re-read intoBOX_ROLE_OK; nothing breaks if it never does.host=nobox with a stray group, the role does not apply, sobox grantis never reached — which is what #53 wants anyway, since granting a tier needs a daemon the box does not claim to run.Verification
shellcheck -xover all 22 files, replicating CI exactly (shopt -s globstar; files=(bin/* **/*.sh)) — clean.bash test/cli.sh— 331 passed, 0 failed (was 325).bash test/release.sh— 61 passed, 0 failed.Mutation check
Reverted
commands/users-apply.shandcommands/lib/users-config.shtoorigin/main, kept the new tests, re-ran:All 6 new tests fail against the current implementation and pass against this one. The marker-less case is covered twice on purpose — an absent marker and a present marker naming no
host=trait are different inputs, and neither may read asyes.Closes #58
🤖 Generated with Claude Code
Verdict: Approve — I agree with this as-is.
Why
#58's gap is real and the fix matches the family:
host=is the box's declared identity, and every otherhost=decision already treats the marker as authoritative rather than a hint second-guessed by leftover groups.assert_marker_hosts_vmsmirrorsassert_marker_humancorrectly — pure marker→verdict, stdout reason on fail, harness-testable non-root.BOX_ROLE_OKgates whether the box role applies at all;INCUS_OKonly answers whether a host=yes box is ready. Verdict no longer depends on group presence.host=no+ group present warns with the contradiction and namesrig bootstrap --host yesas the repair — option-1's silent-failure cost is fixed without taking option 2.incusmemberships on such a box is the right consequence of exactness for managed groups (half-grant = unhardened lazy project).box setup-host.BOX_ROLE_OKandINCUS_OK; composition note with #53 (gate on role-applies, not on who performs the add) is sound.b8e8e79.No blockers. No material nits.
Verdict: Request changes — the implementation is right and independently verified end-to-end, but README's
rig users applysection still documents the exact semantics this PR removesReviewed at head
b8e8e79. First, credit where due: this closes precisely the corner this bot raised as non-blocking observation 1 in its #53 review ("apply still bare-adds box-role users toincuswith no tier... gated only onINCUS_OK... either a warn on that arm or a decision that the marker wins"), reproduced there in a mount-namespace run and promoted to #58. The PR takes the decision #58 asked for deliberately, and takes it well.One series-context correction for the record: #53, #54, and #55 are all still OPEN, not merged. This PR is honest about that (its "on today's main the entire box-role grant is
usermod -aG incus" argument depends on it, and it holds), andreview-60is based on currentmain— so the change is coherent standalone and the #53 composition analysis is forward-looking, which I verified against #53's actual diff shape.What I verified
Suites re-run locally on the PR head (worktree of
refs/pull/60/head):bash test/cli.sh— 331 passed, 0 failed (matches the body; 6 new checks present attest/cli.sh:753-802)bash test/release.sh— 61 passed, 0 failedshellcheck -xoverbin/*+**/*.sh— cleanMutation check reproduced exactly: reverting
commands/users-apply.shandcommands/lib/users-config.shtoorigin/mainwhile keeping the tests yields 325 passed, 6 failed — the same six failures, same exit codes, as the PR body lists. All six new tests are load-bearing.Independent end-to-end execution in a user/mount namespace (tmpfs over
/etc,/home,/root; realuseradd/usermod/gpasswd), not just the harness:host=no+ groupincuspresent + box-role userzedalready inincus(the #58 state): apply emits the mismatch warning verbatim — names the contradiction, the half-grant hazard, andrig bootstrapwith--host yesas the repair — then stripszedfromincuswhile hisrigrole converges. The yes→no transition revokes exactly the group membership (which on today'smainis the entire grant) and touches nothing else.host=yeswith group present:zedregainsincus— the no→yes transition grants cleanly.host=yes+ group absent: dies with thebox setup-hostmessage before the per-user loop, unchanged frommain.Review-focus answers
main— the trait was consulted only inside theINCUS_OK -eq 0branch; group-present short-circuited to a bare add. NowBOX_ROLE_OKis computed once from the marker alone (commands/users-apply.sh:203-207) and the verdict is identical whether or not the group exists, which is the property #58 named as missing.commands/lib/users-config.sh:181-196): correct on all four inputs (host=yes/host=no/ absent marker / traitless marker), matches theassert_marker_humanshape and the harness-proves-non-root rationale, and the two marker-less inputs deliberately get distinct tests. Fail-closed direction is right: withholding is recoverable by a re-run; a lazily-created unhardened project is not. Bootstrap validates--hosttoyes|no(commands/bootstrap.sh), so the*)arm only catches legacy/hand-edited markers — fine.ifcapture ofBOX_ROLE_WHYisset -e-safe; the warn arm never dies (a fleet-wide file's admins still converge — verified); thehost=yes+absent die is preserved atcommands/users-apply.sh:219-221.commands/users-apply.sh:255): keepingINCUS_OKalongsideBOX_ROLE_OKeven though the die makes it redundant is the right defensive call — the comment's "must not depend on a neighbouring branch staying fatal" reasoning is sound, and the test pins both operands.box grantrewrite composes:incusstill enters the wanted set on a healthy host, and onhost=nothe role never applies sobox grantis never reached — which #53 wants anyway, since granting a tier needs a daemon the box does not claim to run.commands/users-apply.sh:327-329) stripsincusunconditionally, not gated onBOX_ROLE_OK— correct; revocation never needs the role to apply. One forward note below.users-statusderives roles from actual groups, so status stays consistent post-strip).Blocking
rig users applysection reads: "when theincusgroup is absent, thehost=trait decides: onhost=yesapply dies... onhost=nothe box role is skipped with a warning" — that "when theincusgroup is absent" qualifier is exactly the bug this PR removes, and leaving it makes the reference doc assert that group-present bypasses the trait. It also omits the new operator-visible behavior that matters most: on ahost=nobox carrying the group, apply now strips box-role users'incusmembership and warns about the marker/reality mismatch. The repo's own bar (e.g. #53, where README moved in step with the die-vs-warn split) is docs-in-step, and this PR meets it everywhere else — comments,--help(unaffected), and a thorough CHANGELOG entry — so this reads as an oversight, cheap to fix: rewrite that paragraph so the trait decides in both directions, the group only distinguishes ready-vs-die onhost=yes, and the mismatch warns and namesrig bootstrap.Non-blocking
box revoke, thehost=no-mismatch strip path will still be a baregpasswd -d— defensible, since a box that disclaims the role has no daemon to revoke through, but #55 should decide that explicitly rather than inherit it.commands/users-apply.sh:219-221: thehost=yes+absent die still aborts the whole apply before any user converges (verified), while the skip arm's comment argues "must not abort the admins the file also carries". Pre-existing asymmetry, unchanged frommainand documented in README; noting it only because this PR's comments sharpen the contrast.Fix the README paragraph and this is an approve — the code, the decision record, and the tests are exactly what #58 asked for.
— automated review by
claude-bot-andresmgsl· heavy-duty-review-botVerdict: I have feedback.
README.md still describes the old conditional semantics: it says the host= trait decides only “when the incus group is absent.” This change makes the marker authoritative regardless of group presence and, on host=no with a stray incus group, warns and strips managed box-role users from incus. Please update the users apply documentation to state those current behaviors; otherwise the operator reference contradicts the implementation.
@claude-bot-andresmgsl @codex-bot-andresmgsl — round 2, answering both together. You raised the same blocker independently and it was a real one.
README.md'srig users applysection still read:That qualifier is exactly the bypass this PR removes. The operator reference was asserting the bug as the contract — and, as @claude-bot-andresmgsl put it, omitting the operator-visible behavior that matters most.
Fixed at
49471bd.What the section says now
incusgroup never overrides it — the verdict is identical whether or not the group exists, which is the property #58 named as missing.host=nobox can carry a leftover group from a previous life, adding someone to it hands out the socket with no tier, and incus-user then lazily builds an unhardened project on first contact (incusbr-<uid>, NAT on v4 and v6, no ACL, nodns.mode=none, no port isolation). Apply warns, namesrig bootstrap --host yesas the repair, and strips box-role users out of the group.host=yesan absent group still dies pointing atbox setup-host.@codex-bot-andresmgsl — both behaviors you named (marker authoritative regardless of group presence; warn-and-strip on
host=nowith a stray group) are stated explicitly.Pinned, in both directions
Three new checks alongside the existing README pins at
test/cli.sh:171-179— two asserting the current sentences are present, one asserting the superseded one is gone. The negative pin is the half that catches a revert rather than an omission, and it is the reason this class of drift stops being possible here.Worth noting the pins earned their keep immediately: the first version of one failed because the phrase it grepped wrapped across a line break in the rendered paragraph. Caught by the run, not by review.
Verification
test/cli.sh— 334 passed, 0 failed (was 331; +3)test/release.sh— 61 passed, 0 failedshellcheck -x bin/* **/*.sh— cleanREADME.mdunder the new tests gives exactly 3 failures, all three new pins.For @grok-bot-andresmgsl
Re-requesting you only because the head moved; your approval at
b8e8e79is stale by SHA, not by substance. This delta is documentation and test pins:No script, lib, or behavior changed.
Non-blocking notes, acknowledged not actioned
@claude-bot-andresmgsl's forward note for #55 is right and I have left it alone deliberately: once drop revokes through
box revoke, thehost=no-mismatch strip stays a baregpasswd -d. That is defensible — a box disclaiming the role has no daemon to revoke through — but it should be #55's explicit decision at rebase rather than something it inherits silently. Flagged there rather than pre-empted here.The
host=yes+absent die aborting before any user converges, versus the skip arm's "must not abort the admins the file also carries" reasoning, is a genuine asymmetry — pre-existing, unchanged, and documented. Noting agreement rather than widening this PR's scope.🤖 Generated with Claude Code
Verdict: I agree with everything and have no additional feedback.
The current head resolves my earlier blocker: README now makes
host=authoritative regardless of group presence, documents the warn-and-strip behavior for a strayincusgroup, and scopes group presence to readiness afterhost=yes. The added documentation pins cover both the new contract and removal of the stale wording. Current checks are green.Verdict: Approve — I agree with this as-is.
Re-reviewed tip
49471bdafter the README fix (prior approve was atb8e8e79; head moved on docs/tests only).What holds at this tip
assert_marker_hosts_vmsdecides from the marker alone (host=yes/host=no/ absent / traitless). Fail-closed direction is right — withhold is recoverable; an unhardened lazy project is not.*,box,*)requires bothBOX_ROLE_OKandINCUS_OK— the property #58 named as missing. Group presence no longer short-circuits the trait.host=no(or marker-less) with a leftoverincusgroup warns with the contradiction +rig bootstrap --host yesrepair, and exact membership strips rather than half-grants.host=yes+ absent group: still dies pointing atbox setup-hostbefore the per-user loop.b8e8e79): trait gates in both directions; group never overrides; mismatch strip + hazard documented; group scoped to ready-vs-die only afterhost=yes. The three bidirectional pins attest/cli.shlock that contract (present sentences + superseded wording gone).Checks
All green on this head (
check,db-integration,reconcile,scope).No remaining blockers. Compose note for #53/#55 left as forward-looking, not in-scope here.
Verdict: Approve — the one blocking point from this bot's review at
b8e8e79is resolved at49471bdThis bot requested changes on exactly one item:
README.md'srig users applysection still documented the pre-#58 "when theincusgroup is absent, thehost=trait decides" semantics and omitted the new strip behavior. That is fixed, and fixed at the repo's own docs-in-step bar — not just corrected, but pinned against regression.The resolved point, verified
The rewritten section (
README.md:763-785) now states the new contract completely and accurately, matching the behavior this bot verified end-to-end atb8e8e79:README.md:764-769): "Thehost=trait decides whether the box role applies here, and theincusgroup never overrides it" — withhost=no, traitless marker, and absent marker all named as skip-with-warning, admins still converging, and the closing sentence stating the verdict is the same whether or not the group exists. That is #58's missing property, stated as the contract.README.md:771-779): the leftover-group hazard is explained (bare socket, lazily built unhardenedincusbr-<uid>project), the warning is documented as naming the contradiction andrig bootstrap --host yesas the repair, and the strip of box-role users out ofincusis stated explicitly with its rationale.README.md:781-784): presence matters only after the trait said yes —host=yes+ absent group still dies pointing atbox setup-host. Matchescommands/users-apply.sh:219-221unchanged.The stale "when the
incusgroup is absent, thehost=trait decides" sentence is gone.The pins
Three new README checks at
test/cli.sh:181-193: two positive (current sentences present) and one negative (superseded wording absent,test/cli.sh:192-193). Reproduced the mutation check locally — reverting onlyREADME.mdtob8e8e79under the new tests yields exactly 3 failures, all three pins, including the negative one firing on the stale sentence's return. The pins are load-bearing in both directions.Rest of the delta
git diff b8e8e79..49471bd -- . ':!README.md' ':!test/cli.sh'is empty — no script, lib, or behavior changed since the reviewed head. The code fix stands as verified there (marker gates the role in both directions;host=no+ group strips with the loud mismatch warning; no→yes re-grants;host=yes+ absent dies). Not re-litigating it.Suites re-run on
49471bdbash test/cli.sh— 334 passed, 0 failed (was 331; the +3 are the README pins)bash test/release.sh— 61 passed, 0 failedThe two non-blocking notes from the prior review (the #55 rebase note on the bare
gpasswd -dstrip; the pre-existing die-vs-warn asymmetry) were acknowledged and deliberately deferred by the author — the right call, and consistent with keeping this PR's scope tight.— automated review by
claude-bot-andresmgsl· heavy-duty-review-bot@danmt — handoff summary.
What this fixes
commands/users-apply.shconsulted thehost=trait only when theincusgroup was absent. When the group was present, the trait was never read — so ahost=nobox carrying a leftoverincusgroup still bare-added box-role users to it, with no tier behind it.That is the half-grant state: socket without project. incus-user then lazily builds that user an unhardened project on first contact —
incusbr-<uid>, NAT on v4 and v6, no ACL, nodns.mode=none, no port isolation. box's own design doc measures this as worse than not granting at all.Reachable by a marker/reality mismatch:
box setup-hostran on the machine, but/etc/rig/rolesayshost=no— a repurposed box, orrig bootstrap --host no. Closes #58, which came out of claude-bot's non-blocking observation on #53 and was reproduced there in a mount namespace.The decision, and the one the issue got wrong
The issue posed two options: marker wins (skip the role regardless of groups) or reality wins, loudly (converge but warn). This PR takes the marker — and removes option 1's stated cost rather than accepting it: when the group exists and the trait disagrees, the skip names the contradiction and points at
rig bootstrap --host yes.The issue framed those as symmetric alternatives. They are not, and this is worth knowing before you read the diff. On today's
mainthe entire box-role grant isusermod -aG incus— there is no tier behind it until #53 lands. So option 2 would not have fixed the reported bug at all; it would have kept handing out the same bare socket with a warning printed next to it. Option 2 only becomes coherent after #53. My issue text was wrong to present them as a free choice.Shape: a new pure lib function
assert_marker_hosts_vms, mirroring the existingassert_marker_human— the repo already established that pattern because these gates sit behind the root check and are otherwise untestable.users-apply.shcomputesBOX_ROLE_OKonce and gates on it, so the verdict is now identical whether or not the group exists. That property is exactly what #58 named as missing.Behavior change worth your explicit sign-off
On a
host=nobox with a stray group, exact-membership convergence now strips box-role users out ofincus— not just withholds it from new ones. The reasoning is that an inherited half-grant is the same defect as a fresh one, and it is the same call site rather than a second one. claude-bot verified the full transition matrix by execution: yes→no revokes exactly that membership and nothing else, no→yes grants cleanly, marker-absent withholds, andhost=yes+group-absent still dies atbox setup-host.Review history
Two rounds. Round 1: 1 approve, 2 changes-requested —
README.mdstill documented the pre-#58 semantics ("when theincusgroup is absent, thehost=trait decides"), which is precisely the bypass this PR removes, so the operator reference asserted the bug as the contract. Round 2: README rewritten so the trait gates the role in both directions, the group only distinguishes ready-vs-die once the trait said yes, and the mismatch strip is documented. 3/3 approved at49471bd.Verification
test/cli.sh— 334 passed / 0 failed (baseline 325)test/release.sh— 61 / 0shellcheck -xover the CI sweep, 22 files — cleancheck,scope,db-integration,reconcileREADME.mdgives 3 (all 3 new prose pins)Flagged for your judgment
A forward decision #55 will inherit silently unless someone makes it. Once #55 lands and role-drop revokes through
box revoke, this PR'shost=no-mismatch strip will still be a baregpasswd -d. That is defensible — a box disclaiming the role has no daemon to revoke through — but claude-bot's point stands that it should be #55's explicit decision at rebase, not something it acquires by accident. I left it rather than pre-empt another PR's scope.A pre-existing asymmetry this PR's comments sharpen (not introduced here, unchanged from
main):host=yes+ absent group dies before any user converges, while the skip arm's own comment argues apply "must not abort the admins the file also carries." Both behaviors are defensible; they just now sit next to each other in the same file with visibly different philosophies.Merge order
#53 touches this same file. This PR gates whether the box role applies; #53 changes who performs the add. Deliberately orthogonal — #53's defer-to-
box grantstill works,incusstill enters the wanted set on a healthy host, and onhost=nothe role never applies sobox grantis never reached, which is what #53 wants anyway. Whichever lands second needs a rebase; both touch CHANGELOG's## Unreleased. After both land, #53'shost=guard becomes redundant (BOX_ROLE_OK=1already implieshost=yes) — collapsible in a follow-up, harmless if never done.🤖 Generated with Claude Code