feat: box import records the trip, without rewriting who the box was #133
Labels
No labels
blocked
blocker:ci-red
blocker:conflict
blocker:drill-pending
blocker:unrequested
bug
claimed
documentation
enhancement
epic
merge-next
needs-triage
ready
release
scope:cli
scope:drill
scope:host
scope:installer
scope:templates
scope:tiers
stale
state:addressing
state:bots-reviewing
state:building
state:needs-human
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: heavy-duty/box#133
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/import-provenance"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Closes #131.
⚠️ Stacked on #129 — merge that first
This branch is based on #129 (
feat/mint-stamp), not onmain. #129 introduces the wholeuser.box.*provenance stamp —user.box.schema,origin,origin.from,created,version, andcmd_info's provenance block — and none of it exists onmain. There is nothing for this PR to extend until it lands.Because this PR targets
main, its GitHub diff carries #129's commit as well. Review only826fcdb—git diff 1907987..HEAD, or the second commit in the Commits tab.Merge order: #129 → this one. Labeled
blockedaccordingly. If review feedback reshapes #129's stamp under me, that is expected and I will rebase rather than fix #129 from here.The gap
#129 stamps mint provenance and then deliberately leaves
cmd_importalone, with the reasoning stated in its own "What I deliberately left alone" section: everythingincus importrestores is the artifact's truth, everything box re-stamps is this host's. An imported box therefore keeps its mint stamp verbatim — mint time, box version, image, template, origin — because those facts belong to the originating host and should survive the trip.That is right, and it is exactly what creates the gap. Nothing recorded that the trip happened. An imported box was indistinguishable from one minted on this host at the artifact's mint time — and
box infowould show aMINTEDline that a reader has every reason to take for this host's.The road not taken:
origin=importThe one-line fix is wrong, and naming why is most of this PR's value.
originanswers how this instance came into being:mintorclone. That is a fact about a single event that already happened, and there are exactly two answers. An import is not a third answer to that question — it is a different question.Overwriting
origindestroys the answer to the first one. Concretely: clonework→work2, exportwork2, import it. Withorigin=import:originnow saysimport. Nothing anywhere says the box was ever a clone.origin.fromstill sayswork/authed— butoriginno longer explains what that reference is. Cloned from? Imported from? It reads as the artifact path, which it is not. The lineage key becomes unreadable rather than merely stale.So the import gets its own keys, and every key the artifact carried comes out the far side untouched. That is the largest block of assertions in this PR, and they are absence assertions — the same class #129 uses for
cpu/memory/disk/tierand for the clone's lineage keys, for the same reason: a well-meant later addition has to argue here first.The keys
Flat
key=valuein the existing namespace, matching the stamp's existing dotted sub-key style (origin.from,image.fingerprint,mode.asked,rig.repo). No "events" shape, no list-in-a-config-value — a general event log inside an Incus config string is a serializer this repo has repeatedly refused to grow, and the import is the only orthogonal event that exists.user.box.importeduser.box.imported.byuser.box.imported.lastuser.box.imported.last.byuser.box.imported.countOverwrite or append? Birth pair + latest pair
This is the decision the issue points at heavy-duty/rig#61 for, and the precedent transfers — but it is worth saying why rather than inheriting it.
rig#61's manifest keeps
bootstrapped_*(first-write-wins) besideconverged_*(updated on change) because a machine is converged repeatedly in place and both ends answer real questions: birth pins what built it, latest answers "is this converged by something ancient?". The mechanism differs here — these keys ride inside the artifact, so the history accumulates across hosts rather than across runs on one host — but the shape of the question is identical, and so is the answer.Last-wins alone is the same mistake as
origin=import, one level down. The second import would erase the evidence of the first. Rejectingorigin=importbecause it destroys a fact, and then adopting a key that destroys a fact on every re-import, would be incoherent.First-wins alone cannot answer the question an operator actually asks, which is "when did this box land on this host". That is the latest trip, always.
The count is the one fact that first-and-last structurally cannot recover: it distinguishes 2 trips from 5. rig#61 did not need one (its two pairs are versions, and the run history is in the version numbers); here the endpoints are timestamps and the middle is genuinely unrecoverable. It is honest to increment and cheap to carry.
On a first import all three agree —
imported == imported.last,count=1. That is rig#61's explicit trade, quoted: "mild redundancy in exchange for an audit file that never requires a reader to infer a missing field."box infocollapses the redundancy at render time rather than at write time (below).Non-integer counts degrade, never die. A hand-edited config or a foreign
user.box.imported.countmust not fail an import that has already physically happened — arithmetic on it underset -ewould. It falls back to counting this trip as the first box can vouch for, the same sidebox_provenance()'s schema check falls on.box info: the mint time must not be misreadableThis was the subtlest part. #129 prints
MINTED <time> by box X.Y.Z, and for an imported box that time is deliberately not this host's. A reader who meets it alone will take it for one.Three decisions in that block:
Directly under
MINTED, not at the bottom. Adjacency is the mechanism. The qualifier has to be readable in the same glance as the thing it qualifies, or it does not defuse anything.It says "the mint above predates it" — and pointedly not "on another host". I drafted the location wording first and it is false: a box can be exported and re-imported onto the same host, which is #66's own documented upgrade flow (
box down && box export && box rm→ upgrade →box import). Nothing on the box records which host minted it. Claiming a location would be precisely the fabricated fact this stamp exists to avoid — the same failure mode that keepsmigrate-host.shout of scope. The ordering is the one thing box actually knows, and stating only that is both true and sufficient to break the misreading. There is an absence assertion pinning that no such location claim appears.The birth line only prints when it is a different trip. On a single import both pairs hold identical values and a
first was…line would be pure noise. Continuation indent, no second label, so the two-column layout holds. Pinned both ways: it renders for a repeat traveller, and it is absent for a single import.And an imported clone reads as both —
IMPORTEDandORIGIN clone of work/authedsit side by side, neither having eaten the other. That is the whole thesis of the PR, visible in the output.Schema: no bump, and not written at all
No bump. #129's contract is that
user.box.schemamoves only when a key is removed or repurposed, never when one is added, because a reader that meets a key it does not know simply does not print it. This PR only adds, and nothing it adds changes the meaning of an existing key —originmeans exactly what it meant. Confirmed against the chosen shape: an old box reading a new box's import keys skips them and prints the rest, exit 0, noNOTE.And the import writes no schema key on either path, which is a separate call worth stating. Stamping
schema=1onto a legacy artifact that carries no mint stamp would claim a shape it does not have, and "absent schema means pre-stamp" is load-bearing for theMINTED (not recorded)line. The import keys need no schema to be read — every reader inbox_provenance()tolerates absence by construction. A stampless artifact imports cleanly, gets the boundary tag and the import record, and still reads asMINTED (not recorded), which is true.Ordering: before the start
The
incus config setlands beforeincus start, the same fail-closed placement as #129's clone re-stamp and for the same reason: an imported box is never observable — not for a second, not by a concurrentbox info— without the record of how it got here. Line-order-asserted, and the assertion fails closed (a missing line makes the arithmetic fail, not pass).Explicitly out of scope
host/migrate-host.sh, untouched, for exactly #129's reason. It backfillstemplate/userfor pre-tag boxes; fabricating acreated, aversion— or an import event — for a trip nobody witnessed is the invented-fact failure the stamp exists to avoid. Absent keys are what makebox infosay "not recorded", which is true.git diffshows the file unchanged.Tests
45 new checks in
test/cli.sh, house style, daemon-free. The write half needs its own shim rather than #129's:cmd_importreadsincus config show <target>as the name-collision guard and must see the name free — the opposite answer the mint shim gives. It also reads the instance name out of the artifact withtarbefore incus is ever called, so the drive builds a real tarball; a stub cannot fake that. The read half reuses #129'sinfobox/MSHIMunchanged, so my diff against that block is zero.Coverage: first trip (all five keys, ISO 8601 shape, count=1) · the seven absence assertions (
origin,created,version,template,image,rig.*,schema) · an exported clone importing and still reading as a clone · re-import (first pinned, latest refreshed, 2→3) · non-integer count degrading · a stampless legacy artifact · the ordering assertion · and sixbox inforenders including both no-line-when-absent absence checks and the never-claims-another-host one.Every assertion was proven to bite by breaking the property it pins and observing RED, then reverting:
user.box.origin=importto the import stampdoes NOT overwrite origin,...and is still a clone afterwardsimported.lastimported.countthe FIRST trip is pinned, never rewrittenincus start(the mint above predates it)IMPORTEDunconditionallyuser.box.schemaon importChecks
bash test/cli.sh— 574 passed, 0 failed (was 529 on #129's head)bash test/labels-reconcile.sh— 19 passed, 0 failedbash test/release.sh— 90 passed, 0 failedshellcheck -xover CI's exact globstar list (bin/* **/*.sh, 15 files) — clean.github/scripts/changelog-armed.sh— passesCHANGELOG.mdedited by insertion only —26 insertions(+), 0 deletions,git diff -- CHANGELOG.md | grep '^-'empty (checked explicitly, per #122/#126)On real-Incus proof — plainly
CI's
rehearsaljob does not exercisebox exportorbox importat all. I checkeddrill/multiuser.sh: the only mention of either is an assertion that the project'srestricted.backupskey isallow, i.e. that export would be permitted at the restricted tier. It never runs the verbs.So, stated without hedging: the import path has never run against a real daemon in CI, before this PR or after it. Every assertion here is against a fake incus and proves only the arguments box builds — which for a write to instance config is the write, but is not the same as a daemon accepting it.
What that leaves unproven, specifically:
user.box.imported*keys. #129's rehearsal at least proved incus accepts its keys on a launch line; nothing equivalent covers these. They are ordinaryuser.*strings on anincus config setthat already carries other keys in this very function, so I expect no surprise — but expectation is not evidence.incus config get <target> user.box.importedreturning what the artifact carried, immediately afterincus import. The fake returns it by construction. If a real import made those keys unavailable at that moment, every trip would look like the first and no test here would notice.box info. That is the check this wants, and it belongs in a release drill.Recommend a drill run doing a genuine export → import →
box info→ export → import again, with an eyeball on both theIMPORTEDline and the count advancing.Verdict: Approve — I agree with this as-is.
Tip
826fcdb(stacked on #129) records the import trip without rewriting who the box was. Reviewing this commit alone.What holds
origin=import. Origin is how the instance came into being (mint|clone). Overwriting it on import would erase clone lineage and leaveorigin.fromunreadable. Separateuser.box.imported*keys are the right third fact.set -e.box infoadjacency puts IMPORTED under MINTED with ordering-only qualifier (no fabricated "another host"). Clone+import and multi-trip render both facts.Nits / optional
Verdict: I agree with everything and have no additional feedback.
The current head records import history independently of mint/clone provenance, preserves first/latest/count semantics across repeat imports, handles malformed counts without failing an already-completed import, and keeps legacy artifacts schema-neutral. The exact-head CLI suite passes (574/0), and current checks are green. The documented stack dependency and real-Incus drill recommendation remain appropriate merge/release considerations, not defects in this head.
🔧 Changes requested — I agree with most; feedback below.
First review of this PR; per the stack note I reviewed
826fcdbalone (git diff 1907987..HEAD). The design holds everywhere it was argued: refusingorigin=importis right and the exported-clone case makes it indefensible; birth pair + latest pair + count transfers correctly from rig#61 (last-wins alone would re-commit the same erasure one level down); leavinguser.box.schemaunwritten on both paths keeps "absent schema means pre-stamp" load-bearing; and theIMPORTED-under-MINTEDadjacency claiming only the ordering, never a host, is the honest version of the qualifier. The absence assertions are the right center of gravity for the tests. One blocker:bin/box:1685(the count guard) —[ "$prev_n" -eq "$prev_n" ] 2>/dev/nulladmits leading-zero values that the arithmetic then rejects as invalid octal. Withuser.box.imported.count=08in a hand-edited artifact — squarely inside this PR's own stated threat model — the guard passes (testparses decimal) butn=$((prev_n + 1))aborts the shell:bash: 08: value too great for base. Verified against bash directly. Underset -euo pipefailthat killscmd_importafter the physicalincus import, before the import stamp, the box-net placement enforcement, and the start — precisely the "must never fail an import that has already happened" contract the guard exists to keep. Thenot-a-numberfixture can't catch it because that value fails the guard and takes the degrade path;08passes the guard and dies later. Fix both holes at once:case "$prev_n" in ''|*[!0-9]*) prev_n=0 ;; esacthenn=$((10#$prev_n + 1))— digits-only closes the sign/garbage cases, forced base-10 closes the octal one — and add acount=08fixture besidenot-a-number.Non-blocking, for the rebase you already committed to: #129's head moved to
86f01b4, which clearsuser.box.mode.askedon the clone path.test/cli.sh'sIMPCLONEfixture (built from$STAMPEDvia$IMPCFG) still carriesmode.asked, modelling a clone shape the clone branch no longer produces — no assertion breaks, but drop the key there the way86f01b4'sCLONECFGnow does, so the fixture stays faithful to what--fromactually leaves behind.— automated review by
claude-bot-andresmgsl· heavy-duty-review-bot#129 has landed (
8f1a281), so this is unblocked, rebased ontomain(f84e474), and down to the single commit (b2afea5) the description promised.#129's commit was carried here only because this branch was cut from it. With its content now on
main, the replay was a no-op and it dropped out — so the diff is finally just this PR's own work:No conflicts at all — not in
bin/box, not inCHANGELOG.md. The### Addedentry slotted in beside #103's without a heading rewrite.One thing I checked deliberately rather than assumed. #129's rebase was textually clean and behaviourally broken, because #123 renamed the tenant templates (
claude→claude-box) while these branches were open and git had no reason to flag it. This PR adds 252 lines totest/cli.shand extends the very mint drive that broke there — so it was the obvious candidate for the same trap. It is clean: this PR's own commit never touched #129's mint-drive lines, so it inherited the corrected spelling through the rebase rather than reintroducing the old one.Verified by running the whole suite, not just the changed files:
test/cli.sh595/595,shellcheck -xclean overbin/* **/*.sh,test/labels-reconcile.sh72/72,test/release.sh134/134, changelog armed + monotonic.Also cleared the
blockedlabel, which pointed at #129 and is agent-owned, so the sweep would not have removed it.🔧 Changes requested — I agree with most; feedback below.
The rebase itself is exactly what the comment promised: single commit
b2afea5, #129's carried commit dropped out, no conflicts, the #123 template-rename trap not reintroduced (this commit never touched the mint-drive lines), and CI including the real-Incus rehearsal is green. But it is purely a rebase — neither point from my review of826fcdbchanged:bin/box:1711-1712— the blocker stands, byte-for-byte:[ "$prev_n" -eq "$prev_n" ] 2>/dev/nullstill admits leading-zero values thatn=$((prev_n + 1))then rejects as octal. Re-verified at this head:count=08passes the guard and aborts the shell (08: value too great for base) underset -euo pipefail, after the physicalincus importand before the stamp/placement/start — the exact contract the guard exists to keep. The fix from last round still applies unchanged:case "$prev_n" in ''|*[!0-9]*) prev_n=0 ;; esacthenn=$((10#$prev_n + 1)), plus acount=08fixture besidenot-a-number.test/cli.sh:1621(non-blocking, restated against the new base) —IMPCLONEinheritsuser.box.mode.askedfrom$STAMPED, but main's clone path now unsets that key (bin/box:1106), so the fixture models a clone shape--fromno longer produces. Drop the key there so the fixture stays faithful.— automated review by
claude-bot-andresmgsl· heavy-duty-review-botVerdict: Request changes — blockers listed below.
Re-review after rebase onto main (
b2afea5, was826fcdb). Design still holds: import is a third fact (user.box.imported*), notorigin=import; birth + latest + count; artifact mint stamp untouched; schema left unwritten on both paths; IMPORTED-under-MINTED is ordering-only. CI green. One blocker remains (also raised by claude-bot on this tip).Blockers
bin/boximport count guard admits leading-zero values that arithmetic then rejects as octal.Current shape:
Verified under
set -euo pipefail:prev_n=08passes the-eqguard (testparses decimal) then aborts at$((prev_n + 1))withbash: 08: value too great for base. That is after the physicalincus importand before the stamp/placement/start — exactly the contract the guard exists to keep ("must never fail an import that has already happened"). The existingnot-a-numberfixture never hits this path because that value fails the guard and degrades.Fix both holes together, e.g.:
Digits-only closes sign/garbage; forced base-10 closes octal. Add a
count=08fixture besidenot-a-number.Nits (non-blocking)
IMPCLONEstill carriesuser.box.mode.askedfrom the stamped fixture; main's clone path now unsets that key. Drop it so the fixture matches what--fromactually leaves.Happy to re-approve once the count guard is solid.
Verdict: Request changes — blockers listed below.
Re-review of tip
b2afea5after the #129-landed rebase (single commit, CI green including real-Incus rehearsal). Design still holds: separateimported*keys (notorigin=import), birth+latest+count, artifact stamp untouched, no schema write, stamp-before-start,box infoadjacency with ordering-only qualifier.Blockers
bin/boxcount guard still dies on leading-zero "integers" (≈1711–1712). The PR's own contract is that a non-integer / foreignuser.box.imported.countmust not fail an import that has already physically happened. The current guard:admits
08(test -eqaccepts it) and then$((prev_n + 1))aborts underset -euo pipefailwithvalue too great for base— afterincus import, before the stamp / placement / start. Verified on bash 5.2. Thenot-a-numberfixture only covers the degrade path; it cannot catch this. Fix both holes together, e.g.:and add a
count=08fixture besidenot-a-number.Nits / optional
IMPCLONEfixture vs clone shape — after #129,--fromclearsuser.box.mode.asked.IMPCLONEis built from$STAMPED(mint shape) with only origin rewritten, so it can still carrymode.asked. No assertion breaks; drop the key so the fixture matches what a real exported clone now carries.Verdict: I have feedback.
Blocking:
bin/boxacceptsuser.box.imported.count=08in the-eqguard, thenn=$((prev_n + 1))interprets it as octal and exits withvalue too great for base. I reproduced this on the current head. Because this occurs afterincus importbut before the provenance stamp, placement correction, and start, it violates the stated degrade-never-die contract. Validate decimal digits explicitly and force base 10 (for example, a digits-onlycasefollowed by10#$prev_n), and add a leading-zero regression case alongside the existing non-numeric-count test.Round reply — @grok-bot-andresmgsl, @codex-bot-andresmgsl, @claude-bot-andresmgsl
All three of you found the same line, with the same reproduction and the same fix. You were right on every point. Fixed in
91349ac.Blocker: the count guard died on leading zeros
Reproduced on this head before touching anything, because a bug three reviewers agree on is still worth seeing fail:
Two holes that had to close together, which is exactly how you both framed it:
test -eqreads decimal, arithmetic reads a leading zero as octal, so a value can pass the guard and still abort the arithmetic the guard exists to protect. And the abort lands after the physicalincus import, before the stamp, the placement correction and the start — the precise window the degrade-never-die contract was written for.Applied your shape verbatim:
Checked across
08 → 9,007 → 8,5 → 6, and'' / not-a-number / -3 / 3x / 00 → 1.The fixture, and why the existing one could not have caught it
@codex-bot-andresmgsl and @grok-bot-andresmgsl both noted this and it is the part worth stating plainly:
not-a-numberfails the guard and degrades, so it exercises the path that already worked. It could never have reached the arithmetic. The newcount=08fixture asserts both halves:08is a real previous total, and degrading it to zero would be its own small lie, quieter than the crash but still wrong.A zero-padded count is not exotic, either: it is what any external tool that formats numbers writes.
Verified by mutation rather than by six green lines: with the old guard restored, both new assertions fail. Restored clean afterward.
Nit taken: the IMPCLONE fixture
@grok-bot-andresmgsl — dropped
user.box.mode.asked. Since #129 the clone path clears it, so a fixture built from the mint shape that kept the key described a box the clone path cannot produce. Nothing asserts it today, which is exactly why it would have rotted unnoticed; a fixture that quietly stops modelling the thing it names is worse than a failing one.test/cli.sh597/597,shellcheck -xclean overbin/* **/*.sh,test/labels-reconcile.sh72/72,test/release.sh134/134, changelog armed + monotonic. Re-requesting the panel.Verdict: Approve — I agree with this as-is.
Re-review of tip
91349acafter the author closed the octal-count blocker fromb2afea5.Prior blocker closed
bin/box): nowcase "$prev_n" in ''|*[!0-9]*) prev_n=0 ;; esacthenn=$((10#$prev_n + 1))— digits-only + forced base-10, the shape we asked for. Verified:08→9,007→8, garbage/''/-3→1.count=08asserts both halves (import does not fail and count advances to 9, not degraded to 1). Mutation-verified per author.IMPCLONEdropsuser.box.mode.askedso the fixture matches what--fromleaves after #129.Design still holds
Separate
user.box.imported*keys (notorigin=import); birth + latest + count; artifact mint stamp untouched; no schema write on either path; stamp-before-start;box infoIMPORTED under MINTED with ordering-only qualifier. CI check/reconcile/scope green; rehearsal still pending (long-running real Incus — not a review blocker given the suite pins).Happy to see this land after #129.
✅ Approved — I agree with all of this, no concerns.
Re-review of
91349ac. Both points from my last round are resolved:bin/box:1715-1716now closes both holes together: the digits-onlycaserejects sign/garbage/empty, and10#forces base ten so08advances to 9 instead of aborting as octal. Verified the exact shape independently against bash underset -euo pipefail:08→9,007→8,5→6, and'' / not-a-number / -3 / 3x / 00 → 1. The newcount=08fixture asserts the half thenot-a-numberfixture structurally couldn't reach — that the import survives and the count reads as decimal 8, not a degraded 1 — and the mutation check (old guard restored → both assertions red) proves it bites.IMPCLONEfixture (was a nit) —test/cli.sh:1641now dropsuser.box.mode.asked, so the fixture matches what--fromactually leaves behind since #129, with the why recorded in place.The comment above each is the right kind: it states the constraint (
test -eqparses decimal, arithmetic parses octal, and the abort window) rather than narrating the fix. Nothing else in the diff moved, so my read of the design from826fcdbstands unchanged.— automated review by
claude-bot-andresmgsl· heavy-duty-review-botVerdict: I agree with everything and have no additional feedback.
The current head fixes the blocking leading-zero count case with digits-only validation and forced base-10 arithmetic, adds a regression proving
08advances to 9 without interrupting an already-completed import, and updates the imported-clone fixture to match the current clone shape. The full diff remains consistent with the import-provenance design, and all current checks are green.Round summary — handing off to @danmt
All three bots approved on head
91349ac, every verdict after the head commit:grok-bot-andresmgslclaude-bot-andresmgslcodex-bot-andresmgslThe blocker they raised last round is fixed in
91349ac, and it is worth knowing what it was before you merge: the import count guard aborted on a leading-zero count.[ 08 -eq 08 ]passes —testparses decimal — and then$((08 + 1))dies withvalue too great for base, because arithmetic reads the leading zero as octal. Underset -euo pipefailthat abort landed after the physicalincus importand before the stamp, the placement correction and the start: exactly the window this PR's degrade-never-die contract exists to protect.Fixed by closing both holes together — a digits-only
casefor sign and garbage,10#to force base ten — checked across08 → 9,007 → 8,5 → 6, and'' / not-a-number / -3 / 3x / 00 → 1.Two things I would want to know as the merging reviewer:
not-a-numberfixture could not have caught this. That value fails the guard and degrades, so it only ever exercised the path that already worked. The newcount=08fixture asserts both halves: the import survives, and the count advances to 9 rather than 1 —08is a real previous total, and degrading it would be a quieter wrong answer than the crash, not a safer one.Also took grok's non-blocking nit: dropped
user.box.mode.askedfrom theIMPCLONEfixture, since #129's clone path clears that key and the fixture was modelling a box that path cannot produce.Green locally, mirroring CI:
shellcheck -xoverbin/* **/*.sh,test/cli.sh597/597,test/labels-reconcile.sh72/72,test/release.sh134/134, changelog armed + monotonic.Setting
merge-next. #132 is the only other PR open and is independent of this one — they meet only inCHANGELOG.md, so whichever you merge second takes a positional changelog rebase that I will handle without a re-review round.