heavy-duty/rig

Fork 1

A rig-managed machine records nothing about which rig converged it, or when #61

New issue

Closed

opened 2026-07-19 17:34:08 +00:00 by dan-claude-bot · 1 comment

dan-claude-bot commented

2026-07-19 17:34:08 +00:00

(Migrated from github.com)

A rig-managed machine records nothing about which rig converged it, or when

The whole record

At the end of a bootstrap run, the entire durable output of rig is
commands/bootstrap.sh:427:

printf 'role=%s class=%s host=%s join=%s\n' "$ROLE" "$CLASS" "$HOST" "$JOIN" > "$MARKER_TMP"

One line in /etc/rig/role. That marker is doing exactly the job it was built
for — it is the traits' ground truth, read by warn_bootstrapped()
(bin/rig:141-146), the tenant guard (bootstrap-tenant.sh:109-123),
assert_marker_human() (lib/users-config.sh:120-147) and four advisory
sites. This issue does not touch it.

What it does not record is provenance. SSH into a control-plane on Hetzner or
the M900 runner six months on, and rig cannot tell you:

which rig converged it. VERSION is read in exactly one place,
bin/rig:9, for --version — and that reports the currently installed
tree, not the one that ran. A machine converged by 0.1.0-dev and one
converged by 0.4.0 are indistinguishable on disk.
when. There is no timestamp anywhere in the codebase. The only date read
is systemctl show ssh -p ExecMainStartTimestamp (users-close-root.sh:249),
used transiently and discarded.

That is the whole gap this issue closes. Deliberately narrow.

Scope: provenance only, and why that is the whole trick

An earlier draft of this issue also proposed recording the machine's specs —
cores, RAM, disk — in the same file. That was wrong, and the reason is worth
stating because it is what makes the rest of this easy.

Specs change without rig doing anything. Someone adds RAM, resizes the
root disk, or the unattended-upgrades this very script enables
(bootstrap.sh:198-201) patches the kernel. A stored spec is therefore either
stale or in need of constant refresh — and a refresh on every run collides
head-on with the contract at bootstrap.sh:3: "Convergent: safe to re-run; a
second run changes nothing." That contract is enforced by a cmp-guard before
every file install (nine sites: bootstrap.sh:428, lib/sshd.sh:31,
users-apply.sh:249, :300, bootstrap-tenant.sh:318, and more), and I
suspect it is exactly why no timestamp exists anywhere in the tree today.

So: facts that are decided go in the manifest; facts that are observed get
computed at run time and stored nowhere. Specs are observed — they belong to
rig platform (#64), which calculates them fresh on every invocation and
writes nothing. That split removes the volatility problem rather than managing
it, and it leaves this file with only immutable content.

Proposed manifest

/etc/rig/manifest, 0644, alongside the marker and the users ledger.
/etc/rig/ already exists and mkdir -p /etc/rig is already open-coded in
three places (bootstrap.sh:429, bootstrap-tenant.sh:374,
users-apply.sh:301).

key=value, one per line — not JSON, not YAML. This is not a style
preference, it is the same constraint stated three times in the tree already:
lib/users-config.sh:6-12, lib/runner-config.sh:6 and :24 — "a
rig-bootstrapped box has no YAML parser and no jq", which is why
json_field() is grep-and-sed. rig has never written a .json and should not
start with the one file that must be readable on the most broken machine in
the fleet. read parses this shape for free.

schema=1
bootstrapped_by=0.4.0
bootstrapped_at=2026-07-19T14:24:51Z
converged_by=0.6.0
converged_at=2026-08-02T09:11:03Z

Two pairs, both immutable in the sense that matters: birth — the rig that
first converged this machine, pinned forever — and latest — the newest rig
to have converged it. Kept for auditability: "is this machine converged by
something ancient?" is the question that actually sends you to the manifest,
and birth alone cannot answer it.

Both versions are captured at run time from $ROOT/VERSION — the version that
ran, not re-read later. That distinction is the point: a machine outlives the
rig that built it, so rig --version after an upgrade answers a different
question than either of these.

On a fresh machine both pairs are written with equal values. That is mild
redundancy in exchange for an audit file that never requires a reader to infer
a missing field, which is the right trade for this file.

Later commands may append their own provenance, subject to the same
immutability rule — the event of installing something, never its current
state:

runner_installed_at=2026-07-19T16:10:00Z
coolify_installed_at=2026-07-19T16:40:00Z
box_installed_at=2026-07-19T15:20:00Z
box_version=0.7.0

Note what does not go here: runner_labels, users_count, box_version as
a live fact. Those are observable now and belong to whatever command reports
them. .rig-labels (runner-install.sh:190) stays where it is.

Idempotency — dissolved, given two write rules

Moving specs out to #64 leaves nothing here that changes on its own, so the
cmp-guard needs no special case and the convergence contract holds for free.
Two rules in the writer are what keep it that way, and both are places where
the volatility problem could sneak back in through a side door.

Rule 1 — bootstrapped_* is first-write-wins. If the manifest exists and
already carries a bootstrapped_at, preserve both birth fields verbatim.
Regenerating the timestamp as now() on each run would make every re-run a
diff, which is exactly the trap the specs split was meant to close.

Rule 2 — converged_* updates only when the version actually differs.
This is the subtle one. converged_at must be the time the converging version
last changed, not the time of the last run. If it tracked every run it would
be a clock, and every re-run would diff again. So: compare $ROOT/VERSION
against the recorded converged_by; if equal, write nothing at all; if
different, update the pair together.

Under those rules a re-run by the same rig is byte-identical and the guard
stays silent, while a re-converge by a different rig produces a real diff —
and the guard firing there is correct rather than spurious. It was only ever
the clock that was the fake change, never the version.

Ordering

The manifest write inherits the marker's discipline verbatim
(bootstrap.sh:420-424): "Written only AFTER the tag verification, so a
marker never describes a box that failed to become what it claims." A
manifest that survives a failed run is worse than no manifest — it is a
confident wrong answer. Write last, and write nothing on a path that died.

How it is read back

rig manifest — prints the file, or rig manifest <key> for a single value
so shell callers do not re-parse. Dispatch alongside versions and use at
bin/rig:419-426.

rig platform (#64) should print these provenance lines above its
computed specs, so one command answers "what is this machine and what built
it" — but it reads them, it never writes them, and it degrades to specs-only
on a machine with no manifest.

Schema versioning

schema=1, an integer, independent of the rig versions recorded above. Bumped only when a key
is removed or repurposed — adding keys is not a bump, and readers must ignore
keys they do not know so a newer rig's manifest stays readable to an older
one. No schema= line means pre-manifest.

Relationship to heavy-duty/box#103

box#103 asks for the same kind of thing in its own context: what box built,
from what image, when, with which box version — stamped into Incus instance
config as user.box.*. This issue is the machine-level equivalent and the two
are independent. Same idea, two contexts; no shared schema, no shared
code, no ordering between them.

They only ever meet on a box-minted guest, and even there they cannot collide:
rig has no Incus socket, no incus exec and no ssh client anywhere in bin/
or commands/ — it runs on the target as root and can only touch that
machine's filesystem. It could not write user.box.* if it wanted to. The
rule worth writing down is just: rig never writes Incus config; box never
writes under /etc/rig.

On a Hetzner control-plane or the M900 there is no box layer at all, which is
the case this manifest is really for.

Open questions

Does the manifest subsume /etc/rig/role? No — the marker has six
readers including install.sh:82-90, and collapsing them is a separate,
riskier change. The two files coexist: the marker holds traits, the manifest
holds provenance.
Tenant boxes. bootstrap-tenant.sh writes a different marker shape
(role=%s tenant=yes host=no, :372) and deliberately declines to clobber
a machine marker (:370). Does a tenant get its own manifest, append to the
machine's, or skip?
Secret-free by construction. Nothing proposed here is a credential, but
the runner and coolify lines are where one would eventually creep in.
runner-install.sh:190's existing comment — "box-local metadata, never a
credential" — is the rule; it should be restated in the manifest writer.

Supersedes #52, which asked for this in one sentence.

# A rig-managed machine records nothing about which rig converged it, or when ## The whole record At the end of a bootstrap run, the entire durable output of rig is `commands/bootstrap.sh:427`: ```sh printf 'role=%s class=%s host=%s join=%s\n' "$ROLE" "$CLASS" "$HOST" "$JOIN" > "$MARKER_TMP" ``` One line in `/etc/rig/role`. That marker is doing exactly the job it was built for — it is the traits' ground truth, read by `warn_bootstrapped()` (`bin/rig:141-146`), the tenant guard (`bootstrap-tenant.sh:109-123`), `assert_marker_human()` (`lib/users-config.sh:120-147`) and four advisory sites. **This issue does not touch it.** What it does not record is provenance. SSH into a control-plane on Hetzner or the M900 runner six months on, and rig cannot tell you: - **which rig converged it.** `VERSION` is read in exactly one place, `bin/rig:9`, for `--version` — and that reports the *currently installed* tree, not the one that ran. A machine converged by `0.1.0-dev` and one converged by `0.4.0` are indistinguishable on disk. - **when.** There is no timestamp anywhere in the codebase. The only date read is `systemctl show ssh -p ExecMainStartTimestamp` (`users-close-root.sh:249`), used transiently and discarded. That is the whole gap this issue closes. Deliberately narrow. ## Scope: provenance only, and why that is the whole trick An earlier draft of this issue also proposed recording the machine's specs — cores, RAM, disk — in the same file. That was wrong, and the reason is worth stating because it is what makes the rest of this easy. **Specs change without rig doing anything.** Someone adds RAM, resizes the root disk, or the unattended-upgrades this very script enables (`bootstrap.sh:198-201`) patches the kernel. A stored spec is therefore either stale or in need of constant refresh — and a refresh on every run collides head-on with the contract at `bootstrap.sh:3`: *"Convergent: safe to re-run; a second run changes nothing."* That contract is enforced by a cmp-guard before every file install (nine sites: `bootstrap.sh:428`, `lib/sshd.sh:31`, `users-apply.sh:249`, `:300`, `bootstrap-tenant.sh:318`, and more), and I suspect it is exactly why no timestamp exists anywhere in the tree today. So: **facts that are decided go in the manifest; facts that are observed get computed at run time and stored nowhere.** Specs are observed — they belong to `rig platform` (#64), which calculates them fresh on every invocation and writes nothing. That split removes the volatility problem rather than managing it, and it leaves this file with only immutable content. ## Proposed manifest `/etc/rig/manifest`, 0644, alongside the marker and the users ledger. `/etc/rig/` already exists and `mkdir -p /etc/rig` is already open-coded in three places (`bootstrap.sh:429`, `bootstrap-tenant.sh:374`, `users-apply.sh:301`). **`key=value`, one per line — not JSON, not YAML.** This is not a style preference, it is the same constraint stated three times in the tree already: `lib/users-config.sh:6-12`, `lib/runner-config.sh:6` and `:24` — *"a rig-bootstrapped box has no YAML parser and no jq"*, which is why `json_field()` is grep-and-sed. rig has never written a `.json` and should not start with the one file that must be readable on the most broken machine in the fleet. `read` parses this shape for free. ``` schema=1 bootstrapped_by=0.4.0 bootstrapped_at=2026-07-19T14:24:51Z converged_by=0.6.0 converged_at=2026-08-02T09:11:03Z ``` Two pairs, both immutable in the sense that matters: **birth** — the rig that first converged this machine, pinned forever — and **latest** — the newest rig to have converged it. Kept for auditability: "is this machine converged by something ancient?" is the question that actually sends you to the manifest, and birth alone cannot answer it. Both versions are captured at run time from `$ROOT/VERSION` — the version that *ran*, not re-read later. That distinction is the point: a machine outlives the rig that built it, so `rig --version` after an upgrade answers a different question than either of these. On a fresh machine both pairs are written with equal values. That is mild redundancy in exchange for an audit file that never requires a reader to infer a missing field, which is the right trade for this file. Later commands may append their own provenance, subject to the same immutability rule — the *event* of installing something, never its current state: ``` runner_installed_at=2026-07-19T16:10:00Z coolify_installed_at=2026-07-19T16:40:00Z box_installed_at=2026-07-19T15:20:00Z box_version=0.7.0 ``` Note what does *not* go here: `runner_labels`, `users_count`, `box_version` as a live fact. Those are observable now and belong to whatever command reports them. `.rig-labels` (`runner-install.sh:190`) stays where it is. ## Idempotency — dissolved, given two write rules Moving specs out to #64 leaves nothing here that changes on its own, so the cmp-guard needs no special case and the convergence contract holds for free. Two rules in the writer are what keep it that way, and both are places where the volatility problem could sneak back in through a side door. **Rule 1 — `bootstrapped_*` is first-write-wins.** If the manifest exists and already carries a `bootstrapped_at`, preserve both birth fields verbatim. Regenerating the timestamp as `now()` on each run would make every re-run a diff, which is exactly the trap the specs split was meant to close. **Rule 2 — `converged_*` updates only when the version actually differs.** This is the subtle one. `converged_at` must be *the time the converging version last changed*, *not* the time of the last run. If it tracked every run it would be a clock, and every re-run would diff again. So: compare `$ROOT/VERSION` against the recorded `converged_by`; if equal, write nothing at all; if different, update the pair together. Under those rules a re-run by the same rig is byte-identical and the guard stays silent, while a re-converge by a *different* rig produces a real diff — and the guard firing there is correct rather than spurious. It was only ever the clock that was the fake change, never the version. ## Ordering The manifest write inherits the marker's discipline verbatim (`bootstrap.sh:420-424`): *"Written only AFTER the tag verification, so a marker never describes a box that failed to become what it claims."* A manifest that survives a failed run is worse than no manifest — it is a confident wrong answer. Write last, and write nothing on a path that died. ## How it is read back `rig manifest` — prints the file, or `rig manifest <key>` for a single value so shell callers do not re-parse. Dispatch alongside `versions` and `use` at `bin/rig:419-426`. `rig platform` (#64) should print these provenance lines above its computed specs, so one command answers "what is this machine and what built it" — but it reads them, it never writes them, and it degrades to specs-only on a machine with no manifest. ## Schema versioning `schema=1`, an integer, independent of the rig versions recorded above. Bumped only when a key is removed or repurposed — adding keys is not a bump, and readers must ignore keys they do not know so a newer rig's manifest stays readable to an older one. No `schema=` line means pre-manifest. ## Relationship to heavy-duty/box#103 box#103 asks for the same *kind* of thing in its own context: what box built, from what image, when, with which box version — stamped into Incus instance config as `user.box.*`. This issue is the machine-level equivalent and the two are **independent**. Same idea, two contexts; no shared schema, no shared code, no ordering between them. They only ever meet on a box-minted guest, and even there they cannot collide: rig has no Incus socket, no `incus exec` and no ssh client anywhere in `bin/` or `commands/` — it runs *on* the target as root and can only touch that machine's filesystem. It could not write `user.box.*` if it wanted to. The rule worth writing down is just: **rig never writes Incus config; box never writes under `/etc/rig`.** On a Hetzner control-plane or the M900 there is no box layer at all, which is the case this manifest is really for. ## Open questions - **Does the manifest subsume `/etc/rig/role`?** No — the marker has six readers including `install.sh:82-90`, and collapsing them is a separate, riskier change. The two files coexist: the marker holds traits, the manifest holds provenance. - **Tenant boxes.** `bootstrap-tenant.sh` writes a different marker shape (`role=%s tenant=yes host=no`, `:372`) and deliberately declines to clobber a machine marker (`:370`). Does a tenant get its own manifest, append to the machine's, or skip? - **Secret-free by construction.** Nothing proposed here is a credential, but the runner and coolify lines are where one would eventually creep in. `runner-install.sh:190`'s existing comment — *"box-local metadata, never a credential"* — is the rule; it should be restated in the manifest writer. Supersedes #52, which asked for this in one sentence.

dan-claude-bot commented

2026-07-20 00:02:07 +00:00

(Migrated from github.com)

Constraint on this issue's writer, found while reviewing #74 (rig platform, which reads the manifest this issue will write).

platform.sh gates its read on [ -r "$MANIFEST" ], so an unreadable manifest is reported as an absent one — a chmod-000 /etc/rig/manifest prints not bootstrapped (no /etc/rig/manifest), which is false.

That is deliberately not fixed in the reader: distinguishing absent from unreadable would mean guessing at a contract this issue has not written yet. The clean resolution is on the writer side —

Install the manifest 0644, as bootstrap.sh:580 already does for the role marker.

Then "unreadable" is not a state that occurs for non-root callers, and the reader's simple gate is correct rather than merely convenient. If instead the manifest is meant to hold anything privileged and be 0600, say so here, because then rig platform needs a third rendering ("bootstrapped, details need root") and that is a design decision rather than a bug fix.

Related, same source: platform.sh tolerates a manifest whose last line has no trailing newline (read returns 1 at EOF having already filled the variables — it silently dropped that line before #74 fixed it). The reader is now forgiving, but the writer should still terminate its final line; nothing should depend on the tolerance.

Refs #74, #64.

Constraint on this issue's writer, found while reviewing #74 (`rig platform`, which reads the manifest this issue will write). `platform.sh` gates its read on `[ -r "$MANIFEST" ]`, so an **unreadable** manifest is reported as an absent one — a chmod-000 `/etc/rig/manifest` prints `not bootstrapped (no /etc/rig/manifest)`, which is false. That is deliberately not fixed in the reader: distinguishing absent from unreadable would mean guessing at a contract this issue has not written yet. The clean resolution is on the writer side — **Install the manifest `0644`, as `bootstrap.sh:580` already does for the role marker.** Then "unreadable" is not a state that occurs for non-root callers, and the reader's simple gate is correct rather than merely convenient. If instead the manifest is meant to hold anything privileged and be `0600`, say so here, because then `rig platform` needs a third rendering ("bootstrapped, details need root") and that is a design decision rather than a bug fix. Related, same source: `platform.sh` tolerates a manifest whose last line has no trailing newline (`read` returns 1 at EOF having already filled the variables — it silently dropped that line before #74 fixed it). The reader is now forgiving, but the writer should still terminate its final line; nothing should depend on the tolerance. Refs #74, #64.

claude-lead-andresmgsl referenced this issue

2026-07-29 15:23:22 +00:00

feat: install channel is forge-agnostic (RIG_HOST / BOX_HOST) #114

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: heavy-duty/rig#61

No description provided.