From 5c40e906a1bf3a617618ac0e39e513321deb1e83 Mon Sep 17 00:00:00 2001 From: dan-claude-bot Date: Mon, 20 Jul 2026 10:05:13 +0000 Subject: [PATCH 1/2] =?UTF-8?q?feat:=20/etc/rig/manifest=20=E2=80=94=20whi?= =?UTF-8?q?ch=20rig=20converged=20this=20machine,=20and=20when?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A rig-managed machine recorded nothing about its own provenance. The entire durable output of a bootstrap run was one line in /etc/rig/role, and that line says what the box IS, never what built it. VERSION was read in exactly one place (bin/rig:9, for --version) and reports the currently INSTALLED tree, not the one that ran; there was no timestamp anywhere in the codebase. bootstrap now stamps a second file beside the marker: schema=1, a birth pair (bootstrapped_by/_at, pinned forever) and a latest pair (converged_by/_at). key=value, one per line, 0644 — the one file that must stay readable on the most broken machine in the fleet, where there is no YAML parser and no jq. `rig manifest []` reads it back. Only DECIDED facts go in, which is what keeps bootstrap.sh:3's convergence contract intact: bootstrapped_* is first-write-wins, and converged_* updates only when the version actually differs — it is the time the converging version last changed, not the time of the last run. The renderer is pure, so a re-run by the same rig is byte-identical no matter where the clock is, and the cmp-guard stays silent. OBSERVED facts (cores, RAM, disk, kernel) stay out: they go stale on their own and belong to `rig platform` (#64). /etc/rig/role is untouched. Closes #61 Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 31 +++++ README.md | 62 ++++++++++ bin/rig | 11 ++ commands/bootstrap-tenant.sh | 27 +++++ commands/bootstrap.sh | 27 +++++ commands/lib/manifest.sh | 185 ++++++++++++++++++++++++++++++ commands/manifest.sh | 88 +++++++++++++++ test/cli.sh | 213 +++++++++++++++++++++++++++++++++++ 8 files changed, 644 insertions(+) create mode 100644 commands/lib/manifest.sh create mode 100755 commands/manifest.sh diff --git a/CHANGELOG.md b/CHANGELOG.md index b191a11..359ee07 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -45,6 +45,37 @@ on the way to cutting its first release, and this file starts there. limits or the host's totals depends on whether `lxcfs` is in play — it is unverified, so those two lines are unreliable there. +- **`/etc/rig/manifest` records which rig converged a machine, and when** + (#61) — the entire durable output of a bootstrap run was one line in + `/etc/rig/role`, and that line is about what the box *is*, never about what + built it. `VERSION` was read in exactly one place (`bin/rig:9`, for + `--version`) and that reports the *currently installed* tree, not the one + that ran; there was no timestamp anywhere in the codebase. SSH into a + control plane six months on and a machine converged by `0.1.0-dev` was + indistinguishable from one converged by `0.4.0`. Bootstrap — both the + machine roles and the box tenant roles — now stamps a second file beside + the marker: `schema=1`, `bootstrapped_by`/`bootstrapped_at` (the rig that + *first* converged this machine, pinned forever) and + `converged_by`/`converged_at` (the newest rig to have converged it), read + back with a new `rig manifest []`. `key=value`, one per line, `0644` — + never JSON or YAML, because this is the one file that must stay readable on + the most broken machine in the fleet and a rig-bootstrapped box has no YAML + parser and no `jq`. + + Only **decided** facts go in, which is what keeps `bootstrap.sh:3`'s + contract ("a second run changes nothing", enforced by cmp-guards at nine + sites) intact: `bootstrapped_*` is first-write-wins, and `converged_*` + updates **only when the version actually differs** — it is the time the + converging version last changed, not the time of the last run. A naive + timestamp would have made every re-run a diff and had rig report a change it + did not make. **Observed** facts — cores, RAM, disk, kernel — are + deliberately absent: they go stale without rig doing anything, so they + belong to `rig platform` (#64), which computes them fresh and stores + nothing. `/etc/rig/role` is untouched — the marker holds traits and has six + readers; the manifest holds provenance. Readers must ignore keys they do not + know, and the writer preserves lines it does not own, so a manifest written + by a newer rig stays readable to (and survives a rewrite by) an older one. + ### Changed - **BREAKING: `--class human|server` is now `--root-door closed|open`** (#77) — diff --git a/README.md b/README.md index c0013d6..0521698 100644 --- a/README.md +++ b/README.md @@ -247,6 +247,13 @@ the marker later (`rig users` keys root policy off `root-door=`). Written post-join and cmp-guarded, so a marker never describes a box that failed to become what it claims. +Immediately after it, bootstrap stamps `/etc/rig/manifest` — **provenance**: +which rig converged this box and when (see [`rig +manifest`](#rig-manifest)). Same discipline, same guarantee, and the two files +stay consistent because they land together. The marker says what the box *is*; +the manifest says what *built* it. The tenant roles stamp it too — a +box-minted guest is a machine rig converged. + **`join=login` inverts the tag assertion.** A workstation joins as a user-owned device: there is no pre-auth key — a set `TS_AUTHKEY` is a loud usage error (exit 2; unset it, or pass `--join authkey`) — `tailscale up` @@ -1109,6 +1116,61 @@ them back, fresh keys and all. And the sudoers rules land in bad file under `/etc/sudoers.d` can take down *all* of sudo, locking every admin out of the very escalation path apply just granted. +### `rig manifest` + +```sh +rig manifest # the whole provenance record +rig manifest converged_by # one value, for a shell caller +``` + +Prints `/etc/rig/manifest` — **which rig converged this machine, and when**. +`rig bootstrap` writes it as its last durable act, beside the role marker; +this command only reads, needs no root (the file is `0644`), and works on a +machine whose rig has since been upgraded or removed. + +``` +schema=1 +bootstrapped_by=0.4.0 +bootstrapped_at=2026-07-19T14:24:51Z +converged_by=0.6.0 +converged_at=2026-08-02T09:11:03Z +``` + +Two pairs: **birth** — the rig that *first* converged this machine, pinned +forever — and **latest** — the newest rig to have converged it. On a fresh +machine they are equal. The version recorded is the one that **ran**, captured +at run time; `rig --version` reports the tree installed *now*, which after an +upgrade answers a different question, because a machine outlives the rig that +built it. + +Only **decided** facts live here, and that is what keeps bootstrap's +convergence contract intact. `bootstrapped_*` is first-write-wins; +`converged_*` moves **only when the version actually differs** — it records +the time the converging version last changed, not the time of the last run. A +re-run by the same rig therefore renders a byte-identical file and the +cmp-guard stays silent; a re-converge by a *different* rig is a real change and +the guard firing there is correct. + +**Observed** facts — cores, RAM, disk, kernel — are deliberately absent: they +go stale on their own (someone adds RAM; unattended-upgrades patches the +kernel), so storing them would either lie or force a rewrite on every run. They +belong to `rig platform`, which computes them fresh and stores nothing. + +`key=value`, one per line — never JSON, never YAML. This is the one file that +must stay readable on the most broken machine in the fleet, and a +rig-bootstrapped box has no YAML parser and no `jq`. Readers must ignore keys +they do not know, so `schema=` is bumped only when a key is removed or +repurposed; a manifest written by a newer rig stays readable to an older one, +and the writer preserves lines it does not own rather than eating them. Nothing +here is ever a credential. + +Exits 1 when there is no manifest — a machine converged before rig wrote one, +or never converged at all. `RIG_MANIFEST` overrides the path. + +The manifest does **not** replace `/etc/rig/role`. The marker holds *traits* +(what this box is) and has six readers; the manifest holds *provenance* (what +built it). Two files, two jobs. + ### `rig users status` ```sh diff --git a/bin/rig b/bin/rig index a077496..ec74693 100755 --- a/bin/rig +++ b/bin/rig @@ -91,6 +91,13 @@ commands: Shut root SSH on a class=human box once an admin key works. Refuses on class=server — root there is the control plane's automation door — and while no admin holds a key. Run as root. + manifest [] + Print /etc/rig/manifest — which rig converged this machine and when + (bootstrapped_by/_at pin the FIRST convergence forever; converged_by/_at + name the latest). With a key, print that value alone for shell callers. + Written by 'rig bootstrap' only; read-only here, and no root needed. + Provenance is DECIDED, so it is stored; the machine's specs are + OBSERVED, so they are not — those are 'rig platform'. versions List the installed rig versions — install.sh lands each one side by side at /versions/, a 'current' symlink tracks the default @@ -444,6 +451,10 @@ case "$cmd" in ;; esac ;; + manifest) + shift + exec "$ROOT/commands/manifest.sh" "$@" + ;; versions) shift cmd_versions "$@" diff --git a/commands/bootstrap-tenant.sh b/commands/bootstrap-tenant.sh index e65ca87..af78b2d 100755 --- a/commands/bootstrap-tenant.sh +++ b/commands/bootstrap-tenant.sh @@ -23,6 +23,8 @@ HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)" . "$HERE/lib/users-config.sh" # read_role_marker / root_door_of # shellcheck source=SCRIPTDIR/lib/sshd.sh . "$HERE/lib/sshd.sh" # harden_sshd (the staging-box tenant) +# shellcheck source=SCRIPTDIR/lib/manifest.sh +. "$HERE/lib/manifest.sh" # manifest_stamp — provenance, written beside the marker log() { printf 'rig-bootstrap: %s\n' "$*"; } warn() { printf 'rig-bootstrap: WARNING: %s\n' "$*" >&2; } @@ -405,6 +407,31 @@ else log "machine role marker present (${EXISTING_MARKER}); leaving it alone" fi +# --- provenance manifest ------------------------------------------------------ +# A tenant gets a manifest, in the SAME /etc/rig/manifest, through the same +# writer — and UNCONDITIONALLY, outside the marker gate above (#61's open +# question, answered here). +# +# The reason the marker needs that gate is that it holds TRAITS, and a guest's +# traits and the traits it earns after an operator-run `rig bootstrap workload` +# join are two different, competing statements about one box — so the marker +# has to pick, and it picks the truer one. Provenance has no such conflict. +# "Which rig converged this guest, and when" is a fact whichever bootstrap ran, +# and the two-pair shape composes across them exactly as designed: a staging +# guest later joined as a workload keeps the TENANT bootstrap as its birth — +# that genuinely is when this machine was first converged — and the machine +# bootstrap moves converged_* forward. Skipping the write on a joined guest +# would lose the birth stamp that only this run knows. +# +# One file, not a tenant-shaped second one: the manifest answers a question +# about the MACHINE, and a guest is a machine. The marker already carries +# `tenant=yes` for anyone who needs to know which kind. +if manifest_stamp "$(manifest_running_version "$HERE/..")"; then + log "provenance manifest written: $(manifest_path)" +else + log "provenance manifest already current" +fi + log "done — tenant ${ROLE}, user ${TENANT_USER}" if [ "$ROLE" = "staging-box" ]; then log "next (operator-run, holds a credential): box shell → sudo rig bootstrap workload-server --hostname with a tagged pre-auth key" diff --git a/commands/bootstrap.sh b/commands/bootstrap.sh index 316f7a1..92ce3af 100755 --- a/commands/bootstrap.sh +++ b/commands/bootstrap.sh @@ -10,6 +10,8 @@ HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)" . "$HERE/lib/sshd.sh" # harden_sshd — shared with the staging tenant # shellcheck source=SCRIPTDIR/lib/users-config.sh . "$HERE/lib/users-config.sh" # parse_users_file — the --users PRE-FLIGHT only +# shellcheck source=SCRIPTDIR/lib/manifest.sh +. "$HERE/lib/manifest.sh" # manifest_stamp — provenance, written beside the marker # The users lib is sourced for validation, never for convergence: `users apply` # stays the single owner of what a users file DOES to a box (#51). Bootstrap # borrows the parser so a typo'd users file is caught in the same breath as a @@ -619,6 +621,31 @@ else fi rm -f "$MARKER_TMP" +# --- provenance manifest ------------------------------------------------------ +# /etc/rig/manifest records WHICH rig converged this machine and WHEN (#61). +# The marker above says what the box IS; this says what BUILT it — two files, +# two jobs, and the marker is deliberately untouched (it has six readers, +# install.sh:82-90 among them). +# +# Placed HERE, immediately after the marker, so the two agree by construction +# and both inherit the marker's discipline verbatim: written only AFTER the tag +# verification, so neither ever describes a box that failed to become what it +# claims. A manifest that survives a failed run is worse than no manifest — it +# is a confident wrong answer. It deliberately does NOT trail the box install +# and the users phase below: those are the host EXTRA and the operator phase, +# and a box whose people failed to converge was still converged BY this rig at +# this time. Stamping provenance is not a claim that everything after it +# succeeded — the marker beside it makes exactly the same claim, and the two +# landing together is what keeps them readable as one statement. +# +# The version stamped is the one that RAN, captured now — not what `rig +# --version` would answer after a later upgrade. +if manifest_stamp "$(manifest_running_version "$HERE/..")"; then + log "provenance manifest written: $(manifest_path)" +else + log "provenance manifest already current" +fi + # --- box install (host=yes only) ------------------------------------------- # A host=yes box exists to run guest boxes, so bootstrap finishes the job rather # than printing a to-do: it installs the box CLI globally and lets box's OWN diff --git a/commands/lib/manifest.sh b/commands/lib/manifest.sh new file mode 100644 index 0000000..939f966 --- /dev/null +++ b/commands/lib/manifest.sh @@ -0,0 +1,185 @@ +#!/usr/bin/env bash +# /etc/rig/manifest — PROVENANCE: which rig converged this machine, and when +# (#61). Sourced by bootstrap.sh, bootstrap-tenant.sh and the test harness. +# +# The file is `key=value`, one per line — not JSON, not YAML. Same constraint +# stated three times in the tree already (lib/users-config.sh:6-12, +# lib/runner-config.sh:6 and :24): a rig-bootstrapped box has no YAML parser +# and no jq, which is why json_field() is grep-and-sed. This is the one file +# that must stay readable on the most broken machine in the fleet, so `read` +# parses it for free. +# +# WHAT GOES IN HERE: facts that are DECIDED. Which rig ran, and when it ran. +# Facts that are OBSERVED — cores, RAM, disk, kernel — belong to `rig platform` +# (#64), which computes them fresh and stores nothing. That split is not +# tidiness: a stored spec goes stale on its own (someone adds RAM; the +# unattended-upgrades bootstrap.sh itself enables patches the kernel), and +# refreshing it on every run collides head-on with bootstrap.sh:3 — +# "Convergent: safe to re-run; a second run changes nothing." Keeping only +# immutable content removes that problem instead of managing it. +# +# NEVER A CREDENTIAL. This file is 0644 and world-readable by design — it is +# an audit record, and an audit record nobody can read is not one. Later +# commands may append their own provenance (runner_installed_at, +# coolify_installed_at, box_version), subject to the same two rules: the EVENT +# of installing something, never its current state, and never a secret. Same +# law runner-install.sh:190 already states for `.rig-labels` — "box-local +# metadata, never a credential." + +# The schema version, an INTEGER, independent of the rig versions recorded in +# the file. Bumped only when a key is REMOVED or REPURPOSED — adding a key is +# not a bump, and readers must ignore keys they do not know, so a newer rig's +# manifest stays readable to an older one. No `schema=` line means pre-manifest. +MANIFEST_SCHEMA=1 + +# The five keys rig's provenance block owns. Everything else in the file is a +# later command's line and is preserved verbatim (see manifest_foreign). +MANIFEST_KEYS='schema bootstrapped_by bootstrapped_at converged_by converged_at' + +# manifest_path — where the manifest lives. RIG_MANIFEST overrides it so tests +# point at fixtures (repo precedent: RIG_ROLE_MARKER, bin/rig:148). +manifest_path() { + printf '%s' "${RIG_MANIFEST:-/etc/rig/manifest}" +} + +# manifest_value — the value of one key, or nothing when the file +# or the key is absent. First occurrence wins. NO policy here: what a missing +# key MEANS is each caller's call (this reader only reads — repo precedent: +# read_role_marker). +manifest_value() { + [ -r "$1" ] || return 0 + local k v + while IFS='=' read -r k v; do + [ "$k" = "$2" ] || continue + printf '%s' "$v" + return 0 + done < "$1" +} + +# manifest_has — is the key PRESENT, regardless of its value. +# Separate from manifest_value because "absent" and "present but empty" are +# different answers and command substitution collapses both to the empty +# string. Key comparison is a string equality, never a pattern: `rig manifest` +# passes operator input straight in, and a key of `.*` must find nothing rather +# than match the first line. +manifest_has() { + [ -r "$1" ] || return 1 + local k + while IFS='=' read -r k _; do + [ "$k" = "$2" ] && return 0 + done < "$1" + return 1 +} + +# manifest_foreign — every line rig's provenance block does NOT own, +# in file order. A newer rig (or a later command) may have written keys this +# one has never heard of; rewriting the file must not eat them, or the schema's +# "readers ignore keys they do not know" promise would only hold for readers +# and not for the writer. +manifest_foreign() { + [ -r "$1" ] || return 0 + local line key + while IFS= read -r line; do + [ -n "$line" ] || continue + key="${line%%=*}" + case " $MANIFEST_KEYS " in + *" $key "*) continue ;; + esac + printf '%s\n' "$line" + done < "$1" +} + +# manifest_render — the WHOLE convergence contract, as +# a pure text→text function: existing file + running version + a clock reading +# in, the file's desired content out. No side effects, so the harness proves +# the rules non-root against fixtures (repo precedent: parse_users_file, +# assert_marker_human). +# +# Purity is what makes convergence testable rather than asserted. The rendered +# content is a function of (existing file, running version) ALONE — is +# consulted only on the paths that were going to change anyway — so calling +# this twice with two DIFFERENT clock readings must produce byte-identical +# output. test/cli.sh pins exactly that, which is stronger than re-running the +# writer fast enough that the second matches by luck. +# +# Rule 1 — bootstrapped_* is FIRST-WRITE-WINS. Birth is pinned forever. If the +# file already carries a bootstrapped_at, both birth fields are preserved +# verbatim. Regenerating it as now() on every run would make every re-run a +# diff, which is the exact trap that keeping specs out of this file closed. +# +# Rule 2 — converged_* updates ONLY when the version actually differs. +# converged_at is "the time the converging version last changed", NOT the time +# of the last run. If it tracked every run it would be a clock, and a clock in +# a cmp-guarded file makes every re-run a fake change. So: compare the running +# version against the recorded converged_by; equal means the pair is already +# true and is copied through untouched. +# +# Under those two rules a re-run by the SAME rig renders byte-identical content +# and the cmp-guard stays silent, while a re-converge by a DIFFERENT rig +# renders a real diff — and the guard firing there is correct, not spurious. It +# was only ever the clock that was the fake change, never the version. +manifest_render() { + local path="$1" ver="$2" now="$3" + local b_by b_at c_by c_at + b_by="$(manifest_value "$path" bootstrapped_by)" + b_at="$(manifest_value "$path" bootstrapped_at)" + c_by="$(manifest_value "$path" converged_by)" + c_at="$(manifest_value "$path" converged_at)" + + # Rule 1. bootstrapped_at is the field that decides, because it is the one + # that can never be reconstructed: a machine's birth version can at least be + # guessed at, its birth INSTANT cannot. So an existing at-stamp pins the + # pair, and a birth-stamp with no birth version records `unknown` rather + # than backfilling today's version as if it had always been there — a + # manifest that lies about which rig built the box is worse than one that + # admits it does not know. + if [ -z "$b_at" ]; then + b_by="$ver"; b_at="$now" + elif [ -z "$b_by" ]; then + b_by=unknown + fi + + # Rule 2. The empty-at case is a one-time repair of a truncated file, not a + # clock: once written it satisfies the equality on every later run. + if [ "$c_by" != "$ver" ] || [ -z "$c_at" ]; then + c_by="$ver"; c_at="$now" + fi + + printf 'schema=%s\n' "$MANIFEST_SCHEMA" + printf 'bootstrapped_by=%s\n' "$b_by" + printf 'bootstrapped_at=%s\n' "$b_at" + printf 'converged_by=%s\n' "$c_by" + printf 'converged_at=%s\n' "$c_at" + manifest_foreign "$path" +} + +# manifest_running_version — the version that IS RUNNING, captured +# at run time from the tree's own VERSION file. NOT `rig --version` read back +# later: a machine outlives the rig that built it, so what is installed today +# answers a different question than what converged it. First line only — a +# stray second line would inject a bogus key into a key=value file. +manifest_running_version() { + local v="" + [ -r "$1/VERSION" ] && v="$(head -n1 "$1/VERSION")" + printf '%s' "${v:-unknown}" +} + +# manifest_stamp — the writer. Renders, cmp-guards like every file +# rig converges, installs 0644 beside the role marker and the users ledger. +# Returns 0 when it CHANGED the file and 1 when the file was already current, +# so the caller owns the log line (and, under set -e, must call it in an `if`). +manifest_stamp() { + local path tmp rc + path="$(manifest_path)" + tmp="$(mktemp)" + manifest_render "$path" "$1" "$(date -u +%Y-%m-%dT%H:%M:%SZ)" > "$tmp" + if cmp -s "$tmp" "$path" 2>/dev/null; then + rc=1 + else + mkdir -p "$(dirname "$path")" + install -m 0644 "$tmp" "$path" + rc=0 + fi + rm -f "$tmp" + return "$rc" +} diff --git a/commands/manifest.sh b/commands/manifest.sh new file mode 100755 index 0000000..8db8a4c --- /dev/null +++ b/commands/manifest.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash +# rig manifest — print this machine's provenance record, /etc/rig/manifest +# (#61). Reads only: this command NEVER writes the file. `rig bootstrap` is its +# single writer, and it stamps it as its own last durable act. +set -euo pipefail + +HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)" +# shellcheck source=SCRIPTDIR/lib/manifest.sh +. "$HERE/lib/manifest.sh" # manifest_path / manifest_value + +die() { printf 'rig-manifest: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; } + +usage() { + cat <<'EOF' +usage: rig manifest [] + +Print /etc/rig/manifest — which rig converged this machine, and when. With a +key, print that key's value alone, unquoted and newline-terminated, so a shell +caller does not re-parse the file: + + rig manifest schema=1 + bootstrapped_by=0.2.0 + bootstrapped_at=2026-07-19T14:24:51Z + converged_by=0.4.0 + converged_at=2026-08-02T09:11:03Z + + rig manifest converged_by 0.4.0 + +Both pairs are provenance, both immutable in the sense that matters: BIRTH — +the rig that first converged this machine, pinned forever — and LATEST — the +newest rig to have converged it. On a fresh machine the two are equal. Ask the +second pair "is this machine converged by something ancient?"; ask the first +"what built it". + +The version recorded is the one that RAN. `rig --version` reports the tree +installed NOW, which after an upgrade is a different question — a machine +outlives the rig that built it. + +Reads only. Needs no root (the file is 0644), no network, and works on a +machine whose rig has since been upgraded or removed. Specs — cores, RAM, +disk, kernel — are NOT here: they are observed rather than decided, so they go +stale on their own and belong to `rig platform`, which computes them fresh and +stores nothing. + +Exit 1 when there is no manifest — a machine converged before rig wrote one, +or never converged at all. + + RIG_MANIFEST override the path (default /etc/rig/manifest) +EOF +} + +KEY="" +for a in "$@"; do + case "$a" in + -h|--help) usage; exit 0 ;; + -*) + printf 'rig-manifest: unknown option: %s\n' "$a" >&2 + usage >&2 + exit 2 + ;; + *) + if [ -n "$KEY" ]; then + printf 'rig-manifest: manifest takes at most one key\n' >&2 + usage >&2 + exit 2 + fi + KEY="$a" + ;; + esac +done + +MPATH="$(manifest_path)" +[ -r "$MPATH" ] || die "no manifest at $MPATH — this machine has not been converged by a rig that writes one (rig bootstrap writes it)" + +if [ -z "$KEY" ]; then + cat "$MPATH" + exit 0 +fi + +# A key that is absent and a key whose value is empty are different answers to +# a shell caller, and $(...) collapses both to "". So the ABSENCE is the exit +# code, and only a present key ever prints — a caller reading `rig manifest +# converged_by` into a variable can trust that an empty result it accepted was +# a real empty value, not a missing key. +manifest_has "$MPATH" "$KEY" \ + || die "no such key: $KEY (keys present: $(cut -d= -f1 "$MPATH" | tr '\n' ' '))" +manifest_value "$MPATH" "$KEY" +echo diff --git a/test/cli.sh b/test/cli.sh index 566d047..d15183f 100644 --- a/test/cli.sh +++ b/test/cli.sh @@ -1907,6 +1907,219 @@ rm -f "$DUMP_TMP" check "no main-shell os-release sourcing" 1 "" \ grep -rnE '^[[:space:]]*\.[[:space:]]+/etc/os-release' "$ROOT/commands" +# --------------------------------------------------------------------------- +# /etc/rig/manifest — provenance (#61) +# +# The writer is a pure text→text renderer behind a cmp-guard, for the same +# reason assert_marker_human and parse_users_file are pure: the write itself +# sits behind bootstrap's root check, so every RULE is proven here, non-root, +# against fixtures — and the rules are the whole feature. +# --------------------------------------------------------------------------- +# Every helper sources the lib in a SUBSHELL, so the harness's own $PASS/$FAIL +# and the lib's globals never meet (repo precedent: the bash -c gates above, +# same isolation, one less quoting layer). +render() { # render + ( set -euo pipefail + . "$ROOT/commands/lib/manifest.sh" + manifest_render "$1" "$2" "$3" ) +} +stamp() { # stamp — the side-effecting writer + ( set -euo pipefail + . "$ROOT/commands/lib/manifest.sh" + RIG_MANIFEST="$1" manifest_stamp "$2" ) +} +running_version() { # running_version + ( set -euo pipefail + . "$ROOT/commands/lib/manifest.sh" + manifest_running_version "$1" ) +} +mode_of() { stat -c %a "$1"; } +mtime_of() { stat -c %Y "$1"; } +text_is() { [ "$(cat "$1")" = "$2" ]; } +MF="$(mktemp -d)" + +# -- the shape on a virgin machine ------------------------------------------ +check "manifest: a fresh render carries schema=1" 0 "schema=1" \ + render "$MF/absent" 0.4.0 2026-07-19T14:24:51Z +check "manifest: a fresh render pins birth to the running version" 0 "bootstrapped_by=0.4.0" \ + render "$MF/absent" 0.4.0 2026-07-19T14:24:51Z +check "manifest: a fresh render stamps birth with now" 0 "bootstrapped_at=2026-07-19T14:24:51Z" \ + render "$MF/absent" 0.4.0 2026-07-19T14:24:51Z +# Both pairs equal on a fresh machine: mild redundancy, in exchange for a file +# no reader ever has to infer a missing field from. +check "manifest: a fresh render writes latest equal to birth" 0 "converged_by=0.4.0" \ + render "$MF/absent" 0.4.0 2026-07-19T14:24:51Z +check "manifest: a fresh render stamps latest with now" 0 "converged_at=2026-07-19T14:24:51Z" \ + render "$MF/absent" 0.4.0 2026-07-19T14:24:51Z +# key=value, one per line, nothing else — the shape a machine with no jq and no +# YAML parser can read with `read`. Five lines, no quoting, no nesting. +render_is_flat_kv() { # render_is_flat_kv + local out + out="$(render "$1" "$2" 2026-07-19T14:24:51Z)" || return 1 + [ "$(printf '%s\n' "$out" | wc -l)" -eq 5 ] || return 1 + ! printf '%s\n' "$out" | grep -qvE '^[a-z_]+=[^ ]*$' +} +check "manifest: the render is bare key=value, one per line" 0 "" \ + render_is_flat_kv "$MF/absent" 0.4.0 + +# -- THE CRUX: convergence --------------------------------------------------- +# bootstrap.sh:3 promises "a second run changes nothing", enforced by a +# cmp-guard before every file install. A naive `converged_at=$(now)` would +# break that promise on every single re-run — the file would differ by a +# timestamp, the guard would fire, and rig would report a change it did not +# make. Rules 1 and 2 make the rendered content a function of (existing file, +# running version) alone. +# +# Proven the strong way: render the SAME fixture twice with two clock readings +# a year apart and diff. Byte-identical output means the clock cannot reach the +# file at all — a stronger claim than re-running the writer quickly enough that +# the two stamps happen to match by luck. +clock_cannot_reach() { # clock_cannot_reach + diff <(render "$1" "$2" 2027-01-01T00:00:00Z) <(render "$1" "$2" 2028-06-06T06:06:06Z) +} +reproduces_itself() { # reproduces_itself + diff <(render "$1" "$2" 2029-09-09T09:09:09Z) "$1" +} +BORN="$MF/born" +render "$MF/absent" 0.4.0 2026-07-19T14:24:51Z > "$BORN" +check "manifest: re-render by the SAME rig is byte-identical across a year of clock" 0 "" \ + clock_cannot_reach "$BORN" 0.4.0 +check "manifest: re-render by the same rig equals the file it read" 0 "" \ + reproduces_itself "$BORN" 0.4.0 +# The same property through the WRITER, which is what bootstrap actually calls: +# a first stamp writes (exit 0), a second by the same rig reports the file +# already current (exit 1) and touches nothing. +STAMPED="$MF/stamped" +check "manifest: the first stamp writes the file" 0 "" stamp "$STAMPED" 0.4.0 +check "manifest: the file landed 0644 — an audit record nobody can read is not one" \ + 0 "644" mode_of "$STAMPED" +BEFORE="$(cat "$STAMPED")" +check "manifest: a second stamp by the same rig reports already-current" 1 "" stamp "$STAMPED" 0.4.0 +check "manifest: a second stamp by the same rig changed no byte" 0 "" \ + text_is "$STAMPED" "$BEFORE" + +# -- a DIFFERENT rig: a real diff, and only where it belongs ---------------- +# The cmp-guard firing here is CORRECT, not spurious. It was only ever the +# clock that was the fake change, never the version. +check "manifest: a re-converge by a newer rig moves converged_by" 0 "converged_by=0.6.0" \ + render "$BORN" 0.6.0 2026-08-02T09:11:03Z +check "manifest: a re-converge by a newer rig moves converged_at" 0 "converged_at=2026-08-02T09:11:03Z" \ + render "$BORN" 0.6.0 2026-08-02T09:11:03Z +# Rule 1, the load-bearing half: birth is FIRST-WRITE-WINS. bootstrapped_by +# must survive every later convergence — "what built this box" is unanswerable +# by any other means once the run is over. +check "manifest: a re-converge leaves the birth version pinned" 0 "bootstrapped_by=0.4.0" \ + render "$BORN" 0.6.0 2026-08-02T09:11:03Z +check "manifest: a re-converge leaves the birth stamp pinned" 0 "bootstrapped_at=2026-07-19T14:24:51Z" \ + render "$BORN" 0.6.0 2026-08-02T09:11:03Z +check "manifest: the writer sees a version change as a real change" 0 "" stamp "$STAMPED" 0.6.0 +check "manifest: and settles again on the new version" 1 "" stamp "$STAMPED" 0.6.0 + +# -- a DOWNGRADE is a change too -------------------------------------------- +# converged_by is "the rig that last converged this", not "the highest one ever +# seen". Rolling back with `rig use` and re-converging must be recorded, or the +# file would name a version that is no longer what runs here. +check "manifest: re-converging with an OLDER rig is recorded, not ignored" 0 "converged_by=0.1.0" \ + render "$BORN" 0.1.0 2026-09-09T09:09:09Z + +# -- forward compatibility: unknown keys survive the rewrite ---------------- +# The schema promises readers ignore keys they do not know. That promise is +# worthless if the WRITER eats them: a manifest touched by a newer rig, or +# carrying a later command's own provenance line, must come back whole. +FOREIGN="$MF/foreign" +{ cat "$BORN"; printf 'runner_installed_at=2026-07-19T16:10:00Z\n'; printf 'schema_future_key=x\n'; } > "$FOREIGN" +check "manifest: a later command's provenance line survives a rewrite" 0 "runner_installed_at=2026-07-19T16:10:00Z" \ + render "$FOREIGN" 0.6.0 2026-08-02T09:11:03Z +check "manifest: a key from a newer schema survives a rewrite" 0 "schema_future_key=x" \ + render "$FOREIGN" 0.6.0 2026-08-02T09:11:03Z +# And preserving them must not cost convergence: a file carrying foreign keys is +# still byte-stable under a same-version re-render. +check "manifest: foreign keys do not break convergence" 0 "" \ + reproduces_itself "$FOREIGN" 0.4.0 + +# -- damaged files: repair once, then settle -------------------------------- +# A truncated or hand-edited manifest must converge back to a whole one and +# then STAY PUT — a repair that re-fires on every run is a clock by another +# name, and would break convergence exactly where it is hardest to notice. +printf 'bootstrapped_at=2020-01-01T00:00:00Z\n' > "$MF/noby" +check "manifest: a birth stamp with no birth version records unknown, never today's" \ + 0 "bootstrapped_by=unknown" render "$MF/noby" 0.4.0 2026-07-19T14:24:51Z +check "manifest: ...and still keeps the birth stamp it does have" \ + 0 "bootstrapped_at=2020-01-01T00:00:00Z" render "$MF/noby" 0.4.0 2026-07-19T14:24:51Z +printf 'schema=1\nbootstrapped_by=0.4.0\nbootstrapped_at=2020-01-01T00:00:00Z\nconverged_by=0.4.0\n' > "$MF/noat" +REPAIRED="$MF/repaired" +render "$MF/noat" 0.4.0 2026-07-19T14:24:51Z > "$REPAIRED" +check "manifest: a converged_by with no converged_at is repaired once" 0 "converged_at=2026-07-19T14:24:51Z" \ + cat "$REPAIRED" +check "manifest: the repair settles — it does not re-fire on the next run" 0 "" \ + reproduces_itself "$REPAIRED" 0.4.0 + +# -- the version that RAN, not the one installed now ------------------------ +# The whole point: a machine outlives the rig that built it, so this is read +# from the tree at run time and never re-derived from `rig --version` later. +check "manifest: the running version comes from the tree's own VERSION" 0 "$(cat "$ROOT/VERSION")" \ + running_version "$ROOT" +check "manifest: a tree with no VERSION records unknown, not an empty key" 0 "unknown" \ + running_version "$MF" + +# -- the marker is NOT touched ---------------------------------------------- +# /etc/rig/role has six readers, install.sh:82-90 among them; the manifest is a +# second file beside it, never a replacement. Assert the writer cannot reach it. +check "manifest: no CODE in the manifest lib reaches the role marker" 1 "" \ + grep -nE '^[^#]*(/etc/rig/role|RIG_ROLE_MARKER)' "$ROOT/commands/lib/manifest.sh" +check "bootstrap: the role marker write is still its own cmp-guarded block" 0 "" \ + grep -qE '^MARKER=/etc/rig/role$' "$ROOT/commands/bootstrap.sh" + +# -- ordering: provenance is written after the tag verification ------------- +# The marker's discipline, inherited verbatim — a manifest that survives a run +# which failed to become what it claims is a confident wrong answer. +mfstamp_at="$(grep -nE '^if manifest_stamp ' "$ROOT/commands/bootstrap.sh" | head -n1 | cut -d: -f1)" +verify_at="$(grep -nE '^[[:space:]]*verify_effective_tag back-out$' "$ROOT/commands/bootstrap.sh" | head -n1 | cut -d: -f1)" +check "bootstrap: both ordering anchors were found (guards the greps above)" 0 "" \ + test -n "${mfstamp_at:-}" -a -n "${verify_at:-}" +check "bootstrap: the manifest stamp follows the tag verification" 0 "" \ + test "${mfstamp_at:-0}" -gt "${verify_at:-999999}" +# Both bootstrap paths stamp it: a box-minted guest is a machine rig converged, +# and "which rig, when" is a fact whichever bootstrap ran. +check "bootstrap-tenant: a tenant gets a manifest too" 0 "" \ + grep -q 'manifest_stamp' "$ROOT/commands/bootstrap-tenant.sh" + +# -- `rig manifest`, the reader --------------------------------------------- +key_prints_exactly() { # key_prints_exactly + [ "$(RIG_MANIFEST="$1" "$ROOT/commands/manifest.sh" "$2")" = "$3" ] +} +check "manifest: --help exits 0" 0 "usage: rig manifest" "$ROOT/commands/manifest.sh" --help +check "manifest: dispatches through bin/rig" 0 "usage: rig manifest" "$ROOT/bin/rig" manifest --help +check "manifest: rig --help lists the command" 0 "manifest []" "$ROOT/bin/rig" --help +check "manifest: unknown flag exits 2" 2 "unknown option" \ + env RIG_MANIFEST="$STAMPED" "$ROOT/commands/manifest.sh" --nope +check "manifest: two keys is a usage error" 2 "at most one key" \ + env RIG_MANIFEST="$STAMPED" "$ROOT/commands/manifest.sh" converged_by schema +check "manifest: an absent manifest exits 1 by name" 1 "no manifest at" \ + env RIG_MANIFEST="$MF/absent" "$ROOT/commands/manifest.sh" +check "manifest: bare prints the file" 0 "converged_by=0.6.0" \ + env RIG_MANIFEST="$STAMPED" "$ROOT/commands/manifest.sh" +check "manifest: a key prints the value ALONE, for shell callers" 0 "" \ + key_prints_exactly "$STAMPED" converged_by 0.6.0 +check "manifest: an unknown key exits 1 and names the keys present" 1 "keys present" \ + env RIG_MANIFEST="$STAMPED" "$ROOT/commands/manifest.sh" nosuchkey +# Operator input reaches the key lookup, so the lookup is a string equality and +# never a pattern — a key of '.*' must find nothing rather than match line one. +check "manifest: a regex-shaped key matches nothing" 1 "no such key" \ + env RIG_MANIFEST="$STAMPED" "$ROOT/commands/manifest.sh" '.*' +# The reader writes NOTHING — bootstrap is the manifest's single writer. +MTIME_BEFORE="$(mtime_of "$STAMPED")" +RIG_MANIFEST="$STAMPED" "$ROOT/commands/manifest.sh" >/dev/null 2>&1 || true +check "manifest: reading it does not write it" 0 "$MTIME_BEFORE" mtime_of "$STAMPED" + +# Secrets: this file is 0644 by design, so the rule has to be stated where the +# next command that appends a line will read it (repo precedent: +# runner-install.sh:190's ".rig-labels — box-local metadata, never a credential"). +check "manifest: the never-a-credential rule is stated in the writer" 0 "never a credential" \ + cat "$ROOT/commands/lib/manifest.sh" + +rm -rf "$MF" + # --------------------------------------------------------------------------- # The versioned install (box#79's layout, ported — #35). RIG_INSTALL_SOURCE # bypasses the network, so these are REAL runs of install.sh against throwaway -- 2.45.2 From d2e03ce993c93c0c0cfb182a08294c9cb43fac7c Mon Sep 17 00:00:00 2001 From: dan-claude-bot Date: Mon, 20 Jul 2026 11:27:55 +0000 Subject: [PATCH 2/2] fix: read a manifest's final line when the file has no trailing newline MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit manifest_value, manifest_has and manifest_foreign each read the file with a bare `while read`, which stops at EOF without ever handing over a populated partial line. An unterminated final line therefore read as ABSENT — and absent is exactly the input both convergence rules key off, so the file's last line was the one least able to survive the miss. Three failures, in descending order of how much they cost: - A file truncated mid-write ends AT bootstrapped_at, so the unreadable line is the birth stamp itself. Rule 1 saw no at-stamp and regenerated the pair as now() — overwriting the one field that can never be reconstructed. A fixture born 2020-01-01 came back stamped with the current clock. - A whole file whose last line is converged_at read as empty, so Rule 2's one-time repair re-fired on EVERY run: the render stopped being a function of (existing file, running version) and the clock reached the file after all. This is the crux property of the feature, broken by a missing byte. - manifest_foreign dropped an unterminated foreign line entirely, so the writer ate a later command's provenance — the exact preservation contract the function exists to keep. The idiom is the repo's own: lib/users-config.sh:49 reads `|| [ -n "$line" ]` for the same reason. Reading the line correctly also repairs the file, since the rewritten copy is newline-terminated — asserted as "the source plus the newline it was missing, and nothing else", because a plain ends-in-newline check stays green on an implementation that drops the final line. 7 tests, each observed RED against the unfixed reader. Co-Authored-By: Claude Opus 4.8 --- commands/lib/manifest.sh | 6 ++--- test/cli.sh | 54 ++++++++++++++++++++++++++++++++++++++++ 2 files changed, 57 insertions(+), 3 deletions(-) diff --git a/commands/lib/manifest.sh b/commands/lib/manifest.sh index 939f966..ebc933b 100644 --- a/commands/lib/manifest.sh +++ b/commands/lib/manifest.sh @@ -49,7 +49,7 @@ manifest_path() { manifest_value() { [ -r "$1" ] || return 0 local k v - while IFS='=' read -r k v; do + while IFS='=' read -r k v || [ -n "$k" ]; do [ "$k" = "$2" ] || continue printf '%s' "$v" return 0 @@ -65,7 +65,7 @@ manifest_value() { manifest_has() { [ -r "$1" ] || return 1 local k - while IFS='=' read -r k _; do + while IFS='=' read -r k _ || [ -n "$k" ]; do [ "$k" = "$2" ] && return 0 done < "$1" return 1 @@ -79,7 +79,7 @@ manifest_has() { manifest_foreign() { [ -r "$1" ] || return 0 local line key - while IFS= read -r line; do + while IFS= read -r line || [ -n "$line" ]; do [ -n "$line" ] || continue key="${line%%=*}" case " $MANIFEST_KEYS " in diff --git a/test/cli.sh b/test/cli.sh index d15183f..f7b6695 100644 --- a/test/cli.sh +++ b/test/cli.sh @@ -2054,6 +2054,60 @@ check "manifest: a converged_by with no converged_at is repaired once" 0 "conver check "manifest: the repair settles — it does not re-fire on the next run" 0 "" \ reproduces_itself "$REPAIRED" 0.4.0 +# -- a final line with NO trailing newline ---------------------------------- +# A bare `while read` stops at EOF without ever handing over a populated +# partial line, so the last record of an unterminated file reads as ABSENT. +# That is not a cosmetic parse miss here: absent is exactly the input both +# rules key off, so the file's last line is the one least able to survive it. +# A hand-edit with an editor that adds no final newline, or a truncated write, +# is enough to produce one. The reader idiom is the repo's own — +# lib/users-config.sh:49 reads `|| [ -n "$line" ]` for the same reason. +NONL="$MF/nonl-owned" +printf 'schema=1\nbootstrapped_by=0.4.0\nbootstrapped_at=2020-01-01T00:00:00Z\nconverged_by=0.4.0\nconverged_at=2020-01-01T00:00:00Z' > "$NONL" +# The crux assertion, applied to the case that broke it: with converged_at +# unreadable, Rule 2 saw an empty at-stamp and re-fired the "one-time" repair +# on EVERY run, so the clock reached the file after all. +check "manifest: an unterminated final line does not let the clock back in" 0 "" \ + clock_cannot_reach "$NONL" 0.4.0 +check "manifest: an unterminated converged_at is read, not re-stamped" 0 "converged_at=2020-01-01T00:00:00Z" \ + render "$NONL" 0.4.0 2026-07-19T14:24:51Z +# Rule 1 on the field that can never be reconstructed: a file truncated mid-way +# ends AT bootstrapped_at, so the unterminated line is the birth stamp itself — +# and regenerating it is the one loss no later run can undo. +printf 'schema=1\nbootstrapped_by=0.4.0\nbootstrapped_at=2020-01-01T00:00:00Z' > "$MF/nonl-birth" +check "manifest: an unterminated birth stamp stays pinned, not reborn today" 0 "bootstrapped_at=2020-01-01T00:00:00Z" \ + render "$MF/nonl-birth" 0.4.0 2026-07-19T14:24:51Z +# And the preservation contract, whose whole subject is the file's tail: a +# later command's line is very often the last one written. +NONLF="$MF/nonl-foreign" +printf 'schema=1\nbootstrapped_by=0.4.0\nbootstrapped_at=2020-01-01T00:00:00Z\nconverged_by=0.4.0\nconverged_at=2020-01-01T00:00:00Z\nrunner_installed_at=2026-07-19T16:10:00Z' > "$NONLF" +check "manifest: an unterminated FOREIGN final line is not eaten by the rewrite" 0 "runner_installed_at=2026-07-19T16:10:00Z" \ + render "$NONLF" 0.4.0 2026-07-19T14:24:51Z +# Reading it correctly also REPAIRS it: the rewritten copy is newline-terminated, +# so an unterminated file converges to a terminated one exactly once and then +# reproduces itself like any other. Asserted as "the source, plus the newline it +# was missing, and NOTHING else" — a plain does-it-end-in-\n check would stay +# green on an implementation that dropped the final line, since a file with the +# tail eaten is newline-terminated too. +NORMALIZED="$MF/normalized" +render "$NONLF" 0.4.0 2026-07-19T14:24:51Z > "$NORMALIZED" +adds_only_the_newline() { # adds_only_the_newline + diff <(cat "$1"; printf '\n') "$2" +} +check "manifest: the rewrite adds the missing final newline and changes nothing else" 0 "" \ + adds_only_the_newline "$NONLF" "$NORMALIZED" +check "manifest: ...and the normalized file then settles" 0 "" \ + reproduces_itself "$NORMALIZED" 0.4.0 +# Presence, not just value: manifest_has answers the absent-vs-empty question +# `rig manifest ` puts in its exit code, and it read the same short file. +has_key() { # has_key + ( set -euo pipefail + . "$ROOT/commands/lib/manifest.sh" + manifest_has "$1" "$2" ) +} +check "manifest: an unterminated final key is PRESENT, not absent" 0 "" \ + has_key "$NONL" converged_at + # -- the version that RAN, not the one installed now ------------------------ # The whole point: a machine outlives the rig that built it, so this is read # from the tree at run time and never re-derived from `rig --version` later. -- 2.45.2