From 92fa2a98606cfc481d1cef9d3a0952e782728393 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 17 Jul 2026 15:27:09 +0000 Subject: [PATCH] bootstrap: infer the tailnet tag from the pre-auth key, verify the granted tag MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit rig used to pass --ts-tag to `tailscale up --advertise-tags`, stating the tailnet tag a second time with no way to know whether its request and the key's own tags agreed. It asserted the tag it REQUESTED, never the tag control GRANTED — the sshd first-wins bug in a different hat, and the same scar (both M900s joined tag:server, retagged by hand, unnoticed). Collapse the two sources of truth onto one: the key. - `tailscale up` drops --advertise-tags; the key's tags apply. - After join, poll `tailscale status --json` for `.Self.Tags` (netmap ground truth, not `debug prefs`) until tags appear or BackendState=Running, on BOTH the fresh-join and already-joined paths. - UNTAGGED -> hard refusal: `tailscale logout` to back the user-owned node out, then die naming the fix (mint a tagged key). - Role policy moves onto the effective tag: a runner must not have tag:server among the tags the key actually granted. Strictly stronger than before. - --ts-tag is removed, and dies exit 2 with a message pointing at the key (consuming its value), not an "unknown flag". - New array-aware reader json_string_array in lib/runner-config.sh (jq-free, never fails under set -e), with its own unit tests; bootstrap sources the lib. Co-Authored-By: Claude Opus 4.8 --- README.md | 54 +++++++++++++---- commands/bootstrap.sh | 106 +++++++++++++++++++++++++++------- commands/lib/runner-config.sh | 28 +++++++++ test/cli.sh | 72 ++++++++++++++++++++++- 4 files changed, 227 insertions(+), 33 deletions(-) diff --git a/README.md b/README.md index 36e0fd1..cdcdd43 100644 --- a/README.md +++ b/README.md @@ -32,15 +32,19 @@ rig bootstrap runner --hostname my-ci-box ``` - `--hostname ` — tailnet hostname (default: the role name) -- `--ts-tag ` — tailnet tag to advertise (default: `tag:server`; - the `runner` role defaults to `tag:ci` instead, and **refuses** - `tag:server` outright — see below) + +There is **no `--ts-tag` flag**. A pre-auth key is minted *with* its tags, so +the key is the single source of truth for the tailnet tag — rig no longer states +a second one it might disagree with. It **verifies** the tag control actually +granted after join instead (see below). Passing `--ts-tag` now exits 2 with a +message pointing you at the key. What it does: installs `curl ca-certificates unattended-upgrades` (and enables periodic unattended upgrades); writes an sshd hardening drop-in (`PermitRootLogin prohibit-password`, `PasswordAuthentication no`) and **verifies it took effect** via `sshd -T`; sets the system hostname; installs -tailscale and joins your tailnet. +tailscale and joins your tailnet — then **verifies the tag the key granted** +(see *The tag comes from the key* below). **`--hostname` converges both names.** On a box that has already joined, `bootstrap` skips `tailscale up` (so a re-run needs no pre-auth key) — but it @@ -64,17 +68,43 @@ admin console keeps that name; rig will not fight it. > on re-run, and refuses to claim success unless `sshd -T` agrees. **The pre-auth key:** provide it via the `TS_AUTHKEY` env var or type it at -the interactive prompt. Use a **single-use, tagged, short-expiry** key. It -lives in process memory only — rig never writes a credential to disk. +the interactive prompt. Use a **single-use, tagged, short-expiry** key — the +**tagged** part is now load-bearing, not advice (see below). It lives in process +memory only — rig never writes a credential to disk. + +**The tag comes from the key, and rig verifies the one control granted.** rig +used to pass `--ts-tag` to `tailscale up --advertise-tags`, stating the tag a +*second* time — with no way to know whether its request and the key's own tags +agreed. It asserted the tag it **requested**, never the tag control **granted**; +this is the same shape as the sshd first-wins bug above, and it left the same +scar (both M900s joined carrying `tag:server` and had to be retagged by hand, +because nothing in rig ever read the effective tag back). So rig stops +overriding the key: `tailscale up` carries no `--advertise-tags`, the key's tags +apply, and after join rig polls `tailscale status --json` for `.Self.Tags` — the +netmap's ground truth, not `tailscale debug prefs`, which prints what was +*requested* — and asserts on that, on **first join and on every re-run** (which +catches a box bootstrapped before this change, or retagged behind rig's back). + +> **An untagged key is a hard refusal.** Drop `--advertise-tags` and you also +> drop the accidental net that used to tag an untagged key's node anyway. An +> untagged node joins owned by the *key creator's user identity* — it inherits +> that human's ACL grants, expires with the key, and vanishes if the account is +> deleted. That is a fleet-shaped mistake, not a warning: rig runs `tailscale +> logout` to back the half-joined node out and dies telling you to mint a tagged +> key. A wrong tag **cannot** be fixed in place either — `tailscale set` has no +> tag flag, re-tagging needs a fresh key via `up --force-reauth` — so rig detects +> and refuses, and never claims a convergence it cannot perform. `control-plane` and `workload` are identical today except the default hostname; they exist because the boxes diverge over time, and because each -follow-up command applies to exactly one role. `runner` is the box a CI -agent will live on, and it differs behaviorally: it defaults `--ts-tag` to -`tag:ci` and **refuses `tag:server`** — a runner executes repo-controlled -code, and advertising your server tag would extend every grant your servers -hold (SSH between them, say) to that code. The refusal turns the worst -misconfiguration from a documentation warning into a hard error. +follow-up command applies to exactly one role. `runner` is the box a CI agent +will live on, and it differs behaviorally: it **refuses `tag:server`**. That +refusal moved onto the *effective* tag and is strictly stronger for it — it is +no longer "don't advertise `tag:server`" but "the key you actually used must not +grant `tag:server` to repo-controlled code." A runner executes that code, and +`tag:server`'s grants (SSH between your servers, say) must never extend to it; +the check turns the worst misconfiguration from a documentation warning into a +hard, post-join error. ### `rig coolify install --version ` diff --git a/commands/bootstrap.sh b/commands/bootstrap.sh index 22e171d..45f8092 100755 --- a/commands/bootstrap.sh +++ b/commands/bootstrap.sh @@ -3,19 +3,25 @@ # Convergent: safe to re-run; a second run changes nothing. set -euo pipefail +HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)" +# shellcheck source=SCRIPTDIR/lib/runner-config.sh +. "$HERE/lib/runner-config.sh" # json_field / json_string_array read the netmap + log() { printf 'rig-bootstrap: %s\n' "$*"; } warn() { printf 'rig-bootstrap: WARNING: %s\n' "$*" >&2; } die() { printf 'rig-bootstrap: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; } usage() { cat <<'EOF' -usage: rig bootstrap [--hostname ] [--ts-tag ] +usage: rig bootstrap [--hostname ] --hostname system + tailnet hostname (default: the role name) - --ts-tag tailnet tag to advertise (default: tag:server; - role runner defaults to tag:ci and refuses tag:server — - a CI box executes repo-controlled code, and your server - tag's grants must never extend to it) + +The tailnet tag is NOT a rig argument. A pre-auth key is minted WITH its tags, +so the key is the single source of truth: rig no longer requests a tag it might +disagree with. After the box joins, rig reads the tag control actually GRANTED +(tailscale status .Self.Tags) and asserts on THAT — an untagged key is refused +outright, and a runner may not carry tag:server. Mint a correctly-tagged key. Provide the single-use tailscale pre-auth key via the TS_AUTHKEY env var, or enter it at the interactive prompt. It is used once and never written to disk. @@ -32,29 +38,25 @@ case "$ROLE" in esac TS_HOSTNAME="$ROLE" -if [ "$ROLE" = "runner" ]; then - TS_TAG="tag:ci" -else - TS_TAG="tag:server" -fi while [ $# -gt 0 ]; do case "$1" in --hostname) [ $# -ge 2 ] || die "--hostname needs a value" 2 TS_HOSTNAME="$2"; shift 2 ;; --ts-tag) - [ $# -ge 2 ] || die "--ts-tag needs a value" 2 - TS_TAG="$2"; shift 2 ;; + # --ts-tag is GONE, but this is a deliberate death with a message, not an + # "unknown flag": the flag shipped for a month and scripts still pass it, + # so it must explain where the tag went rather than look like a typo. The + # tag is now the key's to state and rig's to verify after join (issue #16 — + # the tag was said twice and rig never checked the two agreed). Consume a + # following value if present so `--ts-tag tag:server` dies on the flag and + # its argument never lands in the *) arm as a mystery unknown flag. + [ $# -ge 2 ] && shift + die "--ts-tag is removed: the tailnet tag comes from the pre-auth key now, not rig. Mint a key with the tag you want; rig verifies the granted tag after join." 2 ;; *) die "unknown flag: $1" 2 ;; esac done -# A runner executes repo-controlled code; advertising the server tag would -# extend every grant your servers hold to that code. Refused, not warned. -if [ "$ROLE" = "runner" ] && [ "$TS_TAG" = "tag:server" ]; then - die "role runner must not advertise tag:server" 2 -fi - # --- guards ------------------------------------------------------------------ [ "$(id -u)" -eq 0 ] || die "must run as root" if [ -r /etc/os-release ]; then @@ -164,6 +166,59 @@ else fi # --- tailscale ---------------------------------------------------------------- +# verify_effective_tag — assert the tag control actually GRANTED this node, never +# the one rig requested. This is the sshd `sshd -T` lesson wearing a tailnet hat: +# rig used to advertise a tag and trust it took, exactly as it once trusted that +# a drop-in FILE existing meant sshd had read it. Both M900s joined carrying +# tag:server and had to be retagged by hand; nothing in rig noticed because +# nothing ever read the effective tag back. +# +# `.Self.Tags` from `tailscale status --json` is the netmap's ground truth. +# `tailscale debug prefs` would LIE here — it prints AdvertiseTags, i.e. what was +# REQUESTED — which is precisely the second source of truth issue #16 deletes. +# Tags ride in with the netmap, not synchronously out of `up`, so a single read +# right after join can legitimately come back empty; poll until tags appear OR +# the backend reaches Running (past which an empty Tags is real, not just early). +verify_effective_tag() { + local deadline=$((SECONDS + 30)) tags="" state="" json + json="$(mktemp)" + while :; do + if tailscale status --json > "$json" 2>/dev/null; then + tags="$(json_string_array "$json" Tags)" + state="$(json_field "$json" BackendState)" + if [ -n "$tags" ] || [ "$state" = "Running" ]; then break; fi + fi + if [ "$SECONDS" -ge "$deadline" ]; then break; fi + sleep 2 + done + rm -f "$json" + + # UNTAGGED is the real hazard and it is silent: with no tags the node joined + # owned by the KEY CREATOR's user identity — it inherits that human's ACL + # grants, expires with the key, and vanishes if the account is deleted. + # Dropping --advertise-tags removed the accidental net that used to tag such a + # node anyway, so rig must now catch this out loud. A wrong tag cannot be fixed + # in place (`tailscale set` has no tag flag; re-tagging needs a fresh key via + # `up --force-reauth`), so back the node out rather than leave a half-joined, + # user-owned device squatting a hostname. + if [ -z "$tags" ]; then + tailscale logout >/dev/null 2>&1 \ + || warn "tailscale logout failed — this node is joined UNTAGGED and user-owned; remove it from the tailnet by hand" + die "joined with NO tag: the pre-auth key was untagged, so this node is owned by the key creator's user identity, not a tag. Backed it out. Fix: mint a TAGGED pre-auth key and re-run." + fi + + # Role policy now rides the EFFECTIVE tag — strictly stronger than the old + # request-time check, which only guarded the tag rig HOPED for. This guards the + # tag the key ACTUALLY granted to repo-controlled code: a runner carrying + # tag:server would extend every grant your servers hold to CI code. Refused, + # never warned. rig can DETECT this but cannot FIX it, so name the repair. + if [ "$ROLE" = "runner" ] && printf '%s\n' "$tags" | grep -qx 'tag:server'; then + die "role runner joined with tag:server (effective tags: $(printf '%s' "$tags" | tr '\n' ' ')). The key you used grants tag:server to repo-controlled code; that must never happen. Re-run bootstrap with a key minted for a CI tag (e.g. tag:ci)." + fi + + log "verified effective tailnet tag(s): $(printf '%s' "$tags" | tr '\n' ' ')" +} + if ! command -v tailscale >/dev/null 2>&1; then log "installing tailscale" curl -fsSL https://tailscale.com/install.sh | sh @@ -190,6 +245,12 @@ if tailscale status >/dev/null 2>&1; then else log "tailnet hostname already ${TS_HOSTNAME}" fi + # Verify the tag on the already-joined path too, not only on first join: this + # catches a box bootstrapped BEFORE rig looked at tags, or one retagged behind + # rig's back, on the very next ordinary re-run. Skipping `tailscale up` here is + # deliberate and stays — re-running an identical tagged-authkey `up` errors — + # but skipping the CHECK was how the M900s stayed mis-tagged unnoticed. + verify_effective_tag else # env override, else prompt; never touches disk if [ -z "${TS_AUTHKEY:-}" ]; then @@ -197,8 +258,13 @@ else echo fi [ -n "${TS_AUTHKEY:-}" ] || die "empty pre-auth key" - log "joining tailnet as ${TS_HOSTNAME} (${TS_TAG})" - tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME" --advertise-tags="$TS_TAG" + # No --advertise-tags: the key's own tags apply (documented default for a + # tagged key), and rig verifies them below instead of stating a second tag it + # cannot reconcile with the key's. A tagged key needs no flag; an untagged one + # cannot be rescued by one (verify_effective_tag refuses it and logs out). + log "joining tailnet as ${TS_HOSTNAME} (tag comes from the pre-auth key)" + tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME" + verify_effective_tag fi log "done — role ${ROLE}, hostname ${TS_HOSTNAME}" diff --git a/commands/lib/runner-config.sh b/commands/lib/runner-config.sh index e8aaf0a..cef94d6 100644 --- a/commands/lib/runner-config.sh +++ b/commands/lib/runner-config.sh @@ -14,6 +14,34 @@ json_field() { | head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//' || true } +# json_string_array — the elements of the FIRST array named , +# one per line, empty when the key is absent or the array is empty. +# +# json_field's sibling for the one shape it cannot read: `.Self.Tags` from +# `tailscale status --json` is a JSON array, and bootstrap must assert on it to +# learn the tag control actually GRANTED the node (the netmap's ground truth), +# not the tag rig requested. Same grep/sed spirit, same jq-free reason: a +# rig-bootstrapped box has no jq and we will not install one to read one field. +# +# `tr -d '\n'` first, because tailscale pretty-prints its JSON and an array +# spans lines — grep is line-oriented and would never see `[ ... ]` whole +# otherwise. `\[[^]]*\]` then captures the first flat array body for +# (tag strings never contain `]`, so this is safe); the inner `grep -o` pulls +# every quoted token out of it, and `sed 1d` drops the key's own name — which +# `"key":[...]` leads with — leaving just the elements. +# +# FIRST array wins by design, and the caller leans on it: `tailscale status +# --json` emits Self before Peer (Go struct field order, stable), so the first +# "Tags" is the node's OWN, never a peer's. An absent key omits itself entirely +# (Go's omitempty) rather than emitting `[]` — which is exactly the untagged, +# user-owned node bootstrap must catch. Never fails under `set -e`+pipefail: a +# non-match is a fact to test for, like json_field, not a reason to die. +json_string_array() { + tr -d '\n' < "$1" 2>/dev/null \ + | grep -o "\"$2\"[[:space:]]*:[[:space:]]*\[[^]]*\]" \ + | head -n1 | grep -o '"[^"]*"' | sed '1d; s/^"//; s/"$//' || true +} + # runner_repo_url — the repository this box's runner is registered # to, empty when nothing is registered there. runner_repo_url() { diff --git a/test/cli.sh b/test/cli.sh index 0332275..6571de0 100644 --- a/test/cli.sh +++ b/test/cli.sh @@ -36,7 +36,17 @@ check "bootstrap: --help exits 0" 0 "usage:" "$ROOT/commands/bo check "bootstrap: unknown role exits 2" 2 "unknown role" "$ROOT/commands/bootstrap.sh" potato check "bootstrap: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/bootstrap.sh" workload --nope check "bootstrap: hostname needs value" 2 "needs a value" "$ROOT/commands/bootstrap.sh" workload --hostname -check "bootstrap: runner refuses tag:server" 2 "must not advertise tag:server" "$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server +# --ts-tag is REMOVED, not demoted: the tag now comes from the pre-auth key and +# rig verifies the GRANTED tag after join. The old runner-refuses-tag:server test +# asserted the request-time refusal THROUGH this flag; that policy now lives on +# the EFFECTIVE tag and needs a real tailnet, so it belongs to the rehearsal, not +# here. What this harness CAN prove is that the flag dies with a message pointing +# at the key (exit 2, a usage error), rather than an "unknown flag" that would +# leave an operator guessing where the tag went — value present or absent. +check "bootstrap: --ts-tag is removed (with value), exit 2" 2 "comes from the pre-auth key" \ + "$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server +check "bootstrap: --ts-tag is removed (no value), exit 2" 2 "comes from the pre-auth key" \ + "$ROOT/commands/bootstrap.sh" runner --ts-tag if [ "$(id -u)" -ne 0 ]; then check "bootstrap: refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" workload check "bootstrap: runner role parses, refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" runner @@ -115,6 +125,66 @@ check "runner install: refuses an unreadable registration" \ 1 "names no repository" guard "$REG_DIR" acme/alpha rm -rf "$REG_DIR" "$EMPTY_DIR" +# --- json_string_array: json_field's array-aware sibling --------------------- +# bootstrap reads `.Self.Tags` (a JSON array) out of `tailscale status --json` to +# assert the tag control GRANTED the node — and a rig box has no jq. Exercise the +# reader against fixture netmaps here, the same shared-lib way the guard above is: +# the bootstrap path that calls it needs a real tailnet this harness cannot fake. +tags() { # tags — prints one tag per line, exactly like the reader + bash -c 'set -euo pipefail + . "$1/commands/lib/runner-config.sh" + json_string_array "$2" Tags' _ "$ROOT" "$1" +} +tags_count() { # tags_count — prints how many tags were read (0 if none) + bash -c 'set -euo pipefail + . "$1/commands/lib/runner-config.sh" + json_string_array "$2" Tags | grep -c . || true' _ "$ROOT" "$1" +} +tags_empty() { # tags_empty — exit 0 iff the reader prints NOTHING + bash -c 'set -euo pipefail + . "$1/commands/lib/runner-config.sh" + [ -z "$(json_string_array "$2" Tags)" ]' _ "$ROOT" "$1" +} +FIX_TAGGED="$(mktemp)" # Self carries two tags; a peer carries a third +FIX_UNTAGGED="$(mktemp)" # Self has no Tags key at all — the untagged hazard +cat > "$FIX_TAGGED" <<'JSON' +{ + "BackendState": "Running", + "Self": { + "HostName": "ci-box", + "Tags": [ + "tag:ci", + "tag:build" + ] + }, + "Peer": { + "nodekey:abc": { + "HostName": "coolify-box", + "Tags": [ + "tag:server" + ] + } + } +} +JSON +cat > "$FIX_UNTAGGED" <<'JSON' +{ + "BackendState": "Running", + "Self": { + "HostName": "user-owned-box" + } +} +JSON +check "json_string_array: reads the first array element" 0 "tag:ci" tags "$FIX_TAGGED" +check "json_string_array: reads a later array element" 0 "tag:build" tags "$FIX_TAGGED" +# Self precedes Peer in the netmap, so the FIRST "Tags" is the node's own: exactly +# two elements read proves the peer's tag:server did not leak into Self's tags. +check "json_string_array: reads Self's array, not a peer's" 0 "2" tags_count "$FIX_TAGGED" +# An absent key omits itself (Go omitempty), never emits []: empty is the signal +# bootstrap turns into a hard untagged-key refusal, so it must read as empty here. +check "json_string_array: absent Tags key prints nothing" 0 "" tags_empty "$FIX_UNTAGGED" +rm -f "$FIX_TAGGED" "$FIX_UNTAGGED" + # The guard is only worth something if it runs BEFORE the box is touched: the # token prompt, the download, configure and svc.sh start all come after it. # Ordering is the whole fix, so assert it rather than trust it.