bootstrap: infer the tailnet tag from the pre-auth key — verify the tag granted, not the one requested #20

Merged
dan-claude-bot merged 1 commit from feat/bootstrap-infer-tailnet-tag into main 2026-07-17 15:47:12 +00:00
4 changed files with 227 additions and 33 deletions
Showing only changes of commit 92fa2a9860 - Show all commits

View file

@ -32,15 +32,19 @@ rig bootstrap runner --hostname my-ci-box
```
- `--hostname <name>` — tailnet hostname (default: the role name)
- `--ts-tag <tag>` — tailnet tag to advertise (default: `tag:server`;
the `runner` role defaults to `tag:ci` instead, and **refuses**
`tag:server` outright — see below)
There is **no `--ts-tag` flag**. A pre-auth key is minted *with* its tags, so
the key is the single source of truth for the tailnet tag — rig no longer states
a second one it might disagree with. It **verifies** the tag control actually
granted after join instead (see below). Passing `--ts-tag` now exits 2 with a
message pointing you at the key.
What it does: installs `curl ca-certificates unattended-upgrades` (and
enables periodic unattended upgrades); writes an sshd hardening drop-in
(`PermitRootLogin prohibit-password`, `PasswordAuthentication no`) and
**verifies it took effect** via `sshd -T`; sets the system hostname; installs
tailscale and joins your tailnet.
tailscale and joins your tailnet — then **verifies the tag the key granted**
(see *The tag comes from the key* below).
**`--hostname` converges both names.** On a box that has already joined,
`bootstrap` skips `tailscale up` (so a re-run needs no pre-auth key) — but it
@ -64,17 +68,43 @@ admin console keeps that name; rig will not fight it.
> on re-run, and refuses to claim success unless `sshd -T` agrees.
**The pre-auth key:** provide it via the `TS_AUTHKEY` env var or type it at
the interactive prompt. Use a **single-use, tagged, short-expiry** key. It
lives in process memory only — rig never writes a credential to disk.
the interactive prompt. Use a **single-use, tagged, short-expiry** key — the
**tagged** part is now load-bearing, not advice (see below). It lives in process
memory only — rig never writes a credential to disk.
**The tag comes from the key, and rig verifies the one control granted.** rig
used to pass `--ts-tag` to `tailscale up --advertise-tags`, stating the tag a
*second* time — with no way to know whether its request and the key's own tags
agreed. It asserted the tag it **requested**, never the tag control **granted**;
this is the same shape as the sshd first-wins bug above, and it left the same
scar (both M900s joined carrying `tag:server` and had to be retagged by hand,
because nothing in rig ever read the effective tag back). So rig stops
overriding the key: `tailscale up` carries no `--advertise-tags`, the key's tags
apply, and after join rig polls `tailscale status --json` for `.Self.Tags` — the
netmap's ground truth, not `tailscale debug prefs`, which prints what was
*requested* — and asserts on that, on **first join and on every re-run** (which
catches a box bootstrapped before this change, or retagged behind rig's back).
> **An untagged key is a hard refusal.** Drop `--advertise-tags` and you also
> drop the accidental net that used to tag an untagged key's node anyway. An
> untagged node joins owned by the *key creator's user identity* — it inherits
> that human's ACL grants, expires with the key, and vanishes if the account is
> deleted. That is a fleet-shaped mistake, not a warning: rig runs `tailscale
> logout` to back the half-joined node out and dies telling you to mint a tagged
> key. A wrong tag **cannot** be fixed in place either — `tailscale set` has no
> tag flag, re-tagging needs a fresh key via `up --force-reauth` — so rig detects
> and refuses, and never claims a convergence it cannot perform.
`control-plane` and `workload` are identical today except the default
hostname; they exist because the boxes diverge over time, and because each
follow-up command applies to exactly one role. `runner` is the box a CI
agent will live on, and it differs behaviorally: it defaults `--ts-tag` to
`tag:ci` and **refuses `tag:server`** — a runner executes repo-controlled
code, and advertising your server tag would extend every grant your servers
hold (SSH between them, say) to that code. The refusal turns the worst
misconfiguration from a documentation warning into a hard error.
follow-up command applies to exactly one role. `runner` is the box a CI agent
will live on, and it differs behaviorally: it **refuses `tag:server`**. That
refusal moved onto the *effective* tag and is strictly stronger for it — it is
no longer "don't advertise `tag:server`" but "the key you actually used must not
grant `tag:server` to repo-controlled code." A runner executes that code, and
`tag:server`'s grants (SSH between your servers, say) must never extend to it;
the check turns the worst misconfiguration from a documentation warning into a
hard, post-join error.
### `rig coolify install --version <pin>`

View file

@ -3,19 +3,25 @@
# Convergent: safe to re-run; a second run changes nothing.
set -euo pipefail
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
# shellcheck source=SCRIPTDIR/lib/runner-config.sh
. "$HERE/lib/runner-config.sh" # json_field / json_string_array read the netmap
log() { printf 'rig-bootstrap: %s\n' "$*"; }
warn() { printf 'rig-bootstrap: WARNING: %s\n' "$*" >&2; }
die() { printf 'rig-bootstrap: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
usage() {
cat <<'EOF'
usage: rig bootstrap <control-plane|workload|runner> [--hostname <name>] [--ts-tag <tag>]
usage: rig bootstrap <control-plane|workload|runner> [--hostname <name>]
--hostname system + tailnet hostname (default: the role name)
--ts-tag tailnet tag to advertise (default: tag:server;
role runner defaults to tag:ci and refuses tag:server —
a CI box executes repo-controlled code, and your server
tag's grants must never extend to it)
The tailnet tag is NOT a rig argument. A pre-auth key is minted WITH its tags,
so the key is the single source of truth: rig no longer requests a tag it might
disagree with. After the box joins, rig reads the tag control actually GRANTED
(tailscale status .Self.Tags) and asserts on THAT — an untagged key is refused
outright, and a runner may not carry tag:server. Mint a correctly-tagged key.
Provide the single-use tailscale pre-auth key via the TS_AUTHKEY env var, or
enter it at the interactive prompt. It is used once and never written to disk.
@ -32,29 +38,25 @@ case "$ROLE" in
esac
TS_HOSTNAME="$ROLE"
if [ "$ROLE" = "runner" ]; then
TS_TAG="tag:ci"
else
TS_TAG="tag:server"
fi
while [ $# -gt 0 ]; do
case "$1" in
--hostname)
[ $# -ge 2 ] || die "--hostname needs a value" 2
TS_HOSTNAME="$2"; shift 2 ;;
--ts-tag)
[ $# -ge 2 ] || die "--ts-tag needs a value" 2
TS_TAG="$2"; shift 2 ;;
# --ts-tag is GONE, but this is a deliberate death with a message, not an
# "unknown flag": the flag shipped for a month and scripts still pass it,
# so it must explain where the tag went rather than look like a typo. The
# tag is now the key's to state and rig's to verify after join (issue #16 —
# the tag was said twice and rig never checked the two agreed). Consume a
# following value if present so `--ts-tag tag:server` dies on the flag and
# its argument never lands in the *) arm as a mystery unknown flag.
[ $# -ge 2 ] && shift
die "--ts-tag is removed: the tailnet tag comes from the pre-auth key now, not rig. Mint a key with the tag you want; rig verifies the granted tag after join." 2 ;;
*) die "unknown flag: $1" 2 ;;
esac
done
# A runner executes repo-controlled code; advertising the server tag would
# extend every grant your servers hold to that code. Refused, not warned.
if [ "$ROLE" = "runner" ] && [ "$TS_TAG" = "tag:server" ]; then
die "role runner must not advertise tag:server" 2
fi
# --- guards ------------------------------------------------------------------
[ "$(id -u)" -eq 0 ] || die "must run as root"
if [ -r /etc/os-release ]; then
@ -164,6 +166,59 @@ else
fi
# --- tailscale ----------------------------------------------------------------
# verify_effective_tag — assert the tag control actually GRANTED this node, never
# the one rig requested. This is the sshd `sshd -T` lesson wearing a tailnet hat:
# rig used to advertise a tag and trust it took, exactly as it once trusted that
# a drop-in FILE existing meant sshd had read it. Both M900s joined carrying
# tag:server and had to be retagged by hand; nothing in rig noticed because
# nothing ever read the effective tag back.
#
# `.Self.Tags` from `tailscale status --json` is the netmap's ground truth.
# `tailscale debug prefs` would LIE here — it prints AdvertiseTags, i.e. what was
# REQUESTED — which is precisely the second source of truth issue #16 deletes.
# Tags ride in with the netmap, not synchronously out of `up`, so a single read
# right after join can legitimately come back empty; poll until tags appear OR
# the backend reaches Running (past which an empty Tags is real, not just early).
verify_effective_tag() {
local deadline=$((SECONDS + 30)) tags="" state="" json
json="$(mktemp)"
while :; do
if tailscale status --json > "$json" 2>/dev/null; then
tags="$(json_string_array "$json" Tags)"
state="$(json_field "$json" BackendState)"
if [ -n "$tags" ] || [ "$state" = "Running" ]; then break; fi
fi
if [ "$SECONDS" -ge "$deadline" ]; then break; fi
sleep 2
done
rm -f "$json"
# UNTAGGED is the real hazard and it is silent: with no tags the node joined
# owned by the KEY CREATOR's user identity — it inherits that human's ACL
# grants, expires with the key, and vanishes if the account is deleted.
# Dropping --advertise-tags removed the accidental net that used to tag such a
# node anyway, so rig must now catch this out loud. A wrong tag cannot be fixed
# in place (`tailscale set` has no tag flag; re-tagging needs a fresh key via
# `up --force-reauth`), so back the node out rather than leave a half-joined,
# user-owned device squatting a hostname.
if [ -z "$tags" ]; then
tailscale logout >/dev/null 2>&1 \
|| warn "tailscale logout failed — this node is joined UNTAGGED and user-owned; remove it from the tailnet by hand"
die "joined with NO tag: the pre-auth key was untagged, so this node is owned by the key creator's user identity, not a tag. Backed it out. Fix: mint a TAGGED pre-auth key and re-run."
fi
# Role policy now rides the EFFECTIVE tag — strictly stronger than the old
# request-time check, which only guarded the tag rig HOPED for. This guards the
# tag the key ACTUALLY granted to repo-controlled code: a runner carrying
# tag:server would extend every grant your servers hold to CI code. Refused,
# never warned. rig can DETECT this but cannot FIX it, so name the repair.
if [ "$ROLE" = "runner" ] && printf '%s\n' "$tags" | grep -qx 'tag:server'; then
die "role runner joined with tag:server (effective tags: $(printf '%s' "$tags" | tr '\n' ' ')). The key you used grants tag:server to repo-controlled code; that must never happen. Re-run bootstrap with a key minted for a CI tag (e.g. tag:ci)."
fi
log "verified effective tailnet tag(s): $(printf '%s' "$tags" | tr '\n' ' ')"
}
if ! command -v tailscale >/dev/null 2>&1; then
log "installing tailscale"
curl -fsSL https://tailscale.com/install.sh | sh
@ -190,6 +245,12 @@ if tailscale status >/dev/null 2>&1; then
else
log "tailnet hostname already ${TS_HOSTNAME}"
fi
# Verify the tag on the already-joined path too, not only on first join: this
# catches a box bootstrapped BEFORE rig looked at tags, or one retagged behind
# rig's back, on the very next ordinary re-run. Skipping `tailscale up` here is
# deliberate and stays — re-running an identical tagged-authkey `up` errors —
# but skipping the CHECK was how the M900s stayed mis-tagged unnoticed.
verify_effective_tag
else
# env override, else prompt; never touches disk
if [ -z "${TS_AUTHKEY:-}" ]; then
@ -197,8 +258,13 @@ else
echo
fi
[ -n "${TS_AUTHKEY:-}" ] || die "empty pre-auth key"
log "joining tailnet as ${TS_HOSTNAME} (${TS_TAG})"
tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME" --advertise-tags="$TS_TAG"
# No --advertise-tags: the key's own tags apply (documented default for a
# tagged key), and rig verifies them below instead of stating a second tag it
# cannot reconcile with the key's. A tagged key needs no flag; an untagged one
# cannot be rescued by one (verify_effective_tag refuses it and logs out).
log "joining tailnet as ${TS_HOSTNAME} (tag comes from the pre-auth key)"
tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME"
verify_effective_tag
fi
log "done — role ${ROLE}, hostname ${TS_HOSTNAME}"

View file

@ -14,6 +14,34 @@ json_field() {
| head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//' || true
}
# json_string_array <file> <key> — the elements of the FIRST array named <key>,
# one per line, empty when the key is absent or the array is empty.
#
# json_field's sibling for the one shape it cannot read: `.Self.Tags` from
# `tailscale status --json` is a JSON array, and bootstrap must assert on it to
# learn the tag control actually GRANTED the node (the netmap's ground truth),
# not the tag rig requested. Same grep/sed spirit, same jq-free reason: a
# rig-bootstrapped box has no jq and we will not install one to read one field.
#
# `tr -d '\n'` first, because tailscale pretty-prints its JSON and an array
# spans lines — grep is line-oriented and would never see `[ ... ]` whole
# otherwise. `\[[^]]*\]` then captures the first flat array body for <key>
# (tag strings never contain `]`, so this is safe); the inner `grep -o` pulls
# every quoted token out of it, and `sed 1d` drops the key's own name — which
# `"key":[...]` leads with — leaving just the elements.
#
# FIRST array wins by design, and the caller leans on it: `tailscale status
# --json` emits Self before Peer (Go struct field order, stable), so the first
# "Tags" is the node's OWN, never a peer's. An absent key omits itself entirely
# (Go's omitempty) rather than emitting `[]` — which is exactly the untagged,
# user-owned node bootstrap must catch. Never fails under `set -e`+pipefail: a
# non-match is a fact to test for, like json_field, not a reason to die.
json_string_array() {
tr -d '\n' < "$1" 2>/dev/null \
| grep -o "\"$2\"[[:space:]]*:[[:space:]]*\[[^]]*\]" \
| head -n1 | grep -o '"[^"]*"' | sed '1d; s/^"//; s/"$//' || true
}
# runner_repo_url <runner_dir> — the repository this box's runner is registered
# to, empty when nothing is registered there.
runner_repo_url() {

View file

@ -36,7 +36,17 @@ check "bootstrap: --help exits 0" 0 "usage:" "$ROOT/commands/bo
check "bootstrap: unknown role exits 2" 2 "unknown role" "$ROOT/commands/bootstrap.sh" potato
check "bootstrap: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/bootstrap.sh" workload --nope
check "bootstrap: hostname needs value" 2 "needs a value" "$ROOT/commands/bootstrap.sh" workload --hostname
check "bootstrap: runner refuses tag:server" 2 "must not advertise tag:server" "$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server
# --ts-tag is REMOVED, not demoted: the tag now comes from the pre-auth key and
# rig verifies the GRANTED tag after join. The old runner-refuses-tag:server test
# asserted the request-time refusal THROUGH this flag; that policy now lives on
# the EFFECTIVE tag and needs a real tailnet, so it belongs to the rehearsal, not
# here. What this harness CAN prove is that the flag dies with a message pointing
# at the key (exit 2, a usage error), rather than an "unknown flag" that would
# leave an operator guessing where the tag went — value present or absent.
check "bootstrap: --ts-tag is removed (with value), exit 2" 2 "comes from the pre-auth key" \
"$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server
check "bootstrap: --ts-tag is removed (no value), exit 2" 2 "comes from the pre-auth key" \
"$ROOT/commands/bootstrap.sh" runner --ts-tag
if [ "$(id -u)" -ne 0 ]; then
check "bootstrap: refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" workload
check "bootstrap: runner role parses, refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" runner
@ -115,6 +125,66 @@ check "runner install: refuses an unreadable registration" \
1 "names no repository" guard "$REG_DIR" acme/alpha
rm -rf "$REG_DIR" "$EMPTY_DIR"
# --- json_string_array: json_field's array-aware sibling ---------------------
# bootstrap reads `.Self.Tags` (a JSON array) out of `tailscale status --json` to
# assert the tag control GRANTED the node — and a rig box has no jq. Exercise the
# reader against fixture netmaps here, the same shared-lib way the guard above is:
# the bootstrap path that calls it needs a real tailnet this harness cannot fake.
tags() { # tags <file> — prints one tag per line, exactly like the reader
bash -c 'set -euo pipefail
. "$1/commands/lib/runner-config.sh"
json_string_array "$2" Tags' _ "$ROOT" "$1"
}
tags_count() { # tags_count <file> — prints how many tags were read (0 if none)
bash -c 'set -euo pipefail
. "$1/commands/lib/runner-config.sh"
json_string_array "$2" Tags | grep -c . || true' _ "$ROOT" "$1"
}
tags_empty() { # tags_empty <file> — exit 0 iff the reader prints NOTHING
bash -c 'set -euo pipefail
. "$1/commands/lib/runner-config.sh"
[ -z "$(json_string_array "$2" Tags)" ]' _ "$ROOT" "$1"
}
FIX_TAGGED="$(mktemp)" # Self carries two tags; a peer carries a third
FIX_UNTAGGED="$(mktemp)" # Self has no Tags key at all — the untagged hazard
cat > "$FIX_TAGGED" <<'JSON'
{
"BackendState": "Running",
"Self": {
"HostName": "ci-box",
"Tags": [
"tag:ci",
"tag:build"
]
},
"Peer": {
"nodekey:abc": {
"HostName": "coolify-box",
"Tags": [
"tag:server"
]
}
}
}
JSON
cat > "$FIX_UNTAGGED" <<'JSON'
{
"BackendState": "Running",
"Self": {
"HostName": "user-owned-box"
}
}
JSON
check "json_string_array: reads the first array element" 0 "tag:ci" tags "$FIX_TAGGED"
check "json_string_array: reads a later array element" 0 "tag:build" tags "$FIX_TAGGED"
# Self precedes Peer in the netmap, so the FIRST "Tags" is the node's own: exactly
# two elements read proves the peer's tag:server did not leak into Self's tags.
check "json_string_array: reads Self's array, not a peer's" 0 "2" tags_count "$FIX_TAGGED"
# An absent key omits itself (Go omitempty), never emits []: empty is the signal
# bootstrap turns into a hard untagged-key refusal, so it must read as empty here.
check "json_string_array: absent Tags key prints nothing" 0 "" tags_empty "$FIX_UNTAGGED"
rm -f "$FIX_TAGGED" "$FIX_UNTAGGED"
# The guard is only worth something if it runs BEFORE the box is touched: the
# token prompt, the download, configure and svc.sh start all come after it.
# Ordering is the whole fix, so assert it rather than trust it.