bootstrap: infer the tailnet tag from the pre-auth key — verify the tag granted, not the one requested #20
4 changed files with 227 additions and 33 deletions
54
README.md
54
README.md
|
|
@ -32,15 +32,19 @@ rig bootstrap runner --hostname my-ci-box
|
|||
```
|
||||
|
||||
- `--hostname <name>` — tailnet hostname (default: the role name)
|
||||
- `--ts-tag <tag>` — tailnet tag to advertise (default: `tag:server`;
|
||||
the `runner` role defaults to `tag:ci` instead, and **refuses**
|
||||
`tag:server` outright — see below)
|
||||
|
||||
There is **no `--ts-tag` flag**. A pre-auth key is minted *with* its tags, so
|
||||
the key is the single source of truth for the tailnet tag — rig no longer states
|
||||
a second one it might disagree with. It **verifies** the tag control actually
|
||||
granted after join instead (see below). Passing `--ts-tag` now exits 2 with a
|
||||
message pointing you at the key.
|
||||
|
||||
What it does: installs `curl ca-certificates unattended-upgrades` (and
|
||||
enables periodic unattended upgrades); writes an sshd hardening drop-in
|
||||
(`PermitRootLogin prohibit-password`, `PasswordAuthentication no`) and
|
||||
**verifies it took effect** via `sshd -T`; sets the system hostname; installs
|
||||
tailscale and joins your tailnet.
|
||||
tailscale and joins your tailnet — then **verifies the tag the key granted**
|
||||
(see *The tag comes from the key* below).
|
||||
|
||||
**`--hostname` converges both names.** On a box that has already joined,
|
||||
`bootstrap` skips `tailscale up` (so a re-run needs no pre-auth key) — but it
|
||||
|
|
@ -64,17 +68,43 @@ admin console keeps that name; rig will not fight it.
|
|||
> on re-run, and refuses to claim success unless `sshd -T` agrees.
|
||||
|
||||
**The pre-auth key:** provide it via the `TS_AUTHKEY` env var or type it at
|
||||
the interactive prompt. Use a **single-use, tagged, short-expiry** key. It
|
||||
lives in process memory only — rig never writes a credential to disk.
|
||||
the interactive prompt. Use a **single-use, tagged, short-expiry** key — the
|
||||
**tagged** part is now load-bearing, not advice (see below). It lives in process
|
||||
memory only — rig never writes a credential to disk.
|
||||
|
||||
**The tag comes from the key, and rig verifies the one control granted.** rig
|
||||
used to pass `--ts-tag` to `tailscale up --advertise-tags`, stating the tag a
|
||||
*second* time — with no way to know whether its request and the key's own tags
|
||||
agreed. It asserted the tag it **requested**, never the tag control **granted**;
|
||||
this is the same shape as the sshd first-wins bug above, and it left the same
|
||||
scar (both M900s joined carrying `tag:server` and had to be retagged by hand,
|
||||
because nothing in rig ever read the effective tag back). So rig stops
|
||||
overriding the key: `tailscale up` carries no `--advertise-tags`, the key's tags
|
||||
apply, and after join rig polls `tailscale status --json` for `.Self.Tags` — the
|
||||
netmap's ground truth, not `tailscale debug prefs`, which prints what was
|
||||
*requested* — and asserts on that, on **first join and on every re-run** (which
|
||||
catches a box bootstrapped before this change, or retagged behind rig's back).
|
||||
|
||||
> **An untagged key is a hard refusal.** Drop `--advertise-tags` and you also
|
||||
> drop the accidental net that used to tag an untagged key's node anyway. An
|
||||
> untagged node joins owned by the *key creator's user identity* — it inherits
|
||||
> that human's ACL grants, expires with the key, and vanishes if the account is
|
||||
> deleted. That is a fleet-shaped mistake, not a warning: rig runs `tailscale
|
||||
> logout` to back the half-joined node out and dies telling you to mint a tagged
|
||||
> key. A wrong tag **cannot** be fixed in place either — `tailscale set` has no
|
||||
> tag flag, re-tagging needs a fresh key via `up --force-reauth` — so rig detects
|
||||
> and refuses, and never claims a convergence it cannot perform.
|
||||
|
||||
`control-plane` and `workload` are identical today except the default
|
||||
hostname; they exist because the boxes diverge over time, and because each
|
||||
follow-up command applies to exactly one role. `runner` is the box a CI
|
||||
agent will live on, and it differs behaviorally: it defaults `--ts-tag` to
|
||||
`tag:ci` and **refuses `tag:server`** — a runner executes repo-controlled
|
||||
code, and advertising your server tag would extend every grant your servers
|
||||
hold (SSH between them, say) to that code. The refusal turns the worst
|
||||
misconfiguration from a documentation warning into a hard error.
|
||||
follow-up command applies to exactly one role. `runner` is the box a CI agent
|
||||
will live on, and it differs behaviorally: it **refuses `tag:server`**. That
|
||||
refusal moved onto the *effective* tag and is strictly stronger for it — it is
|
||||
no longer "don't advertise `tag:server`" but "the key you actually used must not
|
||||
grant `tag:server` to repo-controlled code." A runner executes that code, and
|
||||
`tag:server`'s grants (SSH between your servers, say) must never extend to it;
|
||||
the check turns the worst misconfiguration from a documentation warning into a
|
||||
hard, post-join error.
|
||||
|
||||
### `rig coolify install --version <pin>`
|
||||
|
||||
|
|
|
|||
|
|
@ -3,19 +3,25 @@
|
|||
# Convergent: safe to re-run; a second run changes nothing.
|
||||
set -euo pipefail
|
||||
|
||||
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
|
||||
# shellcheck source=SCRIPTDIR/lib/runner-config.sh
|
||||
. "$HERE/lib/runner-config.sh" # json_field / json_string_array read the netmap
|
||||
|
||||
log() { printf 'rig-bootstrap: %s\n' "$*"; }
|
||||
warn() { printf 'rig-bootstrap: WARNING: %s\n' "$*" >&2; }
|
||||
die() { printf 'rig-bootstrap: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
usage: rig bootstrap <control-plane|workload|runner> [--hostname <name>] [--ts-tag <tag>]
|
||||
usage: rig bootstrap <control-plane|workload|runner> [--hostname <name>]
|
||||
|
||||
--hostname system + tailnet hostname (default: the role name)
|
||||
--ts-tag tailnet tag to advertise (default: tag:server;
|
||||
role runner defaults to tag:ci and refuses tag:server —
|
||||
a CI box executes repo-controlled code, and your server
|
||||
tag's grants must never extend to it)
|
||||
|
||||
The tailnet tag is NOT a rig argument. A pre-auth key is minted WITH its tags,
|
||||
so the key is the single source of truth: rig no longer requests a tag it might
|
||||
disagree with. After the box joins, rig reads the tag control actually GRANTED
|
||||
(tailscale status .Self.Tags) and asserts on THAT — an untagged key is refused
|
||||
outright, and a runner may not carry tag:server. Mint a correctly-tagged key.
|
||||
|
||||
Provide the single-use tailscale pre-auth key via the TS_AUTHKEY env var, or
|
||||
enter it at the interactive prompt. It is used once and never written to disk.
|
||||
|
|
@ -32,29 +38,25 @@ case "$ROLE" in
|
|||
esac
|
||||
|
||||
TS_HOSTNAME="$ROLE"
|
||||
if [ "$ROLE" = "runner" ]; then
|
||||
TS_TAG="tag:ci"
|
||||
else
|
||||
TS_TAG="tag:server"
|
||||
fi
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--hostname)
|
||||
[ $# -ge 2 ] || die "--hostname needs a value" 2
|
||||
TS_HOSTNAME="$2"; shift 2 ;;
|
||||
--ts-tag)
|
||||
[ $# -ge 2 ] || die "--ts-tag needs a value" 2
|
||||
TS_TAG="$2"; shift 2 ;;
|
||||
# --ts-tag is GONE, but this is a deliberate death with a message, not an
|
||||
# "unknown flag": the flag shipped for a month and scripts still pass it,
|
||||
# so it must explain where the tag went rather than look like a typo. The
|
||||
# tag is now the key's to state and rig's to verify after join (issue #16 —
|
||||
# the tag was said twice and rig never checked the two agreed). Consume a
|
||||
# following value if present so `--ts-tag tag:server` dies on the flag and
|
||||
# its argument never lands in the *) arm as a mystery unknown flag.
|
||||
[ $# -ge 2 ] && shift
|
||||
die "--ts-tag is removed: the tailnet tag comes from the pre-auth key now, not rig. Mint a key with the tag you want; rig verifies the granted tag after join." 2 ;;
|
||||
*) die "unknown flag: $1" 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# A runner executes repo-controlled code; advertising the server tag would
|
||||
# extend every grant your servers hold to that code. Refused, not warned.
|
||||
if [ "$ROLE" = "runner" ] && [ "$TS_TAG" = "tag:server" ]; then
|
||||
die "role runner must not advertise tag:server" 2
|
||||
fi
|
||||
|
||||
# --- guards ------------------------------------------------------------------
|
||||
[ "$(id -u)" -eq 0 ] || die "must run as root"
|
||||
if [ -r /etc/os-release ]; then
|
||||
|
|
@ -164,6 +166,59 @@ else
|
|||
fi
|
||||
|
||||
# --- tailscale ----------------------------------------------------------------
|
||||
# verify_effective_tag — assert the tag control actually GRANTED this node, never
|
||||
# the one rig requested. This is the sshd `sshd -T` lesson wearing a tailnet hat:
|
||||
# rig used to advertise a tag and trust it took, exactly as it once trusted that
|
||||
# a drop-in FILE existing meant sshd had read it. Both M900s joined carrying
|
||||
# tag:server and had to be retagged by hand; nothing in rig noticed because
|
||||
# nothing ever read the effective tag back.
|
||||
#
|
||||
# `.Self.Tags` from `tailscale status --json` is the netmap's ground truth.
|
||||
# `tailscale debug prefs` would LIE here — it prints AdvertiseTags, i.e. what was
|
||||
# REQUESTED — which is precisely the second source of truth issue #16 deletes.
|
||||
# Tags ride in with the netmap, not synchronously out of `up`, so a single read
|
||||
# right after join can legitimately come back empty; poll until tags appear OR
|
||||
# the backend reaches Running (past which an empty Tags is real, not just early).
|
||||
verify_effective_tag() {
|
||||
local deadline=$((SECONDS + 30)) tags="" state="" json
|
||||
json="$(mktemp)"
|
||||
while :; do
|
||||
if tailscale status --json > "$json" 2>/dev/null; then
|
||||
tags="$(json_string_array "$json" Tags)"
|
||||
state="$(json_field "$json" BackendState)"
|
||||
if [ -n "$tags" ] || [ "$state" = "Running" ]; then break; fi
|
||||
fi
|
||||
if [ "$SECONDS" -ge "$deadline" ]; then break; fi
|
||||
sleep 2
|
||||
done
|
||||
rm -f "$json"
|
||||
|
||||
# UNTAGGED is the real hazard and it is silent: with no tags the node joined
|
||||
# owned by the KEY CREATOR's user identity — it inherits that human's ACL
|
||||
# grants, expires with the key, and vanishes if the account is deleted.
|
||||
# Dropping --advertise-tags removed the accidental net that used to tag such a
|
||||
# node anyway, so rig must now catch this out loud. A wrong tag cannot be fixed
|
||||
# in place (`tailscale set` has no tag flag; re-tagging needs a fresh key via
|
||||
# `up --force-reauth`), so back the node out rather than leave a half-joined,
|
||||
# user-owned device squatting a hostname.
|
||||
if [ -z "$tags" ]; then
|
||||
tailscale logout >/dev/null 2>&1 \
|
||||
|| warn "tailscale logout failed — this node is joined UNTAGGED and user-owned; remove it from the tailnet by hand"
|
||||
die "joined with NO tag: the pre-auth key was untagged, so this node is owned by the key creator's user identity, not a tag. Backed it out. Fix: mint a TAGGED pre-auth key and re-run."
|
||||
fi
|
||||
|
||||
# Role policy now rides the EFFECTIVE tag — strictly stronger than the old
|
||||
# request-time check, which only guarded the tag rig HOPED for. This guards the
|
||||
# tag the key ACTUALLY granted to repo-controlled code: a runner carrying
|
||||
# tag:server would extend every grant your servers hold to CI code. Refused,
|
||||
# never warned. rig can DETECT this but cannot FIX it, so name the repair.
|
||||
if [ "$ROLE" = "runner" ] && printf '%s\n' "$tags" | grep -qx 'tag:server'; then
|
||||
die "role runner joined with tag:server (effective tags: $(printf '%s' "$tags" | tr '\n' ' ')). The key you used grants tag:server to repo-controlled code; that must never happen. Re-run bootstrap with a key minted for a CI tag (e.g. tag:ci)."
|
||||
fi
|
||||
|
||||
log "verified effective tailnet tag(s): $(printf '%s' "$tags" | tr '\n' ' ')"
|
||||
}
|
||||
|
||||
if ! command -v tailscale >/dev/null 2>&1; then
|
||||
log "installing tailscale"
|
||||
curl -fsSL https://tailscale.com/install.sh | sh
|
||||
|
|
@ -190,6 +245,12 @@ if tailscale status >/dev/null 2>&1; then
|
|||
else
|
||||
log "tailnet hostname already ${TS_HOSTNAME}"
|
||||
fi
|
||||
# Verify the tag on the already-joined path too, not only on first join: this
|
||||
# catches a box bootstrapped BEFORE rig looked at tags, or one retagged behind
|
||||
# rig's back, on the very next ordinary re-run. Skipping `tailscale up` here is
|
||||
# deliberate and stays — re-running an identical tagged-authkey `up` errors —
|
||||
# but skipping the CHECK was how the M900s stayed mis-tagged unnoticed.
|
||||
verify_effective_tag
|
||||
else
|
||||
# env override, else prompt; never touches disk
|
||||
if [ -z "${TS_AUTHKEY:-}" ]; then
|
||||
|
|
@ -197,8 +258,13 @@ else
|
|||
echo
|
||||
fi
|
||||
[ -n "${TS_AUTHKEY:-}" ] || die "empty pre-auth key"
|
||||
log "joining tailnet as ${TS_HOSTNAME} (${TS_TAG})"
|
||||
tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME" --advertise-tags="$TS_TAG"
|
||||
# No --advertise-tags: the key's own tags apply (documented default for a
|
||||
# tagged key), and rig verifies them below instead of stating a second tag it
|
||||
# cannot reconcile with the key's. A tagged key needs no flag; an untagged one
|
||||
# cannot be rescued by one (verify_effective_tag refuses it and logs out).
|
||||
log "joining tailnet as ${TS_HOSTNAME} (tag comes from the pre-auth key)"
|
||||
tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME"
|
||||
verify_effective_tag
|
||||
fi
|
||||
|
||||
log "done — role ${ROLE}, hostname ${TS_HOSTNAME}"
|
||||
|
|
|
|||
|
|
@ -14,6 +14,34 @@ json_field() {
|
|||
| head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//' || true
|
||||
}
|
||||
|
||||
# json_string_array <file> <key> — the elements of the FIRST array named <key>,
|
||||
# one per line, empty when the key is absent or the array is empty.
|
||||
#
|
||||
# json_field's sibling for the one shape it cannot read: `.Self.Tags` from
|
||||
# `tailscale status --json` is a JSON array, and bootstrap must assert on it to
|
||||
# learn the tag control actually GRANTED the node (the netmap's ground truth),
|
||||
# not the tag rig requested. Same grep/sed spirit, same jq-free reason: a
|
||||
# rig-bootstrapped box has no jq and we will not install one to read one field.
|
||||
#
|
||||
# `tr -d '\n'` first, because tailscale pretty-prints its JSON and an array
|
||||
# spans lines — grep is line-oriented and would never see `[ ... ]` whole
|
||||
# otherwise. `\[[^]]*\]` then captures the first flat array body for <key>
|
||||
# (tag strings never contain `]`, so this is safe); the inner `grep -o` pulls
|
||||
# every quoted token out of it, and `sed 1d` drops the key's own name — which
|
||||
# `"key":[...]` leads with — leaving just the elements.
|
||||
#
|
||||
# FIRST array wins by design, and the caller leans on it: `tailscale status
|
||||
# --json` emits Self before Peer (Go struct field order, stable), so the first
|
||||
# "Tags" is the node's OWN, never a peer's. An absent key omits itself entirely
|
||||
# (Go's omitempty) rather than emitting `[]` — which is exactly the untagged,
|
||||
# user-owned node bootstrap must catch. Never fails under `set -e`+pipefail: a
|
||||
# non-match is a fact to test for, like json_field, not a reason to die.
|
||||
json_string_array() {
|
||||
tr -d '\n' < "$1" 2>/dev/null \
|
||||
| grep -o "\"$2\"[[:space:]]*:[[:space:]]*\[[^]]*\]" \
|
||||
| head -n1 | grep -o '"[^"]*"' | sed '1d; s/^"//; s/"$//' || true
|
||||
}
|
||||
|
||||
# runner_repo_url <runner_dir> — the repository this box's runner is registered
|
||||
# to, empty when nothing is registered there.
|
||||
runner_repo_url() {
|
||||
|
|
|
|||
72
test/cli.sh
72
test/cli.sh
|
|
@ -36,7 +36,17 @@ check "bootstrap: --help exits 0" 0 "usage:" "$ROOT/commands/bo
|
|||
check "bootstrap: unknown role exits 2" 2 "unknown role" "$ROOT/commands/bootstrap.sh" potato
|
||||
check "bootstrap: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/bootstrap.sh" workload --nope
|
||||
check "bootstrap: hostname needs value" 2 "needs a value" "$ROOT/commands/bootstrap.sh" workload --hostname
|
||||
check "bootstrap: runner refuses tag:server" 2 "must not advertise tag:server" "$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server
|
||||
# --ts-tag is REMOVED, not demoted: the tag now comes from the pre-auth key and
|
||||
# rig verifies the GRANTED tag after join. The old runner-refuses-tag:server test
|
||||
# asserted the request-time refusal THROUGH this flag; that policy now lives on
|
||||
# the EFFECTIVE tag and needs a real tailnet, so it belongs to the rehearsal, not
|
||||
# here. What this harness CAN prove is that the flag dies with a message pointing
|
||||
# at the key (exit 2, a usage error), rather than an "unknown flag" that would
|
||||
# leave an operator guessing where the tag went — value present or absent.
|
||||
check "bootstrap: --ts-tag is removed (with value), exit 2" 2 "comes from the pre-auth key" \
|
||||
"$ROOT/commands/bootstrap.sh" runner --ts-tag tag:server
|
||||
check "bootstrap: --ts-tag is removed (no value), exit 2" 2 "comes from the pre-auth key" \
|
||||
"$ROOT/commands/bootstrap.sh" runner --ts-tag
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
check "bootstrap: refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" workload
|
||||
check "bootstrap: runner role parses, refuses non-root" 1 "must run as root" env TS_AUTHKEY=x "$ROOT/commands/bootstrap.sh" runner
|
||||
|
|
@ -115,6 +125,66 @@ check "runner install: refuses an unreadable registration" \
|
|||
1 "names no repository" guard "$REG_DIR" acme/alpha
|
||||
rm -rf "$REG_DIR" "$EMPTY_DIR"
|
||||
|
||||
# --- json_string_array: json_field's array-aware sibling ---------------------
|
||||
# bootstrap reads `.Self.Tags` (a JSON array) out of `tailscale status --json` to
|
||||
# assert the tag control GRANTED the node — and a rig box has no jq. Exercise the
|
||||
# reader against fixture netmaps here, the same shared-lib way the guard above is:
|
||||
# the bootstrap path that calls it needs a real tailnet this harness cannot fake.
|
||||
tags() { # tags <file> — prints one tag per line, exactly like the reader
|
||||
bash -c 'set -euo pipefail
|
||||
. "$1/commands/lib/runner-config.sh"
|
||||
json_string_array "$2" Tags' _ "$ROOT" "$1"
|
||||
}
|
||||
tags_count() { # tags_count <file> — prints how many tags were read (0 if none)
|
||||
bash -c 'set -euo pipefail
|
||||
. "$1/commands/lib/runner-config.sh"
|
||||
json_string_array "$2" Tags | grep -c . || true' _ "$ROOT" "$1"
|
||||
}
|
||||
tags_empty() { # tags_empty <file> — exit 0 iff the reader prints NOTHING
|
||||
bash -c 'set -euo pipefail
|
||||
. "$1/commands/lib/runner-config.sh"
|
||||
[ -z "$(json_string_array "$2" Tags)" ]' _ "$ROOT" "$1"
|
||||
}
|
||||
FIX_TAGGED="$(mktemp)" # Self carries two tags; a peer carries a third
|
||||
FIX_UNTAGGED="$(mktemp)" # Self has no Tags key at all — the untagged hazard
|
||||
cat > "$FIX_TAGGED" <<'JSON'
|
||||
{
|
||||
"BackendState": "Running",
|
||||
"Self": {
|
||||
"HostName": "ci-box",
|
||||
"Tags": [
|
||||
"tag:ci",
|
||||
"tag:build"
|
||||
]
|
||||
},
|
||||
"Peer": {
|
||||
"nodekey:abc": {
|
||||
"HostName": "coolify-box",
|
||||
"Tags": [
|
||||
"tag:server"
|
||||
]
|
||||
}
|
||||
}
|
||||
}
|
||||
JSON
|
||||
cat > "$FIX_UNTAGGED" <<'JSON'
|
||||
{
|
||||
"BackendState": "Running",
|
||||
"Self": {
|
||||
"HostName": "user-owned-box"
|
||||
}
|
||||
}
|
||||
JSON
|
||||
check "json_string_array: reads the first array element" 0 "tag:ci" tags "$FIX_TAGGED"
|
||||
check "json_string_array: reads a later array element" 0 "tag:build" tags "$FIX_TAGGED"
|
||||
# Self precedes Peer in the netmap, so the FIRST "Tags" is the node's own: exactly
|
||||
# two elements read proves the peer's tag:server did not leak into Self's tags.
|
||||
check "json_string_array: reads Self's array, not a peer's" 0 "2" tags_count "$FIX_TAGGED"
|
||||
# An absent key omits itself (Go omitempty), never emits []: empty is the signal
|
||||
# bootstrap turns into a hard untagged-key refusal, so it must read as empty here.
|
||||
check "json_string_array: absent Tags key prints nothing" 0 "" tags_empty "$FIX_UNTAGGED"
|
||||
rm -f "$FIX_TAGGED" "$FIX_UNTAGGED"
|
||||
|
||||
# The guard is only worth something if it runs BEFORE the box is touched: the
|
||||
# token prompt, the download, configure and svc.sh start all come after it.
|
||||
# Ordering is the whole fix, so assert it rather than trust it.
|
||||
|
|
|
|||
Loading…
Reference in a new issue