heavy-duty/rig
7
0
Fork 1
Code Issues 10 Pull requests Projects Releases 3 Packages Wiki Activity Actions

Compare commits

base: heavy-duty:main
Branches Tags
heavy-duty:main
heavy-duty:build/139-sbin-path-preflight
heavy-duty:build/144-default-labels-option-b
heavy-duty:build/144-ubuntu-latest-tooling
heavy-duty:build/136-cli-hermetic
heavy-duty:build/133-status-active-is-not-health
heavy-duty:build/129-forgejo-runner-drill-leg
heavy-duty:build/135-runner-cache-dir
heavy-duty:build/131-readme-forgejo-install
heavy-duty:build/111-install-channel-forge-agnostic
heavy-duty:build/112-ceremony-uses-absolute
heavy-duty:build/109-forgejo-ci-box
heavy-duty:0.3.1
heavy-duty:0.3.0
heavy-duty:0.2.0
heavy-duty:0.1.0
..
compare: heavy-duty:0.3.0
Branches Tags
heavy-duty:main
heavy-duty:build/139-sbin-path-preflight
heavy-duty:build/144-default-labels-option-b
heavy-duty:build/144-ubuntu-latest-tooling
heavy-duty:build/136-cli-hermetic
heavy-duty:build/133-status-active-is-not-health
heavy-duty:build/129-forgejo-runner-drill-leg
heavy-duty:build/135-runner-cache-dir
heavy-duty:build/131-readme-forgejo-install
heavy-duty:build/111-install-channel-forge-agnostic
heavy-duty:build/112-ceremony-uses-absolute
heavy-duty:build/109-forgejo-ci-box
heavy-duty:0.3.1
heavy-duty:0.3.0
heavy-duty:0.2.0
heavy-duty:0.1.0

No commits in common. "main" and "0.3.0" have entirely different histories.
main ... 0.3.0

72 changed files with 3252 additions and 7806 deletions
Show stats Expand all files Collapse all files

49
.ceremony/AGENTS.md
View file

@ -1,49 +0,0 @@
# AGENTS.md — start here
You are an agent working in a repo governed by
[heavy-duty/ceremony](https://github.com/heavy-duty/ceremony). This file is
the router: find your role below, read its file, then act. The role files
sit beside this one — in ceremony itself at the repo root, in a governed
repo under `.ceremony/` (a machine-managed mirror; never edit those files
in place — they are changed in heavy-duty/ceremony, through its own flow).
## Your role
You were told your role when you were pointed at this repo ("you are a
reviewer here"). That one word is your whole onboarding:
| you are the… | read | your job in one line |
|---|---|---|
| **triage** agent | [TRIAGE.md](TRIAGE.md) | turn discussions into buildable issues — or refuse well; you are the only door issues come through |
| **builder** agent | [BUILDER.md](BUILDER.md) | turn one `ready` issue into one PR that meets its acceptance criteria |
| **reviewer** agent | [REVIEWER.md](REVIEWER.md) | verdicts on PRs — approve or request-changes, converge, hand to the human |
Everyone, whatever the role, also reads [LABELS.md](LABELS.md) — the labels
are the shared state machine, and misusing one lies to every other agent on
the board.
**Not told a role?** Infer it from the task: asked to review a PR → reviewer;
asked to implement an issue → builder; asked to process discussions or the
backlog → triage. Still ambiguous → ask before acting. Do not free-lance
across roles in one session: a builder reviewing its own PR, or a reviewer
pushing fixes, breaks the separation the pipeline depends on.
## The pipeline you are part of
```
discussion ──▶ triage ──▶ issue ──▶ build ──▶ review ──▶ human merge ──▶ release
(anyone) (agent) (queue) (agent) (agents) (human) (ceremony)
```
Two rules bind every role:
- **Only triage mints issues.** Found work? Open or extend a discussion.
- **Only humans merge.** Convergence ends at `state:needs-human`, never at
a merge button.
## Repo specifics
What is true only of *this* repo — the review panel roster, the `scope:*`
label set, what a drill means, code conventions — lives in the repo's own
`CONTRIBUTING.md`. Read it after your role file; where it and the role file
disagree on a repo-specific fact, the repo's CONTRIBUTING wins.

251
.ceremony/BUILDER.md
View file

@ -1,251 +0,0 @@
# BUILDER.md — the builder role
You turn one issue into one PR. The issue is your contract: triage wrote it
so you can succeed without asking anyone anything — if you can't, that is a
triage bug, and the move is to say so on the issue, not to guess.
## Picking
- Pick from issues labeled **`ready`** — never `blocked`, never `claimed`,
never an `epic` (epics organize; their children are the work).
- Respect dependency order: inside an epic, take the earliest unblocked
unclaimed child. Between epics and strays, prefer the issue that unblocks
the most other work.
- **One build at a time.** You hold at most one issue on which you are
writing or revising a deliverable — finish or release that work before
starting new work. The rule counts build work in flight, not claims: a
claim does not consume the slot while it is **parked**, meaning the next
move belongs to someone else. Exactly five shapes qualify:
1. the issue carries `needs-ruling`, its escalation names a decider, and
its `Blocked:` line stops the remaining work;
2. the deliverable is in a review round where every outstanding verdict
belongs to someone else — either the round is awaiting its first
verdicts, or it was answered whole and the non-approvers re-requested
(the review round, steps 1–2). This is the *live* round; shape 4 is
the *passed* one — they are sequential and do not overlap;
3. every remaining acceptance criterion is operator-owned, stated as such
by triage on the issue;
4. the deliverable is **handed off** — the round passed, no `blocker:*`
stands, and you set `state:needs-human` per Handoff (below). The
remaining move is the human's merge.
5. the claim is **held by directive** — triage or the operator has told
you to stop, the direction names what the hold waits on, and that thing
is not yours to move. This is not "waiting for a good moment": somebody
else has decided the work must not proceed, and only they end it.
And it ends the same way it started: **on the labels.** When the queue
labels and any prose — an issue body header, a triage comment, an
operator's comment — disagree about whether a hold stands, the most
recent queue-label event by the hold's owner governs, and the prose is
stale until someone corrects it. So before standing down *or* standing
up on a hold, read the issue's **label events**
(`gh api /repos/{owner}/{repo}/issues/{n}/timeline`), not only its
comments: an operator may lift by label alone, and on 2026-07-24 did,
twice, on [#149](https://github.com/heavy-duty/ceremony/issues/149)
and [#151](https://github.com/heavy-duty/ceremony/issues/151). Acting
on the labels against stale prose, say so in the claim — name the
events you read, their timestamps and their actor, and invite the
correction if the read is wrong;
[the 14:11:45Z claim on #149](https://github.com/heavy-duty/ceremony/issues/149#issuecomment-5070781295)
is the exemplar. Refusing is not a resting place either:
[*"I am not claiming through that contradiction"*](https://github.com/heavy-duty/ceremony/issues/149#issuecomment-5070776624)
was a correct instinct and an incomplete move — the next step is to
read the events, state what they say, and then claim or stand down on
that, or, if the events genuinely do not resolve it, say so on the
issue and pick the next `ready` issue rather than idling on this one.
Not parked — these are what the rule defends against: waiting on
yourself, waiting on CI, or waiting for a good moment. An issue you have
simply stopped working on is not parked either — that is abandonment,
and its move is unchanged: unassign and restore `ready` (Claiming,
below).
The 2026-07-23 board is why the rule counts work and not claims: one
builder correctly held
[#15](https://github.com/heavy-duty/ceremony/issues/15) (`offsite`,
round answered whole, one verdict outstanding) and
[#16](https://github.com/heavy-duty/ceremony/issues/16) (`needs-ruling`
hard block, triage said hold) parked beside the one active build,
[#73](https://github.com/heavy-duty/ceremony/issues/73).
## Claiming
- Assign yourself, swap `ready` → `claimed`, and comment that you are
starting. The claim is a promise of a draft PR soon — a claim with no PR
and no activity is what the staleness sweep reclaims unless `offsite`
records that its PR lives in another repository.
- **A park is declared, never inferred.** When your claim enters a parked
shape (Picking, above), say so in a comment on that issue, naming what it
waits on and who owns the next move. No new label: the comment is
activity, so it feeds the same reclaim clock the `needs-ruling`
([#52](https://github.com/heavy-duty/ceremony/issues/52)) and `offsite`
([#68](https://github.com/heavy-duty/ceremony/issues/68)) exemptions
already guard — a parked claim nobody can name is an abandoned one.
Shape 4 alone is exempt from the separate comment: the handoff round
summary plus the `state:needs-human` write *is* its declaration — both
halves are already there, what the claim waits on (the merge) and who
owns the next move (the human), and both are visible to any scan as a
`labeled` event with the summary beside it. No second comment is owed
on the issue. Every other shape still declares as above.
- **Pick up `attention` before anything else.** On your claim, first post a
short pickup comment and remove `attention`; the removal is the ack. A
demand on a parked claim is usually its unpark, so take the slot back under
the existing rule below rather than leaving the demand parked. A demand
that *is* the park is different: the pickup comment is the declaration,
so one comment does both jobs, and the demand does not take the slot back.
- **A directed hold keeps its bookkeeping visible.** The PR carries `blocked`
with a comment naming what it waits on; the issue stays `claimed` and
carries `attention` until the builder acknowledges it. Nobody unassigns
the issue, and the 48-hour reclaim does not fire because the claim has an
open PR. Unparking follows the existing rule below.
- **Unparking is a claim like any other.** When the wait ends, the parked
issue is work again and takes the slot. If you are already active
elsewhere, finish or release that work first, and say which you did on
both issues — the slot is still one. Nothing counts claims per builder
and no reconciler path enforces any of this: `claim_decision()` sees one
issue at a time by construction, and no such machinery should be built
expecting it to have been specified here. The discipline is the
declaration, not a counter.
- **Abandoning is fine; ghosting is not.** If you stop, say where you got to,
push the branch if it holds anything useful, unassign, and restore
`ready`.
## Building
- Branch per issue; open the PR **as a draft early**, `Closes #N` in the
body. `Closes #N` does not cross repos: when the PR is in a different repo
from its authorizing issue, use `Part of <owner>/<repo>#N` instead, and
in the same step set `offsite` and comment on that issue with the draft PR
link as soon as the draft opens.
Triage closes the authorizing issue by hand when its acceptance criteria
are met; at that handoff the builder reports whether the cross-repo PR
merged or closed and clears `offsite` in the same comment. The cross-repo
merge never closes the authorizing issue. This codifies the linkage
builders already used on rig#112 and ceremony #13/#16 rather than adding a
new review obligation.
`Closes #N` also does not survive a post-merge criterion: when the issue's
body states that an acceptance criterion can only be checked after the
merge — a live proof of a workflow trigger, a released-artifact check,
anything whose subject does not exist until the change is on the base
branch — the same-repo PR uses `Refs #N` instead, and triage closes the
issue by hand on the evidence, exactly as it does for cross-repo work.
The issue body is what says so; you never judge which issues qualify, and
absent that instruction `Closes #N` remains the default. The exception was
bought the hard way: #143 carried `Closes #137` as doctrine then required,
and the merge closed #137 with its post-merge criterion unmet (#151).
Drafts are invisible to the reviewer panel on
purpose — the draft phase is yours.
- **The issue's acceptance criteria are your definition of done.** Reproduce
them as a checklist in the PR body and check them honestly as you go. If
one turns out to be wrong or unreachable, say so on the issue and get it
amended by triage — do not silently ship less than the issue says.
- Every behavior change writes one fragment, `changelog.d/<issue>.md`,
named for the authorizing issue (`<repo>-<issue>.md` when the work is
cross-repo) — the exact prose that will be published, nothing else: `- `
bullets, and in a grouped repo the `### Added` / `### Changed` /
`### Fixed` headings inside the fragment, creating a rarer kind only when
a change genuinely is one. Never edit `CHANGELOG.md` for an entry — the
release PR assembles the section from the fragments (#112); the monotonic
guard still refuses anything that deletes a shipped heading.
- Follow the repo's conventions file and match the code you touch. Tests are
not optional: the issue's test plan is the floor, not the ceiling.
- **Scope discipline: the PR does the issue — whole, and nothing else.**
Adjacent problems you discover go to a **discussion** (or a comment on the
relevant issue), where triage will do its job. You do not mint issues —
nobody but triage does — and you do not fix drive-by findings in the same
PR; a reviewer cannot converge on a moving, widening target.
## The review round
(If you are reading this as `.ceremony/BUILDER.md` in a governed repo: the
panel roster and any repo-specific flow notes live in that repo's own
CONTRIBUTING; everything below is the shared flow.)
1. Mark ready-for-review; request **the whole panel**. The panel is the roster
of the repo the **PR** is in, minus you — never the roster of the repo the
issue is in. The PR repo's `.github/labels.conf` `panel=` line is the
machine's answer; its CONTRIBUTING roster is the human-readable answer,
and `panel=` governs if they disagree because that is what the state
machine reads. If the PR repo names no roster, ask triage on the
authorizing issue before marking ready-for-review; do not guess. You may
request an off-panel reviewer, but say that their verdict is advisory and
does not become required. On rig#112 this distinction mattered: requesting
codex and grok was correct for rig's panel even though ceremony's bench was
larger, and the doctrine had not said which roster governed.
2. **Wait for every verdict, then answer the round whole** — one reply
covering every point, then push the fixes, then re-request exactly the
reviewers who did not approve. Prefer verification over argument: when a
reviewer doubts behavior, add the test that settles it.
3. Never dismiss a review, never merge, never mark your own work as passed.
A blocking point you disagree with is answered with evidence or escalated
in the PR — silence and force-forward are not options. A panel deadlock
is one kind of human-owned decision; use the ruling ask below
([#50 D11](https://github.com/heavy-duty/ceremony/issues/50)).
## The ruling ask
Set `needs-ruling` whenever a decision belongs to a human: org policy,
published artifacts, secrets, prod, or any choice whose cost lands outside
the PR. A panel deadlock is one instance, not the definition. The builder is
the accountable flag-setter on a PR and consolidates the decision into one
comment rather than forwarding several reviewers' phrasings
([#50 D11](https://github.com/heavy-duty/ceremony/issues/50)).
Keep at most these five lines above the fold and put all other analysis
inside the fold. The field labels are fixed because the ruling machinery
checks for them ([#50 D12](https://github.com/heavy-duty/ceremony/issues/50)):
```text
🧭 needs-ruling — <the decision, one line>
Options: A — <one clause> B — <one clause>
Recommend: A, because <one clause>.
Blocked: <what stops; what continues meanwhile>
Default: <A at 2026-07-23T21:00Z if no ruling> | none — hard block
<details><summary>Analysis</summary>…everything else…</details>
```
The options must be exhaustive and mutually exclusive; more than three means
the question is not ready. `Recommend:` is mandatory — omitting it hands the
whole problem to the human. `Blocked:` names both what stops and what
continues. Write a timed `Default:` only when you are affirmatively confident
the decision is reversible inside the PR before merge. Unsure is not a tie:
it is a hard block. Published artifacts, secrets, prod, and org policy are
hard blocks by construction ([#50 D12–D13](https://github.com/heavy-duty/ceremony/issues/50)).
The ladder is anchored to the current episode's `needs-ruling` **`labeled`
event**, not its `Default:` deadline or the last activity
([#50 D13–D14](https://github.com/heavy-duty/ceremony/issues/50)):
- **0–12h:** proceed when a still-clear, reversible default expires, and say
out loud that you did. A hard block waits.
- **at 12h:** do not fire a stale default. Re-read it against what has landed
and ask whether it still holds and whether reasonable doubt remains. If
doubt has appeared, make it a hard block.
- **at 24h:** proceed regardless, **as a PR**. Pick an option and state in the
PR body which way you went and what doubt remains. Nothing merges by this;
the human still gates the merge.
- **past 24h:** hand the choice to triage. Triage picks the option, records it
as a decision, and remains accountable; the operator can overturn it at
merge.
A re-flag starts a fresh ladder. The ladder applies whatever `Default:` says,
including a hard block, and an active back-and-forth still climbs it. This is
different from the 7-day nudge, which resets on real activity. The machine
observes both clocks but never sets, clears, or decides `needs-ruling`.
The label stays until agreement is *reached*, not until the maintainer
replies. The setter records the ruling, removes the label, and returns the
item to its flow in the same comment ([LABELS.md](LABELS.md)).
## Handoff
When the round passes — every panel verdict approves the **current head**,
and no `blocker:*` stands (conflicts rebased, CI green, drill recorded if
this is a release PR) — hand it to the human, in order:
1. post the round summary (what changed per round, what was verified);
2. request the human's review;
3. set `state:needs-human` yourself.
The label write is optimistic — the reconciler validates it, and takes it
back if the PR is not actually mergeable-right-now. Then stop: the PR is the
human's. The claim is now parked as shape 4 (Picking, above) — the handoff
you just posted is its declaration, and your build slot is free. Address
what comes back (`state:addressing`) and re-hand-off the same way.

197
.ceremony/LABELS.md
View file

@ -1,197 +0,0 @@
# Labels
The taxonomy shared across the heavy-duty repos. Only the `scope:` set
differs per repo (each repo's `.github/labels.conf` names its actual
surfaces); everything else below is core and identical everywhere, created by
the labels workflow's bootstrap dispatch (issue #10).
Two state machines share the taxonomy: the **PR machine** (proven in
box/rig/cast, reconciled by machinery) and the **issue flow** (the
triage → build queue, reconciled by the work-queue sweep). One rule joins
everything: **states are machine-owned, intent
labels are hand-set** — a hand-moved state label is a lie waiting to happen,
and the reconciler recomputes it from GitHub's own facts.
## PR state — who is the ball with? (exactly one per open PR)
| Label | Color | Waiting on |
|---|---|---|
| `state:building` | `#FBCA04` | the builder — PR is a draft |
| `state:bots-reviewing` | `#1D76DB` | the reviewer panel to finish the round (a request is live) |
| `state:addressing` | `#D93F0B` | the builder — round complete without full approval, or nobody was asked, or a blocker is up, or a ruling is pending |
| `state:needs-human` | `#8250DF` | the human — **this PR could be merged right now**: zero blockers, whole panel approved the current head |
`bots-reviewing` vs `addressing` is deliberate: staleness in the first means
*poke the reviewers*, in the second *the builder dropped the ball*. And
`state:needs-human` means exactly one thing — a human could merge this now —
so it requires zero blockers and head-current approvals; anything less and
the reconciler takes it back. The author sets it at handoff (the one
hand-set state); the `labeled` event fires the sweep that validates the
write within seconds.
## PR blockers — what is in the way? (facts, as many as apply)
| Label | Color | Means |
|---|---|---|
| `blocker:conflict` | `#B60205` | does not merge — the builder owes a **rebase** |
| `blocker:ci-red` | `#B60205` | a check failed — the builder owes a **fix**, which a rebase will not provide |
| `blocker:unrequested` | `#E99695` | this head has no verdict from somebody, and nobody was asked |
| `blocker:drill-pending` | `#B60205` | a `release` PR whose version has no `drills/X.Y.Z.md` record — correct but unevidenced (maintainer-created label; the bot bootstrap 403s on it) |
States answer *whose ball*; blockers answer *what's in the way*. They are
separate axes because the single-label version kept lying — independent facts
projected onto one totally-ordered label meant one always won and the losers
vanished off the board (box's `state:needs-rebase`, retired: the reconciler
strips it on sight).
## Issue flow — the work queue (exactly one per open, triaged, non-epic issue)
| Label | Color | Means | Set by |
|---|---|---|---|
| `needs-triage` | `#FBCA04` | an issue that did not come through triage — it owes normalization or conversion back to a discussion | anyone who spots one; cleared by triage |
| `ready` | `#0E8A16` | triaged, spec complete, unblocked — a builder can start now and succeed | triage |
| `claimed` | `#1D76DB` | a builder owns it: assignee set, a draft PR expected shortly | the claiming builder |
| `blocked` | `#6A737D` | waiting on another issue or PR (`Blocked by #N` in the body names it) | triage; anyone may correct it |
| `epic` | `#5319E7` | organizes other issues via a dependency-ordered task list; **builders never pick an epic** | triage |
The work-queue sweep enforces the invariant a board scan relies on: every open issue is either
`needs-triage`, `epic`, or carries exactly one of `ready` / `claimed` /
`blocked`. It flags conflicts rather than guessing intent. A `claimed` issue
with no open PR and no activity for 48 hours is reclaimed by the sweep: it
comments, unassigns the stale owner, and restores `ready`.
## Cross-cutting (PRs and issues)
| Label | Color | Meaning |
|---|---|---|
| `stale` | `#B60205` | no activity for 48h — sweep-managed, never hand-applied |
| `blocked` | `#6A737D` | (see above — same label serves PRs waiting on another PR/issue; legitimately quiet, the staleness sweep skips it) |
| `offsite` | `#CFD3D7` | issue deliverable is a PR in another repository; set by the builder with the draft link and cleared by the builder at handoff |
| `needs-ruling` | `#D4C5F9` | a human-owned decision is required; use BUILDER.md's ruling template and ladder. Set by triage or the builder; a state, not a signal — it clears on agreement, not on a reply |
| `attention` | `#D93F0B` | issue-only demand parked for the assignee; hand-set, and never written by the machine |
| `release` | `#0E8A16` | release flow, versioning, packaging work — and the ceremony PR itself |
| `merge-next` | `#0E8A16` | head of the merge queue — merge this one next. Queue order is *intent*: never set by the reconciler, only cleared by it |
`needs-ruling` marks where the human's turn is when the pending thing is a
*decision*, not a merge ([#50 D1–D14](https://github.com/heavy-duty/ceremony/issues/50)).
It applies to any human-owned decision — org policy, published artifacts,
secrets, prod, or any choice whose cost lands outside the work. A panel
deadlock is one instance, not the definition (D11). It is not
`state:needs-human`: that label means exactly "this PR could be merged right
now", and the retired `state:needs-rebase` is the family's proof that a
label meaning two things lies about both. It is not a `blocker:*` either:
every blocker names work the *builder* owes, a ruling is owed by the human —
and the flag must live on issues too, where blockers do not exist. On issues
it coexists with the queue labels (the one-of-three invariant above ignores
it); its color is the light shade of `state:needs-human`'s, so the human
axis reads as one family. It is a state, not a signal: set only with the
[canonical escalation contract](BUILDER.md#the-ruling-ask) (D12). A bare
flag is noise. The comment carries exhaustive, mutually exclusive options
(at most three), a mandatory recommendation, what stops and what continues,
and either a default affirmatively known to be reversible inside the PR or
`none — hard block`. Unsure is a block; published artifacts, secrets, prod,
and org policy are hard blocks by construction (D13).
The ruling ladder runs from the current episode's `needs-ruling` **`labeled`
event** (D13–D14):
- **0–12h:** a clear, reversible decision may proceed when its stated default
expires, saying out loud that it did; anything with reasonable doubt waits
as a hard block.
- **at 12h:** the setter re-reads the default against what has landed and asks
whether it still holds and whether doubt remains. A stale default does not
fire; new doubt makes it a hard block.
- **at 24h:** the builder proceeds regardless, **as a PR**, stating the option
chosen and the doubt that remains. Nothing merges by this; the human still
gates the merge.
- **past 24h:** triage picks the option, records it as a decision, and remains
accountable. The operator may overturn it at merge.
A re-flag starts a new ladder. The rungs apply whatever `Default:` says,
including a hard block. Active discussion still climbs the ladder; by
contrast, the separate 7-day nudge resets on real activity. The machine
observes the rungs but never sets, clears, or decides `needs-ruling`.
The flag stays up until agreement is *reached* — a human reply alone does not
clear it — and its setter closes it out: records the ruling as a decision in
one comment, removes the label, and returns the item to its flow in that same
comment, never as a side effect. If the human disagrees that agreement was
reached, the label goes back on. The reconciler refuses `state:needs-human`
while it stands (the PR falls to `state:addressing` — the ball on the PR is
the builder's, who carries the ruling in), and the staleness sweep skips it,
because waiting on a human is legitimately quiet. Quiet, but not unwatched
(#52, both surfaces): a flag set with no escalation comment from its setter
is called out by the sweep — comment-only, scoped to the labeled event, the
label never removed — and a ruling with no real activity for 7 days draws a
comment-only nudge addressed to the decider, linking the escalation. The
nudge carries no marker on purpose: the comment is itself activity, so it
resets its own window and never repeats within a quiet week. Label churn is
not activity — the clock reads comments, reviews and commits, or the sweep
would reset itself.
`offsite` is issue-only and records that a claimed issue's deliverable lives
in another repository, where a closing reference cannot make a local open PR
visible to the sweep (#68). The builder sets it in the same step that posts
the cross-repo draft link, then clears it at handoff in the same comment that
reports whether that PR merged or closed. The machine reads the flag and
never writes it. It stops only the claim-reclaim clock: missing assignees are
still flagged, queue-label conflicts and missing queue state are still
repaired, and epic-completion and PR-side stale behavior are unchanged. The
sweep tells the assignee once when every visible cross-referenced PR has
closed; it only tells, and never clears the flag or changes the claim.
`attention` is issue-only and says a demand is parked on an issue for its
assignee. Anyone who needs that assignee's hands — triage, the operator, or a
sibling agent — sets it. The assignee alone clears it, as the first act of
pickup together with a short comment; that removal is the acknowledgement
and re-arms the flag for the next demand. If the session dies before the ack,
the still-visible flag launches the next pickup instead. An unanswered flag
is auditable evidence on the board.
The flag is additive: it composes with `ready`, `claimed`, or `blocked` and
with `needs-ruling`, and never substitutes for queue state. It pauses no
clock. Unlike `offsite` and `needs-ruling`, which make silence legitimate,
unanswered `attention` is exactly the silence the 48-hour reclaim should
take. It is hand-set doctrine only: nothing in `actions/` sets, clears,
reads, or validates it, and no reconciler enforces the assignee requirement.
An `attention` issue without an assignee is therefore a board bug, not a
demand; anyone may assign it or remove the flag.
The three signals are mutually distinct: `attention` means an assignee owes
a move; `needs-ruling` means a human owes a decision under
[the escalation contract and ladder](BUILDER.md#the-ruling-ask); and a bare
`@`-mention is an FYI that demands nothing and remains perfectly fine. A
demand that is itself a human decision carries `needs-ruling`, never both.
This distinction records the
[#16 missed-ruling incident](https://github.com/heavy-duty/ceremony/issues/16#issuecomment-5061051198)
and why the rejected mention poll is not returning: ordinary thread traffic
re-arms mentions, but only the writer can declare that a move is owed (#83).
## Scope — which surface? (PRs and issues, any number)
All scopes share one calm color, `#C5DEF5` — scopes locate, states alert. The
set is per-repo: PRs get theirs from changed paths via the labels workflow's
scope job — an additive write only, so a label applied by hand or by an agent
while the machine runs always survives it (#130) — and issues get theirs from
triage. This file never enumerates a set — it is mirrored
byte-identically into every governed repo, and any list it carried would be
true in one repo and false in the rest (#104). The set for the repo you are
standing in lives in the two places that are true wherever you read them: its
`.github/labels.conf` (the definitions, one `name|color|description` row per
scope) and its own `CONTRIBUTING.md`, beside the other repo-specific facts.
## Issue types
`bug`, `enhancement`, `documentation` — issues only, set by triage. PRs carry
their type in the conventional title (`feat:`, `fix:`, `docs:`); a type label
on a PR would say the same thing twice and drift.
## Maintenance
The labels workflow (issue #10) recomputes PR state statelessly on PR events
plus a 15-minute advisory cron, and bootstraps this taxonomy idempotently on
manual dispatch. The sweep warns when the core taxonomy declares a label the
repository lacks. The same workflow reconciles issue-flow labels on issue
events and during the scheduled sweep. Default GitHub labels (`duplicate`,
`invalid`, `question`, `wontfix`, `help wanted`, `good first issue`) are
deleted at bootstrap — a `question` is a discussion, not an issue.

14
.ceremony/README.md
View file

@ -1,14 +0,0 @@
# .ceremony/ — the vendored doctrine mirror
Machine-managed by heavy-duty/ceremony's `actions/docs-sync`. Never edit
these files here: they are byte-identical copies of
[heavy-duty/ceremony](https://github.com/heavy-duty/ceremony) at this
repository's pinned ref, and CI re-diffs them on every PR — a hand edit
goes red. They are changed in heavy-duty/ceremony, through its own flow,
and arrive here when the pin moves.
The pin lives in `.github/workflows/release.yml` — the single
`uses: heavy-duty/ceremony/.github/workflows/release.yml@<ref>` line. One
pin governs machinery and doctrine alike: bump it and re-sync this mirror
in the same PR (`docs-sync --fix`, or let the red check on the bump PR say
what is stale).

139
.ceremony/REVIEWER.md
View file

@ -1,139 +0,0 @@
# REVIEWER.md — the reviewer role
You are one voice on a panel. The panel's job is to converge — on an
approval the human can trust, or on a precise statement of what is wrong.
The machine reads only your **verdict**; humans read your reasons.
## The verdict doctrine
- **Every review ends in a verdict**: approve, or request changes. A
comment-only review is a non-verdict — it does not say whether the round
passed, the state machine treats it as not-approved, and the PR simply
stalls. If you have an opinion, you have a verdict; commenting without one
only wedges the flow.
- **The verdict carries blockingness only; the body carries the feedback.**
Non-blocking nits ride an **approval**, and the builder addresses them at
their discretion. Anything blocking — including a question whose answer
gates your approval — is **request changes**, saying exactly what
unblocks it.
- **Name what you could not verify, in the verdict body.** Say which checks
you could not run and why, and what you relied on instead: CI, reading, or
a narrower probe. An unstated environment gap reads as coverage — exactly
the blind spot Kimi's [crew report](https://github.com/heavy-duty/crew/blob/main/kimi-bot-andresmgsl/assessment.md)
identified for boxes without `node` or `shellcheck`.
- An approval you would not defend to the human is a defect. You are not
being asked to be agreeable; you are being asked to be right.
## What you review against
In order of authority:
1. **The issue's acceptance criteria** — the PR's `Closes #N`, its
cross-repo `Part of <owner>/<repo>#N`, or its `Refs #N` when the issue
body marks a criterion post-merge, names your spec. That last shape is
not a defect: the issue directs it, triage owns that close, and a
request-changes on the "missing" keyword enforces the bug the shape
exists to fix — `Closes #137` closed its issue with a post-merge
criterion unmet (#151). Check every
criterion; a PR that ships less than the issue says is a request-changes
even if the code is beautiful.
2. **The repo's load-bearing constraints** — the rules bought with
incidents (in ceremony itself: issue #1's constraint list; in a governed
repo: its own CONTRIBUTING plus ceremony's README). A change that
"simplifies away" a constraint gets request-changes with a link to the
incident that made the rule.
- **Verify a pinned consumer at its pin, not ceremony's `main`.** Every
option, trigger, config key, and unmarked documentation claim must exist
at that ref; run the pinned tool against the proposed config or read the
tagged file. On [box#164](https://github.com/heavy-duty/box/pull/164),
`0.1.0`'s `load_config` rejected `triage-actors=...` with
`malformed label row` and `exit=1`. CI green on a conversion PR proves
nothing about the new config: the base branch's workflow is what ran.
3. **The code itself** — correctness first, then tests (does the test plan's
floor exist? do the failure cases actually fail?), then conventions.
Changelog line present for behavior changes; comments carry why, not
what.
**Verify over opine.** Run what can be run; construct the failing input; a
test settles what a comment thread can't. A review that says "I ran X and
saw Y" outranks one that says "this looks like it might".
## Where you review
- **A review request on you is your authorization** in any `heavy-duty` repo
and on any fleet member's fork. You need no separate permission and do not
wait for the repo to appear on a list: review is reversible
read-plus-comment work, and the requester already decided it should happen.
- **A request is authorization, not panel membership.** Convergence is
measured against the target repo's `panel=` roster minus the author. If you
are requested off-panel, post the verdict anyway and say in its body that
it is advisory; neither your silence nor your request-changes is a gate the
reconciler enforces. The nine-hour wait for kimi's off-panel verdict on
rig#112 showed why authorization and membership must not be conflated.
- **Being requested is a wake condition of its own.** It is how work in a
repo you have never heard of reaches you; a repo list finds only work in
repos somebody thought to list.
## How you work the queue
- **Your queue is the API, not the search index.** Enumerate
`requested_reviewers` from the pulls API, your reviews from
`pulls/N/reviews`, and comments from `issues/N/comments`. Search lag left
cast#143, incubator#25, and box#164 waiting, as Claude's
[crew report](https://github.com/heavy-duty/crew/blob/main/claude-bot-andresmgsl/assessment.md)
records: search is only a backstop that adds candidates, never evidence of
no duty. `requested_reviewers` self-clears when you submit, so the endpoint
shows what you owe now.
- **Every write is one-shot, keyed to (you, PR, head SHA).** Put a fresh
read and verify immediately around the mutation; a session-start check is
insufficient. If verification says it landed, stop even when the CLI
looked unhappy. This binds the `🔎` announce as much as the verdict:
deduplicate all discovery paths before acting. Duplicate verdicts on
[#26](https://github.com/heavy-duty/ceremony/pull/26),
[#29](https://github.com/heavy-duty/ceremony/pull/29), and
[#39](https://github.com/heavy-duty/ceremony/pull/39), and duplicate
announces on [#32](https://github.com/heavy-duty/ceremony/pull/32), bought
the rule; do not answer a double-post with a third comment.
- **Review each head in a throwaway checkout; keep the main clone clean.**
Use a detached worktree per PR head and remove it after the verdict. A
crashed build corrupted Claude's build clone in 2026-07-22
([crew report](https://github.com/heavy-duty/crew/blob/main/claude-bot-andresmgsl/knowledge.md));
running another tree in the clone you keep risks the whole box.
## What you do not do
- **Re-litigate the spec.** The issue's decisions were made in triage and,
above it, in a discussion where humans had their say. If you think the
spec itself is wrong, say so with reasons — as a comment pointing at the
discussion, while still reviewing the implementation against the spec as
written. Spec changes go through triage, not through a review round.
- **Merge, or tell the builder to merge.** Convergence hands the PR to a
human; only humans merge.
- **Approve a moving target.** Your approval is of a specific head. If the
builder pushes after your approval, GitHub stales it — that is correct,
and the builder owes a re-request, not an assumption.
## The round rhythm
- Review the **whole PR at the current head** each round, not just the diff
since your last comments — the fix for someone else's point can break
yours.
- The builder answers rounds whole and re-requests you; until re-requested,
the ball is not yours (`state:addressing` is the builder working — pile-on
reviews mid-address just churn the target).
- Convergence = every panel verdict approves the current head, no
`blocker:*` standing. Then the builder hands off (`state:needs-human`) and
the panel's job is done.
- Flag an unowned decision when it belongs to a human: org policy, published
artifacts, secrets, prod, or any choice whose cost lands outside the PR. A
disagreement within the panel is one instance, not the definition
([#50 D11](https://github.com/heavy-duty/ceremony/issues/50)). Argue a
panel disagreement in the PR with evidence until one side concedes or the
builder escalates; two reviewers pulling a builder in opposite directions
without resolution is a panel failure, not a builder failure.
`needs-ruling` is set by the **builder**, never by you: one accountable
flag-setter per PR hands the human one consolidated question. State the
unowned decision precisely enough for the builder to write
[the canonical ruling ask](BUILDER.md#the-ruling-ask), including what
stops and what continues ([#50 D12](https://github.com/heavy-duty/ceremony/issues/50);
[LABELS.md](LABELS.md)).

149
.ceremony/TRIAGE.md
View file

@ -1,149 +0,0 @@
# TRIAGE.md — the triage role
You are the only door issues come through. Humans and agents open
**discussions**; you decide what becomes work. The quality of every
downstream stage — a builder succeeding without asking, a reviewer having a
spec to review against — is set here, by you, and nowhere else.
## Why this door exists
Discussions are allowed to be ambiguous; issues are not. An issue is a work
order a builder must be able to execute **without asking anyone anything**.
Keeping one accountable role between the two is what keeps the bar from
eroding — the moment anyone can mint an issue, the backlog fills with
"improve X" entries nobody can build, and builders start guessing. Guessing
is the failure this whole flow exists to prevent.
## Your inputs
- **Every open discussion** in the repo you serve.
- **Stray issues** — anything filed directly, by anyone. Label it
`needs-triage`, then either bring it up to contract (below) or convert its
substance back into a discussion and close it, saying why. Do not shame the
filer; do route the work correctly.
## For each discussion, converge on exactly one outcome
1. **Answer.** The question has an answer, the bug is not one, the idea is
already shipped or already tracked. Reply with the answer (link the code,
the doc, the existing issue), mark answered.
2. **Ask.** Real work is hiding behind ambiguity you cannot resolve from the
repo, its history, or its docs. Ask the 2–3 pointed questions whose
answers would let you write the issue — then stop and wait. Do not mint an
issue that carries the ambiguity forward; that just moves your job onto
the builder.
3. **Escalate.** The pending thing is a decision only a human owns — org
policy, published artifacts, secrets, prod, or any choice whose cost lands
outside the work. A panel deadlock is one instance, not the definition
([#50 D11](https://github.com/heavy-duty/ceremony/issues/50)). Say
precisely what the decision is, name the decider, and use
[BUILDER.md's canonical ruling template](BUILDER.md#the-ruling-ask),
including its options, recommendation, blocked/continues statement, and
reversible-only default rules ([#50 D12–D13](https://github.com/heavy-duty/ceremony/issues/50)).
The discussion is where humans decide; wait there. When the decision
blocks something already on the board — an existing issue, or minted work
a discussion's ruling gates — set `needs-ruling` on it too, so the board
shows where the human's turn is; the issue keeps its queue label.
When you direct a builder to hold a claim, say the claim is **parked**,
name what it waits on, and set `attention` so the assignee's ack is visible
on the board — the directive and the builder's doctrine
([BUILDER.md](BUILDER.md#claiming)) must use one word.
Immediately before asserting label-borne state in prose — a hold, a
claim, a queue state, whether in a comment, a body header, or a
`needs-ruling` ask — re-read that issue's **label events**
(`gh api /repos/{owner}/{repo}/issues/{n}/timeline`), not just its
comments: the answer often arrives as a label with no comment, and a
write that re-read only the thread races it. Both 2026-07-24 failures —
[a header correction on #149](https://github.com/heavy-duty/ceremony/issues/149#issuecomment-5070758613)
asserting a hold 58 seconds after its lift, and
[a `needs-ruling` ask on #151](https://github.com/heavy-duty/ceremony/issues/151#issuecomment-5070768876)
the operator's label events had answered 132 seconds earlier — are this
sentence's absence.
Past 24 hours from the current episode's `labeled` event, if the ruling
still stands and doubt remains, it is triage's duty to pick the option the
builder proceeds on, record that pick as a decision, and stay accountable
for it; the operator may overturn it at merge
([#50 D13–D14](https://github.com/heavy-duty/ceremony/issues/50)). You set
the flag, so you also close it out ([LABELS.md](LABELS.md)): judge when
agreement is reached, record the ruling as a decision in one comment,
remove the label, and return the issue to its flow in that same comment;
when that ruling or any directive or answered builder question delivers
the assignee's next move in prose, set `attention` in the same comment.
This is not a substitute for minting work or for `needs-ruling`.
4. **Decline.** Real idea, wrong repo or wrong time. Say why plainly, link
where it belongs if anywhere, close. A refusal with reasons is a good
outcome; a zombie discussion is not.
5. **Accept.** It justifies work → mint the issue(s). The contract below is
the bar.
## The issue contract
Every issue you mint carries, in this order:
- **A title that names the deliverable** — "lib/version.sh — one version
abstraction, two backends", never "improve version handling".
- **Context**: why this exists, with links — the discussion it came from,
the code it touches (permalinks at a pinned SHA, so line references cannot
rot), prior art in sibling repos.
- **The spec**: decisions made, not options listed. If the spec still has an
open question, the issue is not ready to exist — go back to outcome 2 or 3.
- **Tasks**: the steps, checkboxed, in order.
- **Acceptance criteria**: checkboxed, verifiable, and honest — these become
the builder's definition of done and the reviewer's review spec, verbatim.
A criterion that can only be checked after the merge must carry its own
mechanism, in the criterion itself: that it is post-merge, that triage
owns the close, and that the PR references the issue with `Refs #N`
rather than `Closes #N`. A criterion that survives the merge only if
someone remembers to reopen the issue is an incomplete criterion — #137's
amended body is the worked example, reopened by hand after `Closes #137`
closed it with the criterion unmet (#151).
- **Test plan**: what proves it, including the cases that must fail.
- **Dependencies**: `Blocked by #N` / `Blocks #N`, and `Part of #E` when an
epic organizes it. Name a cross-repo dependency the same way with its
repository qualified (`Blocked by repo#N` or `owner/repo#N`); the sweep
cannot resolve it, so triage verifies it and flips the issue by hand.
- **Labels**: type (`bug`/`enhancement`/`documentation`), `scope:*`, and
exactly one of `ready` / `blocked` (see [LABELS.md](LABELS.md)).
The bar, stated once: **a competent builder who has read only this issue and
the repo can succeed.** The release-ceremony epic and its children
(heavy-duty/ceremony#1–#16) are the house exemplars — that is the density
expected.
## Multi-issue work
When an acceptance produces more than one issue, mint an **epic** (`epic`
label): the approach, the decisions, the constraint list, and a
dependency-ordered task list of child issues. Children reference the epic;
the epic's checklist is the progress view. Builders never pick the epic
itself. Keep the checklist current — a stale epic misleads every scan.
## Backlog hygiene
- **Dedup before minting** — search issues *and* closed issues; extend or
reopen before duplicating.
- The issue-flow sweep flips `blocked` → `ready` when every named dependency
lands, and flags a blocked issue whose dependency declaration is unreadable.
- The sweep reclaims abandoned claims after 48 hours: `claimed` + no open PR
+ no activity → comment, unassign, restore `ready`.
- Automation never guesses intent. Resolve the conflict comments it leaves on
malformed queue states, and close or extend completed epics when nudged.
- **Close obsolete issues** with the reason and a link to what obsoleted
them. Every label on every open issue stays true; the board is only worth
scanning if it does not lie.
- **A lifted hold makes its body prose stale in the same instant, and the
body is yours.** The "stays true" bar above extends past the labels to
the prose that describes them: when a hold lifts, correcting the body
header that described it is your move in the same tick — not the
builder's, and not left for the next reader to diff. On
[#149](https://github.com/heavy-duty/ceremony/issues/149) the lift
arrived by label alone and the body said held for the next five and a
half minutes; two builders read that window to opposite conclusions.
## What you never do
- Write code, review code, or build the thing yourself.
- Assign a builder — builders pick and claim ([BUILDER.md](BUILDER.md)).
- Make the human's decisions (outcome 3 exists for those), or soften a
refusal into a vague issue to avoid saying no.
- Mint an issue to "discuss" something — that is a discussion.

20
.github/DISCUSSION_TEMPLATE/ideas.yml vendored
View file

@ -1,20 +0,0 @@
# Copied from heavy-duty/ceremony@0.3.0 .github/DISCUSSION_TEMPLATE/ideas.yml (heavy-duty/ceremony#24). Hand-maintained: no guard checks this against ceremony.
# Light on purpose: discussions are where ambiguity is ALLOWED — a form
# that demands rigor at the door defeats the room's purpose (heavy-duty/ceremony#24,
# decision 4). If these prompts fight the flow in practice, delete them
# before adding fields to them.
body:
- type: textarea
attributes:
label: What's the itch?
description: >-
Vague is fine — a bug, an idea, a "we should…". Triage turns this
into work (or an answer); you don't have to.
validations:
required: false
- type: textarea
attributes:
label: What would "done" feel like?
description: Optional — a sketch of the outcome, if you can already see it.
validations:
required: false

16
.github/DISCUSSION_TEMPLATE/q-a.yml vendored
View file

@ -1,16 +0,0 @@
# Copied from heavy-duty/ceremony@0.3.0 .github/DISCUSSION_TEMPLATE/q-a.yml (heavy-duty/ceremony#24). Hand-maintained: no guard checks this against ceremony.
# Light on purpose — same reasoning as ideas.yml (heavy-duty/ceremony#24, decision 4):
# optional prompts only, delete before hardening.
body:
- type: textarea
attributes:
label: What's the question?
description: Ask it plainly — context and links help, none are required.
validations:
required: false
- type: textarea
attributes:
label: What have you tried or read already?
description: Optional — saves the answerer a lap.
validations:
required: false

14
.github/ISSUE_TEMPLATE/config.yml vendored
View file

@ -1,14 +0,0 @@
# Copied from heavy-duty/ceremony@0.3.0 .github/ISSUE_TEMPLATE/config.yml (heavy-duty/ceremony#24). Hand-maintained: no guard checks this against ceremony.
# The new-issue interception (heavy-duty/ceremony#24, decision 1): interception over
# instruction — prose alone has already proven insufficient everywhere else
# in this org. With blank issues disabled and exactly one form, a "New
# issue" click shows the discussion link first; that auto-suggest is native
# GitHub behavior, no automation needed.
blank_issues_enabled: false
contact_links:
- name: Ideas, bugs, questions — start a Discussion
url: https://github.com/heavy-duty/rig/discussions
about: >-
Humans (and agents) never file issues here — discussions are where
intent lives. Triage converges every discussion to an outcome, and
only triage mints issues (.ceremony/TRIAGE.md).

76
.github/ISSUE_TEMPLATE/work-order.yml vendored
View file

@ -1,76 +0,0 @@
# Copied from heavy-duty/ceremony@0.3.0 .github/ISSUE_TEMPLATE/work-order.yml (heavy-duty/ceremony#24). Hand-maintained: no guard checks this against ceremony.
# The issue contract (.ceremony/TRIAGE.md "The issue contract") as a form, so every
# minted issue lands with the same bones and a later parser (heavy-duty/ceremony#18) can key on
# stable section headings. Consistency, not a gate: the form applies no
# labels — queue labels are triage's explicit act (.ceremony/LABELS.md), and heavy-duty/ceremony#18's
# sweep is what catches non-triage authors, so the form must not pre-judge
# that (heavy-duty/ceremony#24, decision 2).
name: Work order (triage only)
description: >-
The issue contract (.ceremony/TRIAGE.md) as a form. Only triage mints issues —
everyone else starts a Discussion. Triage may still compose by hand when
the form fights it: `gh issue create --body-file` bypasses forms and stays
legitimate for the triage identity.
body:
- type: markdown
attributes:
value: >-
**The title names the deliverable** — "lib/version.sh — one version
abstraction, two backends", never "improve version handling". The
bar for everything below, stated once: a competent builder who has
read only this issue and the repo can succeed (.ceremony/TRIAGE.md). Labels
(type, `scope:*`, `ready`/`blocked`) are your explicit act after
minting — this form applies none.
- type: textarea
id: context
attributes:
label: Context
description: >-
Why this exists, with links — the discussion it came from, the code
it touches (permalinks at a pinned SHA, so line references cannot
rot), prior art in sibling repos.
validations:
required: true
- type: textarea
id: spec
attributes:
label: Spec
description: >-
Decisions made, not options listed. If the spec still has an open
question, the issue is not ready to exist.
validations:
required: true
- type: textarea
id: tasks
attributes:
label: Tasks
description: The steps, checkboxed, in order.
placeholder: "- [ ] the first step"
validations:
required: true
- type: textarea
id: acceptance-criteria
attributes:
label: Acceptance criteria
description: >-
Checkboxed, verifiable, and honest — these become the builder's
definition of done and the reviewer's review spec, verbatim.
placeholder: "- [ ] a verifiable outcome"
validations:
required: true
- type: textarea
id: test-plan
attributes:
label: Test plan
description: What proves it, including the cases that must fail.
validations:
required: true
- type: textarea
id: dependencies
attributes:
label: Dependencies
description: >-
`Blocked by #N` / `Blocks #N`, and `Part of #E` when an epic
organizes it — or state "None".
validations:
required: true

25
.github/labeler.yml vendored
View file

@ -11,9 +11,7 @@
"scope:runner": "scope:runner":
- changed-files: - changed-files:
- any-glob-to-any-file: - any-glob-to-any-file:
["commands/runner-*.sh", "commands/lib/runner-config.sh", ["commands/runner-*.sh", "commands/lib/runner-config.sh"]
"commands/forgejo-runner-*.sh", "commands/lib/forgejo-runner-config.sh",
"docs/templates/ci-box/**"]
"scope:coolify": "scope:coolify":
- changed-files: - changed-files:
- any-glob-to-any-file: ["commands/coolify-*.sh"] - any-glob-to-any-file: ["commands/coolify-*.sh"]
@ -23,24 +21,3 @@
"scope:installer": "scope:installer":
- changed-files: - changed-files:
- any-glob-to-any-file: ["install.sh"] - any-glob-to-any-file: ["install.sh"]
"scope:platform":
- changed-files:
- any-glob-to-any-file: ["commands/platform.sh"]
"scope:drill":
- changed-files:
- any-glob-to-any-file: ["drill/**", "drills/**"]
"scope:docs":
- changed-files:
- any-glob-to-any-file:
- "README.md"
- "CONTRIBUTING.md"
- "docs/**"
- ".github/ISSUE_TEMPLATE/**"
- ".github/DISCUSSION_TEMPLATE/**"
- ".github/pull_request_template.md"
"scope:labels":
- changed-files:
- any-glob-to-any-file:
- ".github/labels.conf"
- ".github/labeler.yml"
- ".github/workflows/labels.yml"

12
.github/labels.conf vendored
View file

@ -1,12 +0,0 @@
panel=cluade-reviewer-andresmgsl codex-reviewer-andresmgsl grok-reviewer-andresmgsl kimi-reviewer-andresmgsl
triage-actors=dan-claude-bot cluade-reviewer-andresmgsl
scope:bootstrap|C5DEF5|bootstrap — hardening a pristine server into a node
scope:users|C5DEF5|users-* — class model, apply/status, close-root
scope:runner|C5DEF5|runner-* / forgejo-runner-* — CI runner lifecycle, either forge
scope:coolify|C5DEF5|coolify-* — Coolify and backup install
scope:db|C5DEF5|db.sh — dump/restore
scope:installer|C5DEF5|install.sh — how rig lands on a machine
scope:platform|C5DEF5|rig platform — machine facts and identity, converged or not
scope:drill|C5DEF5|drill/ and drills/ — the harness, its legs and its records
scope:docs|C5DEF5|README, CONTRIBUTING, docs/ — the prose contract
scope:labels|C5DEF5|the labels workflow, the reconciler, labels.conf and labeler.yml

21
.github/pull_request_template.md vendored
View file

@ -1,21 +0,0 @@
<!-- Copied from heavy-duty/ceremony@0.3.0 .github/pull_request_template.md (heavy-duty/ceremony#24). Hand-maintained: no guard checks this against ceremony.
Short on purpose: a template longer than a small PR gets deleted by
reflex, not read (heavy-duty/ceremony#24). One issue, one PR — draft while building. -->
Closes #
## Acceptance criteria
<!-- The issue's checkboxes, VERBATIM — reviewers diff this against the
issue. Check them honestly as you go; a criterion you can't check goes back
to the issue for triage to amend, not silently unshipped. -->
## Changelog
- [ ] One `changelog.d/<issue>.md` fragment containing the exact publishable
prose — or no behavior change, stated here.
## Round log
<!-- Append each round's summary here: what changed, what was verified.
Rounds are answered whole — one reply covering every point. -->

244
.github/scripts/changelog-monotonic.sh vendored Executable file
View file

@ -0,0 +1,244 @@
#!/usr/bin/env bash
set -euo pipefail
# changelog-monotonic.sh [<base-ref>] [<changelog>] — assert that no SHIPPED
# release heading was DELETED by this branch: the set of '^## X.Y.Z' headings
# on HEAD must be a SUPERSET of the set at the merge base.
#
# The failure it exists to catch (#98; ported from heavy-duty/box#122, which
# was caught in review of box#118) leaves no trace. An author adding an entry
# under '## Unreleased' REPLACES the line below it instead of inserting above
# it:
#
# -## 0.2.0 — 2026-07-19
# +## Unreleased
# +
# +### Fixed
# +
# +- **An entry**
#
# git merges that cleanly — it is a one-line edit inside a file nobody has
# touched concurrently — so there is no conflict and no signal. 0.2.0's whole
# body is now sitting under '## Unreleased', and 0.2.0 has no section at all.
#
# The arming rule is green on exactly that tree, correctly. changelog_armed()
# in test/release.sh asks only whether the TOP section agrees with VERSION,
# and deleting '## 0.2.0' leaves '## Unreleased' on top. It is not wrong, it
# is narrow — it guards ONE heading, the one a PR is about to write under.
# This guards the REST of the file, the part no single tree can be asked
# about at all, because "a heading disappeared" is not a property of a tree —
# it is a property of a DIFF.
#
# The damage surfaces at the next release, in changelog_section()
# (.github/scripts/release-lib.sh), which anchors on the heading:
#
# awk -v ver="$2" '
# /^## / { if (found) exit; found = ($2 == ver); next }
# ...
#
# No heading, no section — and release.yml's "refusing to publish an empty
# release" assert is the first thing that notices, one whole release too late.
#
# The rule, and why it needs no tuning: release headings are APPEND-ONLY. The
# ceremony (CONTRIBUTING, "Releases") adds one and never removes one; nothing
# else in the documented flow touches them. So SUPERSET is exact — it has no
# legitimate violation to carve an exception for. The stamp is covered for
# free: rewriting '## Unreleased' -> '## X.Y.Z — DATE' ADDS X.Y.Z and removes
# no X.Y.Z heading, because 'Unreleased' is not one. '## Unreleased' is
# deliberately NOT in the set this guards — the arming rule owns that heading,
# keyed on VERSION, and the ceremony legitimately consumes it.
#
# A file of its own, NOT a clause inside test/release.sh's arming check, for
# three reasons. Its input is different (a git history, not two files). Its
# degradation is different (no base ref is a SKIP, not a failure). And the
# arming rule is driven by test/release.sh against constructed VERSION +
# CHANGELOG.md trees that are not git repos at all — folding a git-dependent
# assert into it would make every one of those cases either skip or lie.
# Same discipline as release-lib.sh: its own file so a test can drive it.
base_ref="${1:-${CHANGELOG_MONOTONIC_BASE:-origin/main}}"
changelog="${2:-CHANGELOG.md}"
# Fail-closed switch: CI sets it, so a SKIP that would be a sensible local
# degradation becomes a red run there instead. A guard that can silently
# stop guarding is the failure shape this whole family of checks exists to
# refuse, so the skip path is loud and CI refuses to take it at all.
strict="${CHANGELOG_MONOTONIC_STRICT:-0}"
skip() {
if [ "$strict" = "1" ]; then
echo "changelog-monotonic: $* — and CHANGELOG_MONOTONIC_STRICT=1, so this is a FAILURE, not a skip." >&2
echo " CI sets STRICT because a guard that quietly stops guarding is worse than no guard." >&2
echo " (Uniqueness on HEAD already passed; it is containment that cannot run.)" >&2
echo " Fix the checkout, not this script: the base ref must be fetched (fetch-depth: 0)." >&2
exit 1
fi
echo "changelog-monotonic: containment SKIPPED — $*"
echo " (Uniqueness on HEAD already ran and passed — only the deleted-heading"
echo " half needs the history. In CI this same condition is a hard failure.)"
exit 0
}
[ -f "$changelog" ] || { echo "changelog-monotonic: no such file: $changelog" >&2; exit 1; }
# The set of RELEASE headings: '## <token> ...' where <token> looks like a
# version. Field $2, the same split changelog_section() uses, so the two
# cannot disagree about what a section header is. 'Unreleased' fails the
# shape and is excluded by construction.
headings_raw() {
awk '
/^## / && $2 ~ /^[0-9]+\.[0-9]+\.[0-9]+/ { print $2 }
'
}
headings() { headings_raw | sort -u; }
# --- uniqueness on HEAD (the box#118 class) ----------------------------------
# Containment catches a DELETED heading. It cannot catch a DUPLICATED one: the
# duplicate is head-side SURPLUS, and `comm -23` (base minus head) is blind to
# extras on the head side — with or without `sort -u`, base {0.2.0} minus head
# {0.2.0, 0.2.0} is empty. Multiset comparison does not close it either, for
# the same reason. The assert that does is uniqueness of version headings ON
# HEAD, kept alongside containment rather than replacing it.
#
# This is the shape box#118's bad rebase produced: two `## 0.2.0 — 2026-07-19`
# headings with an incoming entry between them. Every other guard stays green
# — conflict markers absent, the arming rule happy (the top section is still
# right), tests and shellcheck clean.
#
# rig's symptom differs from box's, and the difference matters. box's
# release-notes.sh RE-ARMS its grab on every matching '## ' line, so a
# duplicate makes it ABSORB whatever sits between the copies. rig's
# changelog_section() has `if (found) exit`, so it stops dead at the second
# copy instead: a duplicate TRUNCATES. The published body is only what sits
# BETWEEN the two headings, and everything under the second copy — the real
# body of that release — is silently dropped. Different symptom, same class:
# no conflict, no red run, discovered only by a human reading the published
# notes.
#
# Nothing legitimate repeats a version heading: the ceremony stamps a NEW
# version, and 'Unreleased' fails the version shape and never reaches here.
dupes="$(headings_raw < "$changelog" | sort | uniq -d)"
if [ -n "$dupes" ]; then
{
echo "changelog-monotonic: $changelog has DUPLICATE release heading(s):"
echo
printf '%s\n' "$dupes" | sed 's/^/ ## /'
echo
cat <<EOF
Each version heading must appear exactly once. A repeat splits one release
into two same-named sections, and changelog_section() stops at the FIRST
'## ' line after the one it matched — so the published body for that version
is only what sits BETWEEN the copies, and the real body under the second
copy is dropped from the release notes entirely.
This is the box#118 shape: an entry meant for '## Unreleased' was inserted
after a shipped heading, and the heading re-added below it. The fix is one
heading, with the entry above it under '## Unreleased':
## Unreleased
### Fixed
- **Your entry**
## $(printf '%s\n' "$dupes" | head -1) — DATE <- exactly once
Quick check on any changelog-touching rebase:
diff <(git show origin/main:$changelog | grep '^## ') <(grep '^## ' $changelog)
EOF
} >&2
exit 1
fi
# --- everything below needs the HISTORY --------------------------------------
# Uniqueness is settled. What follows is containment, which compares HEAD
# against the merge base and therefore genuinely depends on the base ref, the
# merge base, and the base blob. Each of those can be unavailable for reasons
# that are not the author's fault (a shallow clone, a fork checkout without the
# upstream remote, the commit that first adds the changelog), so each degrades
# rather than failing — which is exactly why the uniqueness half must NOT live
# down here (#98; fixed upstream in heavy-duty/box#143, where rig's copy of
# this script came from). It asks nothing of the history, and gating it behind
# these conditions let a duplicate exit 0 on a message about deletion.
git rev-parse --is-inside-work-tree >/dev/null 2>&1 \
|| skip "not inside a git work tree, so there is no history to compare against"
git rev-parse --verify --quiet "$base_ref^{commit}" >/dev/null \
|| skip "base ref '$base_ref' does not resolve here (a shallow clone, or a fork checkout without the upstream remote)"
merge_base="$(git merge-base "$base_ref" HEAD 2>/dev/null || true)"
[ -n "$merge_base" ] \
|| skip "no merge base between '$base_ref' and HEAD (unrelated histories, or a clone too shallow to reach one)"
# The changelog may not exist at the merge base at all (the commit that adds
# it). Nothing to have deleted, so nothing to assert.
base_file="$(git show "$merge_base:$changelog" 2>/dev/null || true)"
[ -n "$base_file" ] || {
echo "changelog-monotonic: $changelog does not exist at the merge base ($(git rev-parse --short "$merge_base")) — nothing could have been deleted (uniqueness on HEAD already passed)."
exit 0
}
base_headings="$(printf '%s\n' "$base_file" | headings)"
head_headings="$(headings < "$changelog")"
# comm -23: lines in the base set that are NOT in the head set — exactly the
# headings this branch removed.
missing="$(comm -23 <(printf '%s\n' "$base_headings") <(printf '%s\n' "$head_headings"))"
if [ -n "$missing" ]; then
{
echo "changelog-monotonic: this branch DELETES release heading(s) from $changelog:"
echo
printf '%s\n' "$missing" | sed 's/^/ ## /'
echo
cat <<EOF
Present at the merge base ($(git rev-parse --short "$merge_base")), absent on HEAD.
Release headings are APPEND-ONLY. The ceremony adds one (CONTRIBUTING,
"Releases"); nothing ever legitimately removes one. So this is not a
judgement call — it is a defect, and almost always the same one (#98): an
entry written under '## Unreleased' REPLACED the heading below it instead of
being inserted ABOVE it. The shipped section's body is now sitting under
'## Unreleased', and the version it belonged to has no section at all.
Nothing else will say so. git merges that edit cleanly — no conflict, no
signal — and the arming rule stays green, because the TOP section is still
the right one for this VERSION. The damage surfaces at the NEXT release,
when changelog_section() cannot find the section it extracts by heading and
release.yml refuses to publish an empty release — one whole release late.
The fix is to put the heading back and INSERT above it, never over it:
## Unreleased
### Fixed
- **Your entry**
## $(printf '%s\n' "$missing" | head -1) — DATE <- untouched, still here
If you are genuinely renaming a released version, that is a rewrite of
history this guard is meant to stop; say so in the PR and change the guard
deliberately, in its own commit.
EOF
} >&2
exit 1
fi
count="$(printf '%s\n' "$base_headings" | grep -c . || true)"
head_count="$(printf '%s\n' "$head_headings" | grep -c . || true)"
# The success line has two honest forms, because this step now runs on two
# shapes of event. On a push to main the merge base IS HEAD: containment
# compared the file against itself and asserted nothing, and deletion is
# undetectable on that event by construction. Reporting "all N still present"
# there would be the same dishonesty the skip messages were fixed for in #98 —
# a log claiming a check that did no work. Uniqueness is the half that actually
# ran, so that is the half the line names.
if [ "$merge_base" = "$(git rev-parse HEAD)" ]; then
echo "changelog-monotonic: containment vacuous (the merge base IS HEAD, so nothing could have been deleted between them) — uniqueness on HEAD checked $head_count release heading(s)."
else
echo "changelog-monotonic: all $count release heading(s) at the merge base ($(git rev-parse --short "$merge_base")) are still present in $changelog"
fi

166
.github/scripts/drill-recorded.sh vendored Executable file
View file

@ -0,0 +1,166 @@
#!/usr/bin/env bash
set -euo pipefail
# drill-recorded.sh [<drills-dir>] [<version-file>] — assert that the version
# this tree is about to ship has a DRILL RECORD at <drills-dir>/<version>.md.
#
# defaults: drills VERSION
#
# CONTRIBUTING ("Releasing") says a release carries a real-hardware drill.
# Nothing enforced it, so no release in this family has ever had one: the
# ceremony is four correct mechanical steps — bump VERSION, stamp the
# changelog, re-arm, merge — and every one of them is checked by a script,
# while the one step that costs an afternoon on real hardware was checked by
# a reviewer remembering. Reviewers remember exactly as long as the release is
# interesting, which is never at 0.4.3. A bot finally blocked on it; this is
# that block, moved into CI where it does not depend on anyone's attention.
#
# ONE FILE PER VERSION, which is what this script is now mostly about. The
# first cut of this guard kept every record as a section inside one
# drill/RUNS.md, and paid for it: it needed an awk extractor that matched a
# literal '## Release drill — ' prefix, tolerated an optional ' — DATE' tail,
# compared the version WHOLE so that '0.3.0-rc1' could not answer for '0.3.0',
# and then separately insisted the extracted body hold a non-blank line. Every
# one of those rules existed only because records shared a file. Both sibling
# repos shipped a DEFECT out of that complexity during review — a
# `sed '/./,$!d'` extractor where `.` matches a space, so a heading plus one
# tab satisfied the gate (box#149, cast#138), and heading-grammar drift on the
# other side. Splitting the records makes nearly all of it unrepresentable:
# `0.3.0.md` and `0.3.0-rc1.md` are simply different files, there is no
# heading to parse and no grammar to drift, and the whole-version comparison
# is done by the filesystem.
#
# PER-REPO, and that is the load-bearing design decision. The obvious
# alternative — have rig ask box's repo whether the drill ran — cannot fail
# safely: the lookup needs a network call, a token, and a checkout that may be
# a fork, and every one of those failure modes lands on "could not read", which
# a naive implementation spells `|| true` and reads as PASS. That is exactly
# the UNREADABLE-vs-NONE bug #90 fixed one layer up (an unreadable check rollup
# reading as "nothing is failing"), and re-introducing it in the release gate
# would be worse: it degrades to green on precisely the tree that ships. So rig
# records rig's own legs in rig's own repo, and this script reads a file that
# is either in the checkout or is not.
#
# The directory is `drills/`, NOT `.drills/`. A dot-directory is invisible to
# every glob that has not set `dotglob`, which is how #70 here and box#116 /
# box#118 all happened: a file that exists but that no sweep can see is worse
# than no file, because it reads as covered.
#
# What it asserts is a RECORD, not a RESULT — and that is deliberate, not a
# weakness. A gate that demanded "the drill passed" would have to parse
# somebody's prose for a verdict, and would leave a maintainer who consciously
# ships without a full drill (a doc-only release, a hardware outage) with no
# move except deleting the check. Requiring a record means the waiver is
# WRITTEN DOWN, in a file named for the version it applies to, in a commit a
# reviewer sees. Skipping stays possible; skipping silently does not.
#
# Vacuous on a `-dev` tree, which is why it needs no trigger scoping in
# ci.yml (unlike changelog-monotonic.sh, whose input is a diff): every ordinary
# PR carries a `-dev` VERSION and passes without a drill record existing at
# all. The check has something to say on exactly one tree — the release
# ceremony PR — and that is the tree it must be impossible to merge without.
drills="${1:-drills}"
version_file="${2:-VERSION}"
# An unreadable version file is an ERROR, never a silent pass. There is no
# version to be lenient about, so leniency here could only mean "ship
# unevidenced" — the exact degradation the per-repo decision above exists to
# avoid.
[ -f "$version_file" ] || {
echo "drill-recorded: no such file: $version_file" >&2
exit 1
}
version="$(tr -d '[:space:]' < "$version_file")"
[ -n "$version" ] || {
echo "drill-recorded: $version_file is empty — there is no version to check a drill against." >&2
exit 1
}
# The -dev half. A development tree is not shipping anything, so there is
# nothing to evidence; saying so out loud (rather than exiting 0 in silence)
# is the #98 lesson — a guard that prints nothing is indistinguishable from a
# guard that did nothing.
case "$version" in
*-dev)
echo "drill-recorded: VERSION is $version — a development tree has nothing to assert (the drill gates a RELEASE, and this is not one)."
exit 0
;;
esac
record="$drills/$version.md"
# WHITESPACE IS NOT A RECORD. This is the one surviving piece of the rule set
# the old section-parsing guard needed, and it survives because it is the one
# part that splitting the files does not make unrepresentable: an empty file,
# or a file holding only spaces, tabs and newlines, exists at the right path
# and is still no evidence. It is the same property box#149 and cast#138 both
# got wrong with `sed '/./,$!d'` (`.` matches a space), where a record of one
# tab shipped an evidence-free release. `grep -q '[^[:space:]]'` is the whole
# check now, with no extractor in front of it to get wrong.
#
# The negated form below, matching box's and cast's twins exactly, so there is
# no divergence between the three to explain.
#
# It also avoids a real `set -e` hazard, which is worth naming precisely
# because an earlier draft of this comment named it BACKWARDS. A bare
# `[ -f "$record" ] && grep -q ... "$record"` mid-script does NOT abort when
# the file is missing: the left-hand side of `&&` is exempt from errexit, so a
# miss simply continues. What DOES abort is the other case — the file exists
# and `grep` finds nothing, i.e. exactly the whitespace-only record this guard
# is here to refuse. The script would die on its most interesting input,
# before printing the message that explains it.
#
# Verified rather than reasoned about:
# bash -ec '[ -f /nonexistent ] && r=yes; echo reached' -> prints, exit 0
# bash -ec 'f=$(mktemp); echo " " >"$f"
# [ -f "$f" ] && grep -q "[^[:space:]]" "$f"
# echo reached' -> silent, exit 1
#
# Caught by all three reviewers on #104. The lesson is the same one #149 and
# cast#138 taught: this family's comments get read as contracts, so a comment
# that misstates the semantics is a defect even when the code is correct.
if [ ! -f "$record" ] || ! grep -q '[^[:space:]]' "$record"; then
{
echo "drill-recorded: VERSION is $version, and there is no drill record at $record."
echo
cat <<EOF
This tree is a release ceremony tree — VERSION is bare, so merging it ships
$version. CONTRIBUTING ("Releasing") requires that release to carry a real
hardware drill, recorded in a file named for the version, exactly:
$drills/$version.md
One file per version, so the name IS the match: a record for
$version-rc1 lives at a different path and does not count. The file must
hold at least one non-whitespace character — an empty file, or one of only
spaces and tabs, is not a record.
Two ways to unblock, and both are a commit on this PR:
1. RUN THE DRILL and record it. What ran, on what hardware, the numbers,
and what failed. rig's drill asserts CONVERGENCE — a machine reaches
its role, idempotently — against a PINNED set of candidate refs
(RIG_REPO/RIG_REF and BOX_REF are mint-time variables, so the run pins
the commits under test). Drilling the candidate IS drilling the
release, since a release PR's diff is VERSION + CHANGELOG.md and
nothing executable differs. Cite the run ID and the other repos' SHAs.
The three repos' drills are independent — rig's does not wait on box's.
2. RECORD AN EXPLICIT MAINTAINER WAIVER in that same file, saying who
waived it and why. This guard asks for a RECORD, not a passing result,
so a deliberate skip is allowed — it just has to be visible and
reviewable rather than silent.
See $drills/README.md for what a record should contain.
Do not delete this step to get green. A release that cannot say what was
drilled is the state this check exists to end.
EOF
} >&2
exit 1
fi
lines="$(grep -c . "$record" || true)"
echo "drill-recorded: $record records a drill for $version ($lines non-blank line(s))."

486
.github/scripts/labels-reconcile.sh vendored Normal file
View file

@ -0,0 +1,486 @@
#!/usr/bin/env bash
set -euo pipefail
# labels-reconcile.sh — the automation LABELS.md promises: state labels are
# written by machinery, never by hand. Every run derives each open PR's
# state:* from GitHub's own facts (draft flag, requested reviewers, submitted
# reviews) and converges the labels to it, so a killed run or a hand-moved
# label heals on the next pass. Stale is judged from real activity — commits,
# comments, reviews — never from label churn, or the sweep would un-stale its
# own mark every tick.
#
# The verdict contract (CONTRIBUTING.md): reviews end in approve or
# request-changes. Some live bots are comment-only and post agreement as a
# COMMENTED review — a non-verdict this machine refuses to guess about (body
# parsing is a heuristic, and a wrong guess promotes an unapproved PR). The
# judgment call belongs to the PR AUTHOR, who reads the round and escalates
# by requesting the human's review — an explicit request is a fact, and it is
# the one this machine trusts (see decide_state's top precedence). The
# machine auto-requests the human only in the no-judgment-needed case: three
# formal head-current approvals. Any approval that counts must be bound to
# the CURRENT head SHA: GitHub keeps approvals alive across pushes, and a
# stale approval must never promote unreviewed code to the human.
#
# DRY_RUN=1 narrates every mutation instead of performing it (how this script
# is rehearsed against the live repo). A workflow_dispatch run also bootstraps
# the taxonomy (label create --force) — that heal is dispatch-only; the cron
# sweep tolerates a missing label rather than recreating it.
#
# The state machine below is pure (globals in, state out) and covered by
# fixture tests in test/labels-reconcile.sh.
HUMAN="${HUMAN_REVIEWER:-danmt}"
BOTS=(claude-bot-andresmgsl codex-bot-andresmgsl grok-bot-andresmgsl)
STATES=(state:building state:bots-reviewing state:addressing state:needs-human)
BLOCKERS=(blocker:conflict blocker:ci-red blocker:unrequested)
# Labels this machine used to own and no longer does. Cleared on sight so a
# retirement heals the board instead of stranding a label nothing recomputes.
RETIRED=(state:needs-rebase)
STALE_AFTER=$((48 * 3600))
log() { printf 'labels: %s\n' "$*"; }
run() { # every mutation goes through here — DRY_RUN=1 logs instead of doing
if [ -n "${DRY_RUN:-}" ]; then log "DRY_RUN: $*"; else "$@"; fi
}
# ---------------------------------------------------------------------------
# The state machine. Pure functions over four globals, set per PR:
# DRAFT true|false
# HEAD_SHA the PR's current head commit
# REQUESTED newline-separated logins with a review currently requested
# REVIEWS_JSON JSON array of submitted (non-PENDING) reviews
# MERGEABLE MERGEABLE | CONFLICTING | UNKNOWN (GitHub's own verdict)
# CHECKS SUCCESS | FAILURE | PENDING | NONE (the check rollup)
# ---------------------------------------------------------------------------
requested() { grep -qxF "$1" <<<"$REQUESTED"; }
checks_state() { # rollup JSON on stdin → SUCCESS | FAILURE | PENDING | NONE | UNREADABLE
# UNREADABLE is the absence of the key itself, which is what a failed fetch
# leaves behind — distinct from a present-but-empty rollup, which honestly
# means this PR has no checks. Collapsing the two let an API hiccup present
# as "nothing is failing", i.e. as mergeable-by-a-human: the same
# unknown-certified-as-green shape as the bug this machine exists to stop.
# The caller skips the PR entirely rather than labelling on facts it did not
# read; blocking on it instead would flap the whole board on one bad call.
# The rollup mixes two node types with two different closed enums: CheckRun
# carries `conclusion` (CheckConclusionState), StatusContext carries `state`
# (StatusState). Rather than list the outcomes that block — the version that
# shipped in this PR's first round listed four, and ERROR, CANCELLED and
# STALE fell through its `else` into SUCCESS — this lists the outcomes that
# DON'T, and treats everything else as blocking.
#
# That direction is the point. An outcome we do not recognise is one we
# cannot certify as mergeable, and certifying the unrecognised as green is
# the exact shape of #136. The cost of being wrong is symmetric in form and
# not in consequence: a false FAILURE parks the PR on the agent, who looks;
# a false SUCCESS invites a human to merge a tree that will not merge.
jq -r '
if (has("statusCheckRollup") | not) then "UNREADABLE" else
# NEUTRAL and SKIPPED satisfy branch protection — a skipped required check
# is not a failed one, and path-filtered jobs skip constantly here.
["SUCCESS", "NEUTRAL", "SKIPPED"] as $passing
# "" covers a StatusContext still reported with no state at all.
| ["", "PENDING", "IN_PROGRESS", "QUEUED", "WAITING", "REQUESTED", "EXPECTED"] as $waiting
# A re-run does not evict the run it superseded — the rollup keeps both.
# This PR proved it: its own tip carried a CANCELLED `scope` (15:19:39)
# beside the SUCCESS `scope` (15:19:45) that replaced it, same workflow.
# Once CANCELLED blocks, judging every entry would strand this very PR in
# needs-rebase forever, so collapse each context to its newest entry first.
# Key on workflow + name because a bare job name is only unique within its
# workflow.
#
# Dating a run is the subtle part, and getting it wrong restores the bug.
# A run still in flight has no completion, but `gh` does not omit the
# field: its Go struct marshals the zero time as "0001-01-01T00:00:00Z",
# which is a string, so `//` will not fall through it. Ordering on
# completion therefore sorted the LIVE re-run to the bottom and let `last`
# pick the very run it superseded — reporting the old SUCCESS while a
# replacement was still running, which is #136 again.
#
# So: date a run by when it BEGAN, discarding both spellings of absent
# (null, and the zero sentinel) and falling back only if it never recorded
# a beginning. NOT by the newest stamp of any kind: `max` compares the
# completion of a finished run against the start of a live one, which are
# different quantities and not an ordering on runs. A run cancelled by the
# concurrency group does not stop the instant its replacement starts — the
# runner has to wind down — so predecessor.completedAt > successor.startedAt
# is the ordinary case, and `max` dated the dead predecessor newer than the
# live run that replaced it, narrowing both failures above without closing
# them. The list is already in preference order, so `first` IS that rule.
#
# An entry that carries no usable timestamp at all sorts LAST rather than
# first — something we cannot date is most likely the thing just created,
# and treating it as newest keeps an undateable in-flight run from being
# discarded in favour of a stale success. Every ambiguity resolves toward
# "not settled".
| [ (.statusCheckRollup // [])[]
| { ctx: [.workflowName // "", .name // .context // ""],
at: ([.startedAt, .createdAt, .completedAt]
| map(select(type == "string" and . != ""
and (startswith("0001-01-01") | not)))
| first // ""),
outcome: ((.conclusion // .state // "") | ascii_upcase) } ]
| group_by(.ctx)
| map(sort_by([(.at == ""), .at]) | last | .outcome) as $latest
| if ($latest | length) == 0 then "NONE"
elif (($latest - $passing - $waiting) | length) > 0 then "FAILURE"
elif (($latest - $passing) | length) > 0 then "PENDING"
else "SUCCESS" end
end'
}
bot_verdict() { # $1 = login → MISSING | BLOCK | APPROVE | STALE | FEEDBACK
local review state commit
review="$(jq -c --arg u "$1" \
'[.[] | select(.user.login == $u)] | sort_by(.submitted_at) | last // empty' \
<<<"$REVIEWS_JSON")"
if [ -z "$review" ]; then echo MISSING; return; fi
state="$(jq -r '.state' <<<"$review")"
commit="$(jq -r '.commit_id' <<<"$review")"
case "$state" in
CHANGES_REQUESTED)
# blocks at ANY head — GitHub's own semantic: only a newer review
# from the same reviewer clears it
echo BLOCK ;;
APPROVED)
if [ "$commit" = "$HEAD_SHA" ]; then echo APPROVE; else echo STALE; fi ;;
*)
# COMMENTED and anything else: a non-verdict. The machine does not
# read bodies — if the comment is really an agreement, the AUTHOR
# says so by requesting the human's review.
echo FEEDBACK ;;
esac
}
human_request_needed() { # 0 when needs-human requires a FRESH human request
# already requested → the handoff is live; head-current human approval →
# nothing left to ask. Anything else (never reviewed, an old comment, an
# approval of an older head) stalls the handoff unless we request —
# guarding on "has the human ever reviewed" wedged exactly that way.
if requested "$HUMAN"; then return 1; fi
if [ "$(bot_verdict "$HUMAN")" = APPROVE ]; then return 1; fi
return 0
}
blockers() { # → the blocker:* labels this PR should carry, one per line
# The second axis. These are FACTS ABOUT THE BRANCH, and they are mutually
# independent — a PR can be conflicted and red and unasked at once — so they
# are a set, not an ordering. That is the whole point of splitting them out
# of state:*: every precedence bug this machine has had (needs-human
# surviving a conflict, MISSING swallowing STALE) came from projecting
# independent facts onto one totally-ordered label. A set has no precedence
# to get wrong.
#
# UNKNOWN mergeability is deliberately NOT a conflict: GitHub reports it for
# about a minute after every merge while it recomputes, and flapping every
# open PR on each merge would be worse than the bug. Same for a failed read
# of either fact — both default to the "do not know" value, which blocks
# nothing. An unset global (an older fixture, a failed fetch) must never
# invent a verdict it did not read.
case "${MERGEABLE:-UNKNOWN}" in CONFLICTING) echo blocker:conflict ;; esac
case "${CHECKS:-NONE}" in FAILURE) echo blocker:ci-red ;; esac
# Nobody is on the hook for a verdict somebody still owes. Distinct from
# bots-reviewing, which says a request is live and an answer is coming:
# here the round is stalled because no one was ever asked, and the board
# said "waiting on the bots" for the 48h it took `stale` to notice.
# A draft is exempt (the bots ignore drafts by design), and so is an
# explicit human request — a maintainer claiming a PR early is deliberate,
# not a dropped ball.
if [ "$DRAFT" != true ] && ! requested "$HUMAN"; then
local b v owed=false any_requested=false
for b in "${BOTS[@]}"; do
requested "$b" && any_requested=true
# MISSING and STALE are both verdicts this head does not have: nobody
# reviewed it, or everybody reviewed something else. The agent owes an
# ask either way — the stale round is if anything the worse of the two,
# since it has approvals on the page that no longer describe the tree.
v="$(bot_verdict "$b")"
case "$v" in MISSING | STALE) owed=true ;; esac
done
if [ "$owed" = true ] && [ "$any_requested" = false ]; then
echo blocker:unrequested
fi
fi
}
decide_state() { # → the one state:* label this PR should carry
if [ "$DRAFT" = true ]; then echo state:building; return; fi
local s
s="$(round_state)"
# The one rule joining the two axes: state:needs-human means a human could
# merge this RIGHT NOW, so it requires a clear branch. Any blocker at all
# means the work is the agent's — whatever the review round says — and the
# blocker label says which work it is. Nothing else in this function reads
# the branch, which is what keeps the ordering below purely about reviews.
if [ "$s" = state:needs-human ] && [ -n "$(blockers)" ]; then
echo state:addressing; return
fi
echo "$s"
}
round_state() { # → the state the REVIEW ROUND alone implies; knows no branch facts
local b verdicts=""
for b in "${BOTS[@]}"; do
if requested "$b"; then echo state:bots-reviewing; return; fi
done
# Collect the WHOLE round before applying any precedence. Deciding inside
# the loop let BOTS order pick the winner: a MISSING returned immediately,
# so a STALE belonging to a later bot was never even read, and the mixed
# round (one approval staled by a push, another bot yet to review) came out
# needs-human — the #136 headline shape, with zero reviews bound to the head.
for b in "${BOTS[@]}"; do
verdicts="$verdicts $(bot_verdict "$b")"
done
case "$verdicts" in
# STALE = a verdict for an older head. Unlike MISSING, this outranks the
# human request: every approval it covers was invalidated by a push, so
# NOBODY has reviewed this tree. Handing that to the human is the #136 case
# where everything reads green — mergeable, CI passing, "waiting on the
# human" — over code no reviewer has seen. The agent owes a re-request.
# Checked before MISSING because "unfinished" must not swallow "and also
# stale": a round that is both is a push that outran the re-requests, not
# a maintainer deliberately claiming the PR early.
*STALE*) echo state:addressing; return ;;
esac
case "$verdicts" in
# No verdict at all from some bot, and nothing staled. An explicit human
# request still outranks an unfinished round — a maintainer pulling a PR
# to themselves early is a deliberate act, and the original precedence.
#
# Otherwise it is the AGENT's ball, not the bots'. The loop above already
# returned for every live bot request, so reaching here with a MISSING
# means somebody owes a verdict and nobody was asked for one — the round
# is not running. Calling that bots-reviewing was the lie that let a
# forgotten PR read "waiting on the reviewers" for the 48h it took the
# stale sweep to notice. blocker:unrequested says why.
*MISSING*)
if requested "$HUMAN"; then echo state:needs-human; return; fi
echo state:addressing; return ;;
esac
# an explicit human request outranks the remaining bot outcomes — it is the
# final gate, and a maintainer pulling a PR to themselves early counts too
if requested "$HUMAN"; then echo state:needs-human; return; fi
case "$verdicts" in
# FEEDBACK = a comment with no verdict → the agent owes the round-reply.
*BLOCK* | *FEEDBACK*) echo state:addressing; return ;;
esac
# the bots all approve — but if the human's standing word is
# changes-requested (and nobody re-requested them yet), the agent owes
# fixes, not the human a nag
if [ "$(bot_verdict "$HUMAN")" = BLOCK ]; then
echo state:addressing
else
echo state:needs-human
fi
}
# ---------------------------------------------------------------------------
# The sweep: fetch facts, decide, converge. One PR's failure never aborts the
# others — each PR reconciles in a subshell and a failure just logs.
# ---------------------------------------------------------------------------
bootstrap_labels() { # dispatch-only: ~20 upserts is too chatty for every cron tick
while IFS='|' read -r name color desc; do
[ -n "$name" ] || continue
run gh label create "$name" -R "$REPO" --color "$color" --description "$desc" --force
done <<'EOF'
state:building|FBCA04|PR is a draft — the coding agent is still building
state:bots-reviewing|1D76DB|Waiting on the bot reviewers to finish the round
state:addressing|D93F0B|All bots reviewed — coding agent owes the single reply + fixes
state:needs-human|8250DF|No blockers, all bots approve — waiting on the human reviewer
blocker:conflict|B60205|Does not merge — the branch conflicts and the agent owes a rebase
blocker:ci-red|B60205|A check is failing — the agent owes a fix (not a rebase)
blocker:unrequested|E99695|Somebody still owes a verdict and nobody was asked for one
merge-next|0E8A16|Head of the merge queue — merge this one next (set by hand/agent, cleared here)
stale|B60205|No activity for 48h — needs a poke (sweep-managed)
blocked|6A737D|Waiting on another PR or issue to land first
release|0E8A16|Release flow and version/packaging work
scope:bootstrap|C5DEF5|bootstrap — hardening a pristine server into a node
scope:users|C5DEF5|users-* — class model, apply/status, close-root
scope:runner|C5DEF5|runner-* — GitHub runner lifecycle
scope:coolify|C5DEF5|coolify-* — Coolify and backup install
scope:db|C5DEF5|db.sh — dump/restore
scope:installer|C5DEF5|install.sh — how rig lands on a machine
EOF
}
has_label() { grep -qxF "$1" <<<"$LABELS"; }
reconcile_pr() { # $1 = PR number; relies on the globals set from its fetch
local n="$1" desired remove s args last_activity age
desired="$(decide_state)"
# encode the runbook's last step for the no-judgment case: three formal
# head-current approvals → the human is asked, once. The guard asks whether
# a FRESH human review is needed for THIS head — never "has the human ever
# reviewed", which wedged the handoff after any earlier human comment.
# Idempotent (a live request suppresses it); race-free via the shared
# concurrency group in labels.yml. With a comment-only bot on the panel
# this path stays cold and the AUTHOR requests the human.
if [ "$desired" = state:needs-human ] && human_request_needed; then
run gh api "repos/$REPO/pulls/$n/requested_reviewers" -f "reviewers[]=$HUMAN" --silent
log "#$n: requested $HUMAN (round passed)"
fi
# ---- converge both axes ----
# state:* is exclusive (everything but $desired comes off); blocker:* is a
# set (each one on or off on its own); RETIRED always comes off. One edit
# call for all of it, so a PR never flickers through a half-applied board.
local want_blockers add=""
want_blockers="$(blockers)"
remove=""
for s in "${STATES[@]}"; do
if [ "$s" != "$desired" ] && has_label "$s"; then remove="$remove,$s"; fi
done
for s in "${RETIRED[@]}"; do
if has_label "$s"; then remove="$remove,$s"; fi
done
for s in "${BLOCKERS[@]}"; do
if grep -qxF "$s" <<<"$want_blockers"; then
has_label "$s" || add="$add,$s"
else
has_label "$s" && remove="$remove,$s"
fi
done
add="${add#,}"
remove="${remove#,}"
# Never NAME a label the repo does not have. `gh issue edit --add-label`
# rejects the WHOLE call on one unknown name — nothing is applied — so a
# single missing blocker would take the state convergence down with it, on
# exactly the PRs this change exists to fix, surfacing only as a log line.
# Batching state and blockers into one edit for anti-flicker is what widened
# that blast radius; filtering the add side is what closes it again.
# Removals need no filter: they are built from has_label, so the label
# provably exists. REPO_LABELS unreadable means no filtering rather than
# filtering everything out — a failed read must not silently strip the board.
local skip_edit=false
if [ -n "${REPO_LABELS:-}" ]; then
local kept="" missing="" want
for want in ${add//,/ }; do
if grep -qxF "$want" <<<"$REPO_LABELS"; then kept="$kept,$want"
else missing="$missing $want"; fi
done
add="${kept#,}"
# A missing STATE label skips only the EDIT — never the rest of this
# function. Everything below is independent of the state:* taxonomy, and
# returning here stranded it: `merge-next` kept claiming "merge this one
# next" on a PR the board had moved to the agent, and the stale sweep
# stopped running. That is the original false-invitation bug, reintroduced
# in the very fix meant to survive a cold-start repo — and a regression
# against the old behaviour, which failed the edit and fell through.
if ! grep -qxF "$desired" <<<"$REPO_LABELS"; then
log "#$n: WARNING: state label '$desired' does not exist — skipping the label edit; dispatch the workflow to bootstrap"
skip_edit=true
elif [ -n "$missing" ]; then
log "#$n: WARNING: missing label(s)$missing — state still converged; dispatch the workflow to bootstrap"
fi
fi
if [ "$skip_edit" = false ] && { ! has_label "$desired" || [ -n "$remove" ] || [ -n "$add" ]; }; then
args=(--add-label "$desired${add:+,$add}")
[ -n "$remove" ] && args+=(--remove-label "$remove")
if run gh issue edit "$n" -R "$REPO" "${args[@]}" >/dev/null; then
log "#$n: state -> $desired${add:+ +$add}${remove:+ (cleared $remove)}"
else
# a deleted label must not wedge the sweep — dispatch heals the taxonomy
log "#$n: WARNING: label edit failed (missing label? run the workflow manually to bootstrap)"
fi
fi
# ---- merge-next: cleared, never set ----------------------------------
# Queue order is INTENT — which PR should land first is a judgement about
# conflicts and dependencies that GitHub knows nothing about, so the
# reconciler must not guess it (LABELS.md's rule for `blocked`/`release`).
# What it CAN do is stop the label going stale the way needs-human did:
# the moment the PR is no longer the thing a human should merge next, the
# claim is removed. Setting it stays with whoever owns the queue.
if has_label merge-next && [ "$desired" != state:needs-human ]; then
run gh issue edit "$n" -R "$REPO" --remove-label merge-next >/dev/null
log "#$n: cleared merge-next (state is $desired, not mergeable-by-a-human)"
fi
# ---- stale: real activity only, and blocked is legitimately quiet ----
last_activity="$(
{
jq -r '.created_at' <<<"$PR_JSON"
jq -r '.[].submitted_at' <<<"$REVIEWS_JSON"
gh api --paginate "repos/$REPO/issues/$n/comments" --jq '.[].created_at'
gh api --paginate "repos/$REPO/pulls/$n/comments" --jq '.[].created_at'
gh api --paginate "repos/$REPO/pulls/$n/commits" --jq '.[].commit.committer.date'
} | sort | tail -n1
)"
age=$((NOW - $(date -d "$last_activity" +%s)))
if has_label blocked || [ "$age" -le "$STALE_AFTER" ]; then
if has_label stale; then
run gh issue edit "$n" -R "$REPO" --remove-label stale >/dev/null
log "#$n: unstale"
fi
elif ! has_label stale; then
run gh issue edit "$n" -R "$REPO" --add-label stale >/dev/null
log "#$n: stale ($((age / 3600))h quiet)"
fi
}
main() {
REPO="${REPO:?set REPO to owner/name}"
NOW="$(date +%s)"
if [ "${GITHUB_EVENT_NAME:-}" = workflow_dispatch ]; then
log "workflow_dispatch: bootstrapping the taxonomy"
bootstrap_labels
fi
# The repo's label set, read ONCE per sweep — reconcile_pr filters every
# add against it, because one unknown name fails the whole edit call.
REPO_LABELS="$(gh label list -R "$REPO" --limit 200 --json name --jq '.[].name' 2>/dev/null || echo "")"
[ -z "$REPO_LABELS" ] && log "WARNING: could not read the label set — applying labels unfiltered"
local n
for n in $(gh pr list -R "$REPO" --state open --limit 100 --json number --jq '.[].number'); do
(
PR_JSON="$(gh api "repos/$REPO/pulls/$n")"
DRAFT="$(jq -r '.draft' <<<"$PR_JSON")"
HEAD_SHA="$(jq -r '.head.sha' <<<"$PR_JSON")"
LABELS="$(jq -r '.labels[].name' <<<"$PR_JSON")"
REQUESTED="$(jq -r '.requested_reviewers[].login' <<<"$PR_JSON")"
# PENDING reviews are unsubmitted drafts in someone's browser — not a verdict
REVIEWS_JSON="$(gh api --paginate "repos/$REPO/pulls/$n/reviews" --jq '.[]' \
| jq -s '[.[] | select(.state != "PENDING")]')"
# mergeability + the check rollup, the two facts the state machine was
# blind to (#136). `gh pr view` rather than the REST PR object: the API's
# `mergeable` is a tri-state boolean that GitHub computes lazily, while
# this returns the same MERGEABLE/CONFLICTING/UNKNOWN string the UI shows.
# Failure to read them is NOT fatal and NOT treated as broken — an API
# hiccup must never flap every PR into needs-rebase, so both degrade to
# the "do not know" value that triggers nothing.
GH_VIEW="$(gh pr view "$n" -R "$REPO" --json mergeable,statusCheckRollup 2>/dev/null || echo '{}')"
MERGEABLE="$(jq -r '.mergeable // "UNKNOWN"' <<<"$GH_VIEW")"
CHECKS="$(checks_state <<<"$GH_VIEW")"
# Read failed: leave this PR exactly as it is. Recomputing on facts we
# did not read is how an API hiccup turns into a false "merge me" —
# and the next tick is 15 minutes away, not 15 hours.
if [ "$CHECKS" = UNREADABLE ]; then
log "#$n: could not read mergeability/checks — left alone this pass"
exit 0
fi
reconcile_pr "$n"
) || log "#$n: reconcile failed — continuing with the remaining PRs"
done
log "reconciled."
}
# sourced by test/labels-reconcile.sh for the fixture tests; executed in CI
if [ "${BASH_SOURCE[0]}" = "$0" ]; then
main "$@"
fi

23
.github/scripts/release-lib.sh vendored Normal file
View file

@ -0,0 +1,23 @@
#!/usr/bin/env bash
# Release plumbing shared by .github/workflows/release.yml and the test
# harness (test/release.sh) — pure functions, sourced, never executed on
# their own (repo precedent: labels-reconcile.sh's decide_state, the
# commands/lib/*.sh parsers).
# changelog_section <file> <version>
#
# Print the BODY of that version's CHANGELOG.md section: everything between
# its heading and the next '## ' heading (or EOF). A release heading is
# stamped '## <version> — <date>' and the Unreleased one is bare
# '## Unreleased'; the second field is the version either way, so both
# shapes match. The heading itself is not printed — the release title
# already names the version — and leading blank lines are dropped. Empty
# output means "no such section", which release.yml turns into a refusal: a
# tag with no changelog entry must not ship an empty release.
changelog_section() {
awk -v ver="$2" '
/^## / { if (found) exit; found = ($2 == ver); next }
found && !body && /^[[:space:]]*$/ { next }
found { body = 1; print }
' "$1"
}

137
.github/workflows/ci.yml vendored
View file

@ -9,11 +9,16 @@ jobs:
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with: with:
# changelog-monotonic and changelog-assembled compare HEAD against # fetch-depth: 0, for the changelog-monotonic step below and only
# the merge base; a # for it. That check is about a DIFF — which release headings the
# checkout that cannot resolve it is a hard failure in CI, not # merge base had — so it needs the base branch's history present,
# a skip (a guard that can quietly stop guarding is the failure # and the default depth-1 checkout has none of it. An explicit
# shape these checks exist to refuse). # `git fetch origin <base>` would be narrower, but it has to be
# right on both event types and on fork PRs, and getting it subtly
# wrong degrades to a SKIP (a guard that silently stops guarding —
# the exact failure this repo keeps refusing). Full history on a
# pure-bash tree costs a second; the STRICT flag below turns any
# remaining skip red rather than green.
fetch-depth: 0 fetch-depth: 0
- name: shellcheck - name: shellcheck
# -x follows the `source=SCRIPTDIR/...` directives into commands/lib/. # -x follows the `source=SCRIPTDIR/...` directives into commands/lib/.
@ -26,21 +31,7 @@ jobs:
# The file list is printed so under-coverage shows up in the log, and # The file list is printed so under-coverage shows up in the log, and
# the comm below turns under-coverage into a failure rather than a # the comm below turns under-coverage into a failure rather than a
# thing someone has to notice: every tracked `.sh` must be in the set. # thing someone has to notice: every tracked `.sh` must be in the set.
#
# Install shellcheck when missing (#144). rig's default Forgejo label
# maps ubuntu-latest to catthehacker's act-22.04 (slim), which does not
# ship shellcheck; GitHub-hosted ubuntu-latest does. The conditional
# keeps each forge from paying for the other.
#
# sudo: load-bearing on GitHub (job runs as `runner` with passwordless
# sudo) and a no-op on act-22.04 (jobs run as uid 0; the image has no
# `runner` account). Do not delete it as "dead weight" — that breaks
# the GitHub half the day that image stops preinstalling shellcheck.
run: | run: |
if ! command -v shellcheck >/dev/null 2>&1; then
sudo apt-get update
sudo apt-get install -y shellcheck
fi
shopt -s globstar dotglob shopt -s globstar dotglob
files=(bin/* **/*.sh) files=(bin/* **/*.sh)
printf 'shellcheck: %s\n' "${files[@]}" printf 'shellcheck: %s\n' "${files[@]}"
@ -52,65 +43,61 @@ jobs:
shellcheck -x "${files[@]}" shellcheck -x "${files[@]}"
- name: cli tests - name: cli tests
run: bash test/cli.sh run: bash test/cli.sh
- name: release tests — rig's own surfaces # test/labels-reconcile.sh existed here since #87 but ran nowhere: the
# label state machine gates every PR on this repo and its fixtures were
# green only when someone remembered to run them by hand. Same step, same
# place as heavy-duty/box.
- name: labels state-machine tests
run: bash test/labels-reconcile.sh
- name: release-flow tests
run: bash test/release.sh run: bash test/release.sh
# The drill harness's honesty: refusals, the skip/pass/fail classifier, # No SHIPPED release heading was deleted or DUPLICATED (#98). Its own step
# the idempotence capture-and-diff, the record emitter. Network-free and # rather than a line inside test/release.sh: that suite drives the arming
# root-free — the live four-leg run is a release's drill, not CI's. # rule against constructed VERSION + CHANGELOG.md trees that are not git
- name: drill harness tests — the instrument itself # repos, and this assert needs a git history — folding it in would make
run: bash test/drill.sh # those cases skip or lie. It is also a DIFFERENT invariant: arming is a
# The release guards, doctrine in heavy-duty/ceremony's README (#13's # fact about this tree, monotonicity is a fact about this tree versus its
# conversion). Each one's war story — why it exists, what it refuses — # merge base. STRICT=1 so a checkout that cannot reach the base ref fails
# lives with its implementation upstream; the six pins below and the # here instead of skipping quietly forever.
# two workflow callers must always name the same ceremony tag.
# #
# These six are ABSOLUTE and the two callers in release.yml/labels.yml # NOT pull-request-only, and that is the #98 fix at the workflow level.
# are BARE. That asymmetry is measured, not stylistic (#112) — do not # The two halves have different vacuity: DELETION is vacuous on a push to
# "tidy" either half into the other: # main (the merge base IS HEAD), but DUPLICATION is vacuous on no tree at
# a step `uses:` resolves a bare name through DEFAULT_ACTIONS_URL, # all, so gating the whole script on `pull_request` left a duplicate that
# which this forge points at code.forgejo.org, where ceremony does not # reached main by any other route unasserted forever.
# exist — bare 404s and every guard below fails;
# a reusable-workflow `uses:` never consults DEFAULT_ACTIONS_URL at
# all — it resolves against the runner's own instance, so bare already
# works there. And release.yml's caller is the line ceremony's own
# docs-sync greps for its pin, anchored to the bare spelling: make it
# absolute and the pin goes invisible and docs-sync exits 1.
# #
# changelog-armed: the version-keyed arming rule (rig#66; the # The `|| github.ref_name` fallback is load-bearing, not defensive. On a
# unconditional form rig#44 reverted — this is its correct return). # push event `github.base_ref` is EMPTY, so the argument would collapse to
- uses: https://forgejo.heavyduty.builders/heavy-duty/ceremony/actions/changelog-armed@0.3.0 # a bare `origin/`, which does not resolve — and STRICT=1 correctly
# changelog-monotonic: no shipped heading deleted or duplicated # promotes that to a hard failure, turning every push to main red. With
# (#98, box#122). Strict by default: an unresolvable base ref is red, # the fallback it resolves to the pushed branch, whose merge base with
# never a quiet skip — hence the fetch-depth: 0 above. # HEAD is HEAD or its parent: containment passes vacuously, exactly as the
- uses: https://forgejo.heavyduty.builders/heavy-duty/ceremony/actions/changelog-monotonic@0.3.0 # old `if` intended, while uniqueness now runs on every push.
# changelog-assembled: a release's stamped section must exactly match - name: no shipped changelog heading was deleted or duplicated
# the fragments it consumed. Vacuous on non-release PRs. env:
- uses: https://forgejo.heavyduty.builders/heavy-duty/ceremony/actions/changelog-assembled@0.3.0 CHANGELOG_MONOTONIC_STRICT: '1'
# drill-recorded: a release version carries drills/<version>.md run: bash .github/scripts/changelog-monotonic.sh "origin/${{ github.base_ref || github.ref_name }}"
# (rig's drill meaning: drills/README.md). Vacuous on -dev trees. # The release this tree would ship has a recorded real-hardware drill
- uses: https://forgejo.heavyduty.builders/heavy-duty/ceremony/actions/drill-recorded@0.3.0 # (drills/<version>.md). CONTRIBUTING ("Releasing") has always required one and
# runner-isolated: PR-triggered workflows never execute unreviewed # nothing enforced it, so no release in this family has ever carried one
# branch code on a self-hosted runner. # — the drill was the single ceremony step checked by a reviewer
- uses: https://forgejo.heavyduty.builders/heavy-duty/ceremony/actions/runner-isolated@0.3.0 # remembering rather than by a script.
# docs-sync: the .ceremony/ doctrine mirror is byte-identical to the #
# pin read from release.yml (ceremony#19) — a hand edit or a # Deliberately NOT trigger-scoped, and for the opposite reason to the
# half-done pin bump goes red here. # step above. That one needs a base ref, so its argument has to be right
- uses: https://forgejo.heavyduty.builders/heavy-duty/ceremony/actions/docs-sync@0.3.0 # on both event types; this one reads two files in the checkout and is
# VACUOUS BY CONSTRUCTION on a `-dev` VERSION, which every ordinary PR
# The install LIFECYCLE against a tree install.sh itself produced — the four # and every push to main carries. It has something to say on exactly one
# beats box and cast already run in CI (#106): install from this checkout, # tree — the `release: X.Y.Z` ceremony PR — so an `if:` could only add a
# assert what landed, a converging re-run proven by an EMPTY DIFF (never an # way for that one tree to slip past.
# exit code), uninstall --all ending in the absence assert (`! -e` AND #
# `! -L` — only the second sees a dangling symlink). Separate from `check` # PER-REPO on purpose: rig reads rig's own record, never box's repo. A
# for the same reason db-integration is: fast feedback first. The runner's # cross-repo lookup fails on a token, a network blip or a fork checkout,
# real $HOME is the point — no throwaway roots here; the suite refuses to # and every one of those lands on "could not read" — which degrades to
# run where a rig is already installed, so it cannot eat a real install. # green on precisely the tree that ships (the UNREADABLE-vs-NONE shape
install: # #90 fixed).
runs-on: ubuntu-latest - name: a release version has a recorded drill
steps: run: bash .github/scripts/drill-recorded.sh
- uses: actions/checkout@v4
- name: the install lifecycle — four beats against a real tree
run: bash test/install-lifecycle.sh
# Kept SEPARATE from `check` on purpose: this job pulls a Postgres image and # Kept SEPARATE from `check` on purpose: this job pulls a Postgres image and
# stands up throwaway containers, and a slow image pull must never delay the # stands up throwaway containers, and a slow image pull must never delay the

84
.github/workflows/labels.yml vendored
View file

@ -1,22 +1,78 @@
name: labels name: labels
# The automation LABELS.md promises, now implemented upstream # The automation LABELS.md promises. Two halves:
# (heavy-duty/ceremony#13): scope labeling and the state reconciler live in # scope — path-derived scope:* labels on PRs (actions/labeler)
# the reusable workflow this caller pins. Rig keeps the triggers and # reconcile — the state:* machine + the stale sweep (.github/scripts/labels-reconcile.sh)
# permissions (a called workflow cannot define them), its path map in #
# .github/labeler.yml, and its panel + scope taxonomy in .github/labels.conf. # pull_request_target, not pull_request: every PR here arrives from a fork,
# where pull_request (and pull_request_review) run with a READ-ONLY token and
# cannot label anything. _target is safe in this workflow because no PR code
# is ever checked out or executed — labeler reads changed paths via the API,
# and reconcile checks out the BASE branch only. Keep it that way.
#
# There is no pull_request_review_target, so a review landing cannot wake this
# workflow directly — and the */15 cron is advisory: GitHub deprioritises short
# intervals hard enough that a quiet repo goes hours between ticks. So the
# handoff wakes the sweep itself: the author sets state:needs-human when handing
# the PR to the maintainer (CONTRIBUTING step 6), and `labeled` fires this
# workflow, which confirms or corrects that optimistic write within seconds. The
# cron stays as the last resort, for the round an agent forgets to hand off.
#
# This cannot loop: the reconciler's own label writes use GITHUB_TOKEN, and
# GitHub does not create workflow runs from GITHUB_TOKEN-triggered events. Agent
# writes use a PAT and therefore do trigger — exactly the asymmetry wanted.
on: on:
schedule: [{cron: "*/15 * * * *"}] # advisory; the handoff label is the real wake schedule:
workflow_dispatch: # bootstraps missing labels on a fresh repo - cron: "*/15 * * * *"
workflow_dispatch: # also bootstraps missing labels — run once on a fresh repo
pull_request_target: pull_request_target:
types: [opened, reopened, ready_for_review, converted_to_draft, synchronize, labeled, unlabeled, review_requested, review_request_removed] types:
issues: [
types: [opened, edited, assigned, unassigned, labeled, unlabeled, closed, reopened] opened,
reopened,
ready_for_review,
converted_to_draft,
synchronize,
labeled,
unlabeled,
]
permissions: permissions:
contents: read contents: read
checks: read
statuses: read
issues: write issues: write
pull-requests: write pull-requests: write
jobs: jobs:
labels: scope:
uses: heavy-duty/ceremony/.github/workflows/labels.yml@0.3.0 # Not on labeled/unlabeled: those events change no paths, so labeler has
# nothing new to derive — and label churn is precisely what they are.
if: >-
github.event_name == 'pull_request_target' &&
github.event.action != 'labeled' &&
github.event.action != 'unlabeled'
runs-on: ubuntu-latest
concurrency:
group: labels-scope-${{ github.event.pull_request.number }}
cancel-in-progress: true
steps:
- uses: actions/labeler@v5
with:
# additive only — a hand-applied scope must survive the machine
sync-labels: false
reconcile:
runs-on: ubuntu-latest
# ONE shared group: every reconcile sweeps every open PR, so cron and
# PR-event runs must serialize or two sweeps race the same PR's labels
# and both pass the request-the-human-once guard. GitHub keeps at most
# one queued run per group (older queued runs are superseded), which
# coalesces bursts instead of piling them up.
concurrency:
group: labels-reconcile
cancel-in-progress: false
steps:
- uses: actions/checkout@v4 # base branch only — never the PR's code
- name: reconcile state + stale
env:
GH_TOKEN: ${{ github.token }}
REPO: ${{ github.repository }}
run: bash .github/scripts/labels-reconcile.sh

246
.github/workflows/release.yml vendored
View file

@ -1,22 +1,238 @@
name: release name: release
# The ceremony moved upstream (heavy-duty/ceremony — the doctrine is its # Two ways in, one release out (#47; box#96's design — the merge path — on
# README; ceremony#13 is this conversion). Both doors — merge and tag push — # top of #32/box#83's tag flow, kept verbatim as the fallback):
# live in the reusable workflow this caller pins; rig keeps only what a #
# called workflow cannot define: triggers and permissions. # - MERGE (the paved road): a release is a PR — `release: X.Y.Z`, carrying
# Triggers and permissions MUST live here (a called workflow cannot define them): # the `release` label, bumping VERSION and stamping CHANGELOG.md's
# Unreleased section — and MERGING it is the ship decision. The
# release-on-merge job asserts its way to certainty, then tags the merge
# commit and publishes, same job. No separate, silent-when-forgotten
# tagging step: a forgotten tag produces no red X, a failed run on main
# does — of two unreliabilities, pick the loud one.
# - TAG PUSH (the manual fallback and backfill): tag the merge commit bare
# `X.Y.Z` (no `v` prefix — box's tag scheme) and push; the release job
# below turns it into the GitHub release.
#
# Either way the body is the changelog section — the curated prose, never
# the auto-generated PR list — and no assets are uploaded on purpose: for a
# pure-bash tree, GitHub's source tarball for the tag IS the package
# (install.sh downloads archive/refs/tags/<tag>).
on: on:
# ONE push key, both filters — YAML maps are last-key-wins; a second sibling # ONE push key, both filters — YAML maps are last-key-wins, so a second
# `push:` silently replaces the first and kills a door (rig's review catch). # sibling `push:` would silently REPLACE the first and kill a door
# (grok's round-2 catch: the tag fallback had stopped triggering).
push: push:
tags: ["**"] # every tag — a wrong tag must FAIL the assert loudly, # Every tag, not a shape filter: a tag that mismatches VERSION must fail
# never be skipped by a shape filter that didn't match # LOUDLY below, not be silently skipped by a pattern that didn't match.
tags: ['**']
# The merge-is-the-release path (#47) rides pushes to MAIN, not
# pull_request events: a pull_request run from a public FORK gets a
# READ-ONLY GITHUB_TOKEN — `permissions:` cannot raise that ceiling —
# and every ceremony PR this org merges is cross-repo from the bot
# fork; the tag create would 403 after green asserts. A push to main
# is an in-repo event with the full write token, whoever authored the
# PR. The jobs split on the pushed ref.
branches: [main] branches: [main]
permissions: permissions:
contents: write # tag ref create + release create + the bump push contents: write # the tag ref, the release publish, the bump push
pull-requests: write # decide's label read; the bump-fallback `gh pr create` # Two consumers (a declared permissions: block zeroes every unspecified
issues: write # --label on that fallback PR rides the issues API # scope): the decide step's label read (commits/<sha>/pulls) and the bump
# fallback's `gh pr create --label`.
pull-requests: write
# ...and the --label on that fallback PR rides the ISSUES API (labels.yml
# grants the same pair for the same reason).
issues: write
jobs: jobs:
release: release:
uses: heavy-duty/ceremony/.github/workflows/release.yml@0.3.0 # The tag-push path — a pushed TAG ref. The merge path (a pushed main
with: # head) is release-on-merge below; the two doors split on the ref.
version-source: file if: startsWith(github.ref, 'refs/tags/')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
# The tag names a tree; the tree names its own version. When they
# disagree, creating a release would put a version label on a tree
# that is not that version — exactly the lie the release flow exists
# to end — so: fail, create nothing.
- name: assert the tag matches the tree's VERSION
run: |
ver="$(cat VERSION)"
if [ "$GITHUB_REF_NAME" != "$ver" ]; then
echo "tag '$GITHUB_REF_NAME' != VERSION '$ver' — refusing to create a release for a tree that says it is something else" >&2
exit 1
fi
- name: create the release from the changelog section
env:
GH_TOKEN: ${{ github.token }}
run: |
. .github/scripts/release-lib.sh
notes="$(changelog_section CHANGELOG.md "$GITHUB_REF_NAME")"
if [ -z "$notes" ]; then
echo "CHANGELOG.md has no '## $GITHUB_REF_NAME' section — stamp the Unreleased section in the release PR before tagging" >&2
exit 1
fi
gh release create "$GITHUB_REF_NAME" --verify-tag \
--title "$GITHUB_REF_NAME" --notes "$notes"
# The merge path (#47; box#96): the `release` label is the intent, the
# VERSION transition is the interlock. Four asserts in order, each
# fail-loud and creating NOTHING, then tag + publish in this same job.
# Same-job is load-bearing: the tag is created with GITHUB_TOKEN via the
# API, and GITHUB_TOKEN-created refs do not fire `on: push: tags`
# workflows — so the publish MUST live here (nothing else would run), and
# the fallback job above CANNOT double-publish off our tag. A manually
# pushed tag racing this run is caught by the nothing-exists assert.
# NOTE: test/release.sh pins this block by awk-ing from
# 'release-on-merge:' to EOF — keep it the last job.
release-on-merge:
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
env:
MERGE_SHA: ${{ github.sha }}
BASE_SHA: ${{ github.event.before }}
steps:
- uses: actions/checkout@v4
with:
# The pushed head is what ships. Full history so the before-side
# VERSION is readable for the interlock below.
ref: ${{ github.sha }}
fetch-depth: 0
# The decide step — asserts 1+2 fused, because the `release` label
# carries TWO legitimate meanings (LABELS.md: "release flow and
# version/packaging work"): the ceremony PR that ships a version, and
# ordinary work ON the release machinery — this very PR included.
# The version tells them apart. A `-dev` VERSION left UNTOUCHED by the
# PR is release-flow work: a green no-op, not a red run on main every
# time the flow itself is improved. Everything in between is a
# half-ceremony and dies loudly:
# -dev, unchanged → work under the label: NOTICE + green no-op
# -dev, changed → still a dev tree, so still work — the
# post-release bump PR above all (bare -> -dev
# after every release): green NOTICE no-op
# bare, unchanged,
# already released → work merged in the post-release window
# (ceremony landed, the -dev bump has not):
# NOTICE + green no-op
# bare, unchanged,
# never released → the label says ship, the tree names an
# unshipped version this PR did not mint:
# genuinely ambiguous, refuse
# bare, changed → the ceremony: proceed
- name: 'decide: ceremony, or release-flow work under the label?'
id: decide
env:
GH_TOKEN: ${{ github.token }}
run: |
ver="$(cat VERSION)"
# event.before is all-zeros on a branch-create push; the merge
# commit's first parent is main the instant before, either way.
if ! git cat-file -e "$BASE_SHA" 2>/dev/null; then BASE_SHA="$(git rev-parse "$MERGE_SHA^1")"; fi
base_ver="$(git show "$BASE_SHA:VERSION")"
case "$ver" in
*-dev)
if [ "$base_ver" = "$ver" ]; then
echo "NOTICE: VERSION '$ver' is -dev and unchanged by this PR — release-flow work under the release label, not a ceremony. Nothing to publish."
echo "ceremony=no" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "NOTICE: VERSION changed ('$base_ver' -> '$ver') and still ends -dev — a dev tree is by definition not a release. This is work (the post-release bump, a renumber); nothing to publish."
echo "ceremony=no" >> "$GITHUB_OUTPUT"
exit 0 ;;
esac
if [ "$base_ver" = "$ver" ]; then
if gh release view "$ver" -R "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
echo "NOTICE: VERSION '$ver' is already released and unchanged by this PR — release-flow work merged in the post-release window (before the -dev bump). Nothing to publish."
echo "ceremony=no" >> "$GITHUB_OUTPUT"
exit 0
fi
echo "VERSION '$ver' is bare, unchanged by this PR, and never released — the label says ship but this PR did not mint the version. Refusing to guess." >&2
exit 1
fi
# The version transitioned — now the LABEL, the operator's declared
# intent, read via the API because a push event carries no PR
# payload (and the PR itself lives on a fork — see the trigger
# comment). No release-labeled PR behind this commit = a version
# transition nobody declared: refuse.
if ! gh api "repos/$GITHUB_REPOSITORY/commits/$MERGE_SHA/pulls" \
-q '[.[] | select(.merged_at != null) | .labels[].name] | index("release") != null' | grep -qx true; then
echo "VERSION transitioned ('$base_ver' -> '$ver') but no merged, release-labeled PR is behind this commit — a release is a labeled ceremony PR (#47), not a bare push. Refusing." >&2
exit 1
fi
echo "ceremony=yes" >> "$GITHUB_OUTPUT"
# Assert 3 — the changelog names exactly this version, and the one
# extractor (shared with the tag job and test/release.sh) gets a
# non-empty body out of it. The notes are kept for the publish.
- name: assert the changelog section for this version extracts
if: steps.decide.outputs.ceremony == 'yes'
run: |
. .github/scripts/release-lib.sh
ver="$(cat VERSION)"
changelog_section CHANGELOG.md "$ver" > "$RUNNER_TEMP/notes.md"
if [ ! -s "$RUNNER_TEMP/notes.md" ]; then
echo "CHANGELOG.md has no '## $ver' section at the merge commit — the ceremony PR must stamp it; refusing to publish an empty release" >&2
exit 1
fi
cat "$RUNNER_TEMP/notes.md"
# Assert 4 — nothing exists yet, tag or release: a re-run of this job
# (or a manual tag that beat it) must refuse, not clobber.
- name: assert no tag and no release exist yet (idempotent re-runs)
if: steps.decide.outputs.ceremony == 'yes'
env:
GH_TOKEN: ${{ github.token }}
run: |
ver="$(cat VERSION)"
if git ls-remote --exit-code origin "refs/tags/$ver" >/dev/null 2>&1; then
echo "tag '$ver' already exists — this release already happened (or is mid-flight on the manual path); refusing to re-release" >&2
exit 1
fi
if gh release view "$ver" -R "$GITHUB_REPOSITORY" >/dev/null 2>&1; then
echo "release '$ver' already exists — refusing to re-release" >&2
exit 1
fi
# Act — tag the merge commit via the API, then publish with the notes
# assert 3 extracted. (GITHUB_TOKEN-created tag: no recursive
# workflow runs — see the job comment.)
- name: tag the merge commit and publish the release
if: steps.decide.outputs.ceremony == 'yes'
env:
GH_TOKEN: ${{ github.token }}
run: |
ver="$(cat VERSION)"
gh api -X POST "repos/$GITHUB_REPOSITORY/git/refs" \
-f ref="refs/tags/$ver" -f sha="$MERGE_SHA"
gh release create "$ver" --verify-tag \
--title "$ver" --notes-file "$RUNNER_TEMP/notes.md" \
-R "$GITHUB_REPOSITORY"
# The post-release bump, folded into the release act (#47 followup —
# operator decision: a mechanical one-liner deserves no PR of its
# own). X.Y.(Z+1)-dev is arithmetic, not judgment: derived, committed
# straight to main with this job's token. A GITHUB_TOKEN push fires
# no workflows (anti-recursion), so the bump triggers neither this
# door nor a red run; and should branch protection ever refuse the
# direct push, the step opens the bump PR itself and says so, loudly,
# instead of leaving main armed to impersonate the release.
- name: bump main to the next -dev — the release re-arms main itself
if: steps.decide.outputs.ceremony == 'yes'
env:
GH_TOKEN: ${{ github.token }}
run: |
ver="$(cat VERSION)"
next="$(printf '%s' "$ver" | awk -F. '{ printf "%s.%s.%s-dev", $1, $2, $3 + 1 }')"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch origin main
git checkout -B main origin/main
printf '%s\n' "$next" > VERSION
git add VERSION
git commit -m "chore: bump main to $next — a dev install must not impersonate $ver"
if ! git push origin main; then
echo "direct push refused (branch protection?) — opening the bump PR instead" >&2
git checkout -b "chore/bump-$next"
git push origin "chore/bump-$next"
gh pr create -R "$GITHUB_REPOSITORY" --head "chore/bump-$next" \
--title "chore: bump main to $next" \
--body "The post-release re-arm, opened by release.yml because the direct push was refused. One file, one line." \
--label release
fi

7
AGENTS.md
View file

@ -1,7 +0,0 @@
# AGENTS.md — start at .ceremony/
This repository is governed by
[heavy-duty/ceremony](https://github.com/heavy-duty/ceremony). Read
`.ceremony/AGENTS.md` first — it routes you to your role file, vendored
beside it. Repo specifics (the review panel roster, the scope labels, what
a drill means here, code conventions) live in CONTRIBUTING.md.

29
CHANGELOG.md
View file

@ -4,34 +4,7 @@ History before 0.1.0 lives in git — rig grew its version surface (`VERSION`,
`rig --version`, the side-by-side `versions/<v>` install layout; #35/#36) `rig --version`, the side-by-side `versions/<v>` install layout; #35/#36)
on the way to cutting its first release, and this file starts there. on the way to cutting its first release, and this file starts there.
## 0.3.1 — 2026-07-24 ## Unreleased
### Added
- GitHub entry templates route humans to Discussions and prefill triage work orders and pull requests (#123)
- Platform, drill, docs and labels changes receive dedicated scope labels (#119)
- The `changelog-armed` guard returns, version-keyed (#112, ceremony#13)
- The `.ceremony/` doctrine mirror, verified by `docs-sync` on every PR (#112, ceremony#19)
- `rig template-lint` validates role definitions; rig-templates CI runs it on every PR (#110)
- Drill records cite the rig-templates SHA the converge read (#110)
- `kimi-box` joins the box tenant roles — the Kimi CLI agent guest (#109)
- CI drills the install lifecycle against a real tree — install from the checkout, converge to an empty diff, uninstall to proven absence (#106)
- `drill/drill.sh` — the drill has an instrument: pinned-ref assertion, a mechanical idempotence diff, and a `drills/<version>.md` record emitter (#105)
- `rig platform` prints a stable machine `ID`, derived from `/etc/machine-id`, never the raw value (#95)
- `rig bootstrap --undo` removes only a tailnet join rig can prove it made (#63)
### Changed
- Changelog entries land in per-issue fragments assembled by the release PR (#136)
- Release and labels machinery is consumed from heavy-duty/ceremony@0.1.0 by reference — the workflows shrink to caller stubs, the guard scripts and their tests move upstream (#112, ceremony#13)
- Agent-tenant definitions live in heavy-duty/rig-templates, pinned in-tree and overridable per mint (`RIG_TEMPLATES_DIR`/`_REF`/`_REPO`); the in-tree case arms are gone, `staging-box` stays (#110)
- `bootstrap --host yes` installs a pinned box release instead of `main` (#103)
### Fixed
- The quick-start fence names its channel and carries the release command beside it (#149)
- The drill's docs no longer claim both installers default to `main` — box installs the `BOX_RELEASE` pin, rig the latest release, and its `--box-ref` example is now a tag (#133)
- `kimi-bot-andresmgsl` is on the review panel — the roster predated it joining the bench (#120)
## 0.3.0 — 2026-07-21 ## 0.3.0 — 2026-07-21

229
CONTRIBUTING.md
View file

@ -1,41 +1,66 @@
# Contributing # Contributing
This repo is governed by How change lands in this repo. The short version: PRs are born as drafts,
[heavy-duty/ceremony](https://github.com/heavy-duty/ceremony). **Agents: three reviewer bots take the first rounds, a human takes the last word — and
read [`.ceremony/AGENTS.md`](.ceremony/AGENTS.md) first** — it routes you to labels tell you where everything is without opening anything.
your role file (builder, reviewer, triage), vendored beside it,
byte-identical to ceremony at the pin named in
[`.github/workflows/release.yml`](.github/workflows/release.yml) and
guarded by the `docs-sync` step in CI. The review-round doctrine — drafts,
whole-round replies, verdicts, the handoff — lives there and in
[`.ceremony/LABELS.md`](.ceremony/LABELS.md); this file keeps only what is
genuinely rig's.
## The PR loop, rig specifics ## The PR loop
1. **Fork and branch.** Contributors work from forks; upstream branches are 1. **Fork and branch.** Contributors work from forks; upstream branches are
for maintainers. Title the PR conventionally (`feat:`, `fix:`, `docs:`). for maintainers. Title the PR conventionally (`feat:`, `fix:`, `docs:`).
2. **The review panel** (`.github/labels.conf`'s `panel=` line): 2. **Open as a draft** while you build. Drafts are invisible to the reviewer
`cluade-reviewer-andresmgsl`, `codex-reviewer-andresmgsl`, bots on purpose.
`grok-reviewer-andresmgsl`, `kimi-reviewer-andresmgsl` — 3. **When it's ready**: mark ready-for-review and request all three bots —
the required verdicts for a PR are the panel minus its author. The `claude-bot-andresmgsl`, `codex-bot-andresmgsl`, `grok-bot-andresmgsl`.
maintainer (`danmt`) takes the last word and merges. They poll roughly every 15 minutes.
3. **Checks must be green**: `shellcheck`, `bash test/cli.sh` and 4. **Rounds are answered whole.** Wait until all three have reviewed, then
answer the entire round in a **single reply**, push the fixes, and
re-request the bots that didn't approve. Prefer verification over
argument: a test settles what a comment thread can't.
5. **Reviews end in a verdict.** A reviewer — bot or human — either
**approves** or **requests changes**, never a bare comment. A
comment-only review is a non-verdict: it doesn't say whether the round
passed, and the state machine (and anyone scanning the board) has to
guess. The verdict carries *blockingness only*, the body carries the
feedback: non-blocking nits ride an **approval** and the author addresses
them at their discretion; anything blocking — including a question that
gates the verdict — is **request changes**, saying what unblocks it. The
reconciler treats a comment-only review as not-approved, so commenting
without a verdict only stalls the PR. The machine never reads review
bodies: when a comment-only reviewer's line is really an agreement, that
judgment belongs to the **author** — escalate by requesting the
maintainer's review (step 6), and the reconciler flips the label on that
request, because an explicit request is a fact it can trust.
6. **When the round passes, the author hands the PR to the maintainer** in
three acts, in this order: post the tagged round summary, request the
maintainer's review, then set `state:needs-human` yourself — removing the
state label it replaces. The review request is what *earns* the label,
provided the PR carries **no `blocker:*` label**. A blocker means the work
is still yours whatever the round said, so on a conflicted or red PR
neither the request nor your own label write will stick — the sweep takes
it straight back off. With three formal head-current approvals the labels
workflow requests the maintainer automatically; when part of the panel is
comment-only, reading their agreement is the author's judgment, so the
author makes the request.
Writing the label by hand is an **optimistic write, not a transfer of
ownership**. The machine stays the authority — but because the workflow
wakes on `labeled`, the author's own write fires the sweep that validates
it, and a handoff that had not earned the label is corrected seconds later.
Forgetting the write is not a failure either; it only means the label waits
for the cron, which is the lag this replaced.
7. **Checks must be green**: `shellcheck`, `bash test/cli.sh` and
`bash test/release.sh` locally mirror what CI runs; the db dump/restore `bash test/release.sh` locally mirror what CI runs; the db dump/restore
round-trip (`test/db-integration.sh`) executes in CI where Docker is round-trip (`test/db-integration.sh`) executes in CI where Docker is
present. The release guards (`changelog-armed`, `changelog-monotonic`, present.
`changelog-assembled`, `drill-recorded`, `runner-isolated`, `docs-sync`) 8. **Feature PRs land their changelog entry as part of the PR** (box's
run as ceremony's pinned actions. convention): add it under `CHANGELOG.md`'s `## Unreleased` heading —
4. **Feature PRs land their changelog entry as part of the PR**: write that section becomes the release notes verbatim when a release is cut.
`changelog.d/<issue>.md` — the release PR assembles those fragments into
the release notes verbatim.
## Changelog entries ## Changelog entries
Every PR that changes behaviour writes one `changelog.d/<issue>.md` fragment. Every PR that changes behaviour adds one line to `## Unreleased`. One line is
The fragment keeps the relevant `### Added` / `### Changed` / `### Fixed` the whole rule — if it wraps more than twice in your editor, cut it down.
heading above its entry. One line is the whole rule — if it wraps more than
twice in your editor, cut it down.
- **Say what changed, and stop.** Why it was wrong, how it was found, what it - **Say what changed, and stop.** Why it was wrong, how it was found, what it
cost, what it implies — that belongs in the PR body and the commit message, cost, what it implies — that belongs in the PR body and the commit message,
@ -66,42 +91,134 @@ Not an entry — that is a PR body:
## Releasing ## Releasing
A release is a PR, and merging it is the release. The ceremony — the two A release is a PR, and merging it is the release (#47; box#96's design, on
doors, the decide table, the stamps, the post-release re-arm — is top of #32/box#83's tag flow). It takes the ordinary PR loop above, with one
heavy-duty/ceremony's machinery, consumed by reference: extra gate before the handoff:
[its README](https://github.com/heavy-duty/ceremony/blob/main/README.md)
is the doctrine, `.github/workflows/release.yml` here is the ≤20-line
caller pinning it, and the guards run in `ci.yml` from the same pin.
Bare `X.Y.Z` tags, no `v`; the tag's source tarball is the package
`install.sh` downloads — rig ships no other artifact. Each release deliberately
bumps and drills the `BOX_RELEASE` pin in `commands/bootstrap.sh`; it must never
float to a moving ref.
What stays rig's is the **drill** — the real-hardware gate before the **draft → ready → bot round → drill → `state:needs-human` → maintainer merge
handoff of a release PR, run by `drill/drill.sh` (#105): `rig bootstrap` (which IS the release).**
converging the machine to its role twice with the second run diffed empty,
`test/db-integration.sh`, the runner lifecycle against a fork, a coolify The **drill** is a real-hardware run — tenant guests minted and converged via
install. Rig's drill asserts **convergence** (a machine reaches its role, box, `test/db-integration.sh`, the GitHub runner lifecycle against a fork, a
idempotently), it runs `--host yes` with `BOX_REF=release/<box-version>` so coolify install — recorded in **one file per version**:
it exercises the box that will actually ship, and drills that share a
substrate share **one run ID** so the per-repo records can be joined after ```
the fact. The full meaning — the fixed candidate-ref pinning that dissolves drills/<version>.md
the box↔rig recursion, the per-version record files, the waiver rule — is ```
[`drills/README.md`](drills/README.md); the `drill-recorded` guard enforces
the record on every release tree. named for the version exactly as `VERSION` carries it. See
[`drills/README.md`](drills/README.md) for what a record should contain.
`.github/scripts/drill-recorded.sh` enforces it on every release: a bare
`VERSION` with no non-empty `drills/<version>.md` turns CI red, naming the
version. It is **not a thing a reviewer has to remember** — that is how every
release in this family shipped undrilled until a bot finally blocked on one. On
a `-dev` tree it asserts nothing, so it is invisible to ordinary PRs. rig reads
rig's own record and never box's repo: a cross-repo lookup fails on a token,
a fork checkout or a network blip, and all of those degrade to "pass" —
the UNREADABLE-vs-NONE shape #90 fixed.
One file per version is what keeps the guard small. Records used to share a
single log, which forced a heading grammar, an optional-date tail, a
whole-version comparison and a non-blank-body rule just to read them back — and
both sibling repos shipped a defect out of that complexity in review. Now
`0.3.0.md` and `0.3.0-rc1.md` are simply different files.
**The three repos' drills are INDEPENDENT.** Run them in any order, on any
schedule, in separate sittings. What makes that safe is that every drill **pins
the same fixed set of candidate refs**: rig's drill runs `--host yes` with
`BOX_REF=release/<box-version>`, so it exercises the box that will actually
ship; box's drill mints with `RIG_REF=release/<rig-version>`, so it exercises
the rig that will actually ship. Both measure the same pair.
That — not sequencing — is what dissolves the box↔rig recursion. box and rig
are mutually recursive (`rig bootstrap … --host yes` installs box and runs
box's `setup-host`; box's `box new` seeds converge back through rig's installer
at `@RIG_REPO@/@RIG_REF@`), but the refs are static identifiers that exist as
soon as the release branches do, long before any drill runs, so a cycle at
runtime becomes independent tests against one fixed pair. Within a single drill
you naturally bring the substrate up before probing it — a host before a guest
— but that is how you run a drill, not an ordering rule between repos.
Each repo drills in a **different way** and asserts a different thing: rig
asserts **convergence** (a machine reaches its role, idempotently), box asserts
the **isolation contract** (the VM trust boundary), cast asserts **promotion**
(A→B reproduces, the diff is idempotent). Three different exercises sharing a
substrate, not three phases of one script — which is exactly why the records
are per-repo.
It drills **candidate refs, not released artifacts.** `RIG_REPO`/`RIG_REF` are
mint-time environment variables (default `heavy-duty/rig@main`), so a run pins
the exact commits under test. That is what dissolves the chicken-and-egg: no
repo has to be released before another can be drilled.
**Drilling the candidate IS drilling the release.** A release PR's diff is
`VERSION` + `CHANGELOG.md` and nothing else — no executable difference exists
between the tree that was drilled and the tree that ships.
Drills that share a substrate share **one run ID**. Each repo records *its own*
legs in its own `drills/<version>.md`, citing that run ID and the other two
repos' commit SHAs, so the records can be joined after the fact by anyone
reading them. The guard still reads only this repo's file — there is no
cross-repo lookup anywhere in the gate. Releases do **not** have to be
published in a fixed order. If a defect shows up only in the combination:
patch, re-drill, re-record. The three releases converge on a set that holds
together; they are not required to be right in one pass.
A **maintainer waiver** is possible — a doc-only release, a hardware outage —
but it must be **recorded in `drills/<version>.md` for that version**, saying
who waived it and why. The guard asks for a *record*, not a passing result,
precisely so that skipping is a deliberate, reviewable commit instead of a
silence. Deleting the check is not the move.
The mechanics:
1. A small PR — `release: X.Y.Z`, carrying the `release` label — bumps
`VERSION` from `X.Y.Z-dev` and stamps `CHANGELOG.md`'s Unreleased
section as `## X.Y.Z — YYYY-MM-DD`. **Then re-arm the file in the same
PR**: add a fresh, empty `## Unreleased` immediately above the section
you just stamped (#66). Stamping alone *disarms* main — a PR authored
before the release and merged after it wrote its entry under
`## Unreleased`, and with that heading gone git files the entry under
whatever now occupies the position, which is the release that already
shipped. It lands cleanly, with no conflict and nothing for the author
to notice, so the empty section is the only thing standing between a
late merge and a changelog that misattributes a shipped release. No
workflow does this for you: `release.yml` re-arms `VERSION`, never the
changelog. `test/release.sh` enforces the pairing — whenever `VERSION`
ends in `-dev` the top section must be `## Unreleased`. CI green on it,
same loop as any PR.
2. Merge it — that IS the ship decision. `release.yml`'s
`release-on-merge` job asserts, in order, fail-loud, creating nothing:
the merged tree's `VERSION` is non-`-dev`; this PR is the one that
changed it (a mislabeled ordinary PR fails here); the changelog section
for that version extracts non-empty; no tag or release exists yet.
Then, same job, it tags the merge commit bare `X.Y.Z` (no `v` prefix —
box's tag scheme) and publishes the GitHub release with that section as
the body. No assets — the source tarball for the tag is the package
`install.sh` downloads.
3. The release re-arms main itself: the same workflow run bumps `VERSION`
to `X.Y.(Z+1)-dev` and pushes the commit straight to main — no
follow-up PR (it opens one only if branch protection refuses the
direct push, loudly). A dev install therefore never impersonates the
release in the `versions/<v>` layout. On the *manual* tag path the
bump stays yours: open the one-line PR after publishing.
Manual fallback (and backfill): if the merge-path run fails, fix what it
named, then tag the merge commit `X.Y.Z` by hand and push the tag — the
original tag-push job still turns any correct tag into the release, and
the merge path's nothing-exists-yet assert keeps the two from
double-publishing.
## Labels — who sets what ## Labels — who sets what
The taxonomy and state machine are The full taxonomy lives in [LABELS.md](LABELS.md). What matters day to day is
[`.ceremony/LABELS.md`](.ceremony/LABELS.md); rig's `scope:*` rows live in who sets each kind — most of it is machinery, and hand-moving a
`.github/labels.conf` (reconciled by the labels caller) and their path map machine-owned label just gets corrected on the next pass:
in `.github/labeler.yml`. What matters day to day is who sets each kind —
most of it is machinery, and hand-moving a machine-owned label just gets
corrected on the next pass:
| Labels | Set by | | Labels | Set by |
|---|---| |---|---|
| `state:*` | the labels workflow ([.github/workflows/labels.yml](.github/workflows/labels.yml)) — recomputed from GitHub's own facts on PR events (label changes included) and every 15 minutes. Machine-owned, with one exception: the author sets `state:needs-human` at handoff and the workflow reconciles it. Otherwise never by hand. Exactly one per PR: *whose ball is it.* | | `state:*` | the labels workflow ([.github/workflows/labels.yml](.github/workflows/labels.yml)) — recomputed from GitHub's own facts on PR events (label changes included) and every 15 minutes. Machine-owned, with one exception: the author sets `state:needs-human` at handoff (step 6) and the workflow reconciles it. Otherwise never by hand. Exactly one per PR: *whose ball is it.* |
| `blocker:*` | the same workflow, from the same facts — *what is in the way.* Any number per PR, or none. Never by hand: applying one does not stop a merge, and removing one does not unblock anything. Fix the thing and the next sweep drops the label. | | `blocker:*` | the same workflow, from the same facts — *what is in the way.* Any number per PR, or none. Never by hand: applying one does not stop a merge, and removing one does not unblock anything. Fix the thing and the next sweep drops the label. |
| `stale` | the same workflow — 48h without commits, comments, or reviews. `blocked` PRs are exempt: they are quiet legitimately. | | `stale` | the same workflow — 48h without commits, comments, or reviews. `blocked` PRs are exempt: they are quiet legitimately. |
| `scope:*` on PRs | actions/labeler, from the changed paths ([.github/labeler.yml](.github/labeler.yml)). Additive — you may add more, the machine won't remove them. | | `scope:*` on PRs | actions/labeler, from the changed paths ([.github/labeler.yml](.github/labeler.yml)). Additive — you may add more, the machine won't remove them. |

187
LABELS.md Normal file
View file

@ -0,0 +1,187 @@
# Labels
How this repo uses GitHub labels. The taxonomy is shared across the
heavy-duty repos (box, rig, cast) — only the `scope:` set differs per repo,
because it names this repo's actual surfaces.
## State — who is the ball with? (PRs, exactly one)
Every open PR carries exactly one `state:` label, and it answers the only
question a board scan actually asks: *who is this PR waiting on?* The states
mirror the review loop this repo runs — PRs open as drafts, three reviewer
bots pick up ready PRs with reviews requested, each round is answered in a
single reply, and a human takes the final review.
| Label | Color | Waiting on | Enters when | Leaves when |
|---|---|---|---|---|
| `state:building` | `#FBCA04` | the coding agent, still building | PR opened as draft | marked ready + bot reviews requested |
| `state:bots-reviewing` | `#1D76DB` | the reviewer bots to finish the round | ready with reviews requested, or fixes pushed and reviews re-requested | all three bots have reviewed the round |
| `state:addressing` | `#D93F0B` | the coding agent to reply, fix, or ask | all bots reviewed and not all approved; or nobody was asked; or a blocker is up | the round-reply is posted and fixes pushed — and any blocker named alongside is cleared |
| `state:needs-human` | `#8250DF` | the human reviewer | the PR **could be merged right now**: no blockers, three formal head-current approvals — and the human review is requested | merged — or changes requested, which cycles back to `state:addressing` |
`bots-reviewing` and `addressing` are deliberately distinct: staleness in the
first means *poke the bots*, staleness in the second means *the agent dropped
the ball*. Collapsing them loses exactly the information a sweep needs.
`bots-reviewing` therefore means strictly *a request is live and an answer is
coming* — a PR nobody was asked to review is the agent's ball, not the bots'.
## The second axis: `blocker:*`
State answers *whose ball is it*. Blockers answer *what is in the way*, and
unlike states they are *facts about the branch* — mutually independent, so a
PR carries as many as apply.
| Label | Color | Means | Clears when |
|---|---|---|---|
| `blocker:conflict` | `#B60205` | GitHub says `CONFLICTING` — the agent owes a **rebase** | it merges cleanly |
| `blocker:ci-red` | `#B60205` | a check failed — the agent owes a **fix**, which a rebase will not provide | checks are green |
| `blocker:unrequested` | `#E99695` | this head has no verdict from somebody — never reviewed, or staled by a push — and **nobody was asked** for one | reviews are requested |
| `blocker:drill-pending` | `#E99695` | a `release` PR whose version has **no drill record** at [`drills/<version>.md`](drills/README.md) — the ceremony is correct but *unevidenced* | the drill is run and recorded, or a maintainer waiver is recorded for that version |
`blocker:drill-pending` is the one blocker that is not about the code: the
branch merges, the checks that read the tree are green, and the release is
still not shippable because nothing says it was ever run on real hardware.
`.github/scripts/drill-recorded.sh` is the authority — the label just makes
the reason legible on the board, so a release PR sitting still reads as
"waiting on an afternoon of hardware", not as "forgotten". It only ever
appears on a `release` PR: every `-dev` tree satisfies the guard vacuously.
It is the one `blocker:*` the reconciler does **not** compute — its `BLOCKERS`
set is the three above — so it is applied by hand and, being outside that set,
is not stripped on the next sweep. (The red check itself still shows up as
`blocker:ci-red`; this label says *which* red.)
**A maintainer account must create this label.** The bot account 403s on label
creation, so until someone with push access runs the `gh label create` line
below, use plain `blocked` on such a PR — it carries the right meaning
(waiting on something else to happen first) and the staleness sweep already
exempts it.
One rule joins the axes: **`state:needs-human` requires zero blockers.** Any
blocker means the work is the agent's, whatever the review round says.
This split exists because the single-label version kept lying. Independent
facts were projected onto one totally-ordered label, so one always had to win
and the losers vanished off the board: a PR that was *both* conflicted and red
could only say one of them, and `needs-rebase` told an agent to rebase when
what it actually owed was a bug fix. Precedence between two blockers is not a
question a set has to answer, which is why every ordering bug this machine has
had — `needs-human` surviving a conflict, `MISSING` swallowing `STALE` — lived
on the axis that had to be totally ordered.
`state:needs-rebase` was the first attempt at this and is **retired**; the
reconciler strips it on sight so no PR is left carrying a label nothing
recomputes.
**`state:needs-human` means one thing: a human could merge this right now.**
The label is the only signal a maintainer scanning the board (or a phone)
actually reads, and one that says "your turn" on an unmergeable PR is worse
than no label at all. So beyond the blockers, one review fact also outranks an
explicit human request:
- **nobody reviewed *this* head** — every approval staled by a push → `state:addressing`,
because the agent owes a re-request
That case is more dangerous than any blocker: a blocked PR at least shows an X
or a disabled merge button, while a staled-approval PR reads green, mergeable
and "waiting on the human" over code no reviewer has seen.
`UNKNOWN` mergeability is deliberately **not** treated as a conflict. GitHub
reports it for about a minute after every merge while it recomputes, and
flapping every open PR through `blocker:conflict` on each merge would be worse
than the bug this fixes. A failed read of either branch fact degrades to the
same "do not know" value, for the same reason.
An *unfinished* round still yields to an explicit human request — a maintainer
pulling a PR to themselves early is a deliberate act. `MISSING` (nobody has
reviewed yet) and `STALE` (everyone reviewed something else) are different
facts and are treated differently.
## Cross-cutting (PRs and issues)
| Label | Color | Meaning |
|---|---|---|
| `stale` | `#B60205` | No activity for 48h. Sweep-managed, never hand-applied. `state:building` + `stale` is precisely a forgotten draft. |
| `blocked` | `#6A737D` | Waiting on another PR or issue to land first. Quiet *legitimately* — the staleness sweep skips it. |
| `release` | `#0E8A16` | Release flow, versioning, and packaging work. |
| `merge-next` | `#0E8A16` | Head of the merge queue — **merge this one next**. Queue order is *intent* (which PR lands first, given how they conflict), so the reconciler never sets it: you or the agent maintaining the queue do. The reconciler only **clears** it, the moment the PR stops being something a human could merge — so it cannot go stale the way `state:needs-human` did. |
## Scope — which surface? (PRs and issues, any number)
All scopes share one calm color, `#C5DEF5` — scopes locate, states alert.
| Label | Covers |
|---|---|
| `scope:bootstrap` | `commands/bootstrap.sh` — hardening a pristine server into a node |
| `scope:users` | `commands/users-*` — the root-door model, apply/status, close-root |
| `scope:runner` | `commands/runner-*` — GitHub runner install/remove/repoint/status |
| `scope:coolify` | `commands/coolify-*` — Coolify and its backup install |
| `scope:db` | `commands/db.sh` — dump/restore and the round-trip proof |
| `scope:installer` | `install.sh` — how rig itself lands on a machine |
## Issue types
`bug`, `enhancement`, `documentation` — issues only. PRs carry their type in
the conventional title (`feat:`, `fix:`, `docs:`), so typing a PR with a label
would just say the same thing twice, drifting apart eventually.
## Maintenance
State labels are machine-owned, with exactly one exception. Every state above
is derivable from GitHub's own facts — the draft flag, requested reviewers,
review states, push timestamps — so the labels workflow
([.github/workflows/labels.yml](.github/workflows/labels.yml)) recomputes the
state and reconciles labels statelessly, on PR events (label changes included)
plus a 15-minute cron. A hand-moved label is a lie waiting to happen; the
workflow asserts the effective state instead.
The exception is `state:needs-human`, which the author sets at handoff
([CONTRIBUTING.md](CONTRIBUTING.md), step 6). That is an optimistic write, not
a transfer of ownership: because `pull_request_target: labeled` wakes the
workflow, the author's own label write fires the sweep that validates it, and
a handoff that had not earned the label is corrected within seconds.
It exists because the wake signal was missing. There is no
`pull_request_review_target` — on fork PRs, which is all of them here,
`pull_request_review` runs read-only and cannot label anything — so the moment
the label becomes true, the third approval landing, fired nothing at all. What
was left was the `*/15` cron, and GitHub deprioritises short intervals hard
enough that the delivered rate is closer to hourly. The label could therefore
lag the round it described by hours, worst on the quietest repo: every sweep
reconciles the whole board, so a busy repo stays fresh by piggybacking on
unrelated PR events, while a quiet one depends on the cron most and receives
it least. `scope:` labels on PRs are applied from the changed
paths by actions/labeler ([.github/labeler.yml](.github/labeler.yml));
[CONTRIBUTING.md](CONTRIBUTING.md) says who sets what.
The same workflow bootstraps the taxonomy: a manual dispatch creates any
missing label idempotently. To create them by hand (needs push access):
```sh
gh label create "state:building" --color FBCA04 --description "PR is a draft — the coding agent is still building" --force
gh label create "state:bots-reviewing" --color 1D76DB --description "Waiting on the bot reviewers to finish the round" --force
gh label create "state:addressing" --color D93F0B --description "All bots reviewed — coding agent owes the single reply + fixes" --force
gh label create "blocker:conflict" --color B60205 --description "Does not merge — the branch conflicts and the agent owes a rebase" --force
gh label create "blocker:ci-red" --color B60205 --description "A check is failing — the agent owes a fix (not a rebase)" --force
gh label create "blocker:unrequested" --color E99695 --description "Somebody still owes a verdict and nobody was asked for one" --force
# Needs a MAINTAINER account — the bot 403s on label creation. Until it exists, `blocked` stands in.
gh label create "blocker:drill-pending" --color E99695 --description "Release PR with no drill record at drills/<version>.md — correct but unevidenced" --force
# retired — the reconciler strips it; delete it once no PR carries it
# gh label delete "state:needs-rebase"
gh label create "state:needs-human" --color 8250DF --description "No blockers, all bots approve — waiting on the human reviewer" --force
gh label create "merge-next" --color 0E8A16 --description "Head of the merge queue — merge this one next (set by hand/agent, cleared here)" --force
gh label create "stale" --color B60205 --description "No activity for 48h — needs a poke (sweep-managed)" --force
gh label create "blocked" --color 6A737D --description "Waiting on another PR or issue to land first" --force
gh label create "release" --color 0E8A16 --description "Release flow and version/packaging work" --force
gh label create "scope:bootstrap" --color C5DEF5 --description "bootstrap — hardening a pristine server into a node" --force
gh label create "scope:users" --color C5DEF5 --description "users-* — root-door model, apply/status, close-root" --force
gh label create "scope:runner" --color C5DEF5 --description "runner-* — GitHub runner lifecycle" --force
gh label create "scope:coolify" --color C5DEF5 --description "coolify-* — Coolify and backup install" --force
gh label create "scope:db" --color C5DEF5 --description "db.sh — dump/restore" --force
gh label create "scope:installer" --color C5DEF5 --description "install.sh — how rig lands on a machine" --force
# delete is not an upsert: a label that is already gone exits non-zero. Swallow
# that, so this block converges on re-run instead of erroring after first success.
for L in duplicate invalid question wontfix "help wanted" "good first issue"; do
gh label delete "$L" --yes 2>/dev/null || true
done
```

359
README.md
View file

@ -4,7 +4,7 @@ A CLI that turns a **pristine Debian server into a hardened, tailnet-joined
node** — one curl, one command. A second command installs a version-pinned node** — one curl, one command. A second command installs a version-pinned
Coolify on a control-plane box. And inside a [box](https://github.com/heavy-duty/box)-minted Coolify on a control-plane box. And inside a [box](https://github.com/heavy-duty/box)-minted
guest, the same verb converges the **box tenants** — claude-box, codex-box, guest, the same verb converges the **box tenants** — claude-box, codex-box,
grok-box, kimi-box, staging-box — from thin, creds-free seeds (see *the box tenants* grok-box, staging-box — from thin, creds-free seeds (see *the box tenants*
below). below).
Philosophy (shared with [box](https://github.com/heavy-duty/box)): Philosophy (shared with [box](https://github.com/heavy-duty/box)):
@ -15,15 +15,7 @@ takes arguments, does its work, and stores no credential, ever.
## Install ## Install
```sh ```sh
# the development tree this README documents:
curl -fsSL https://raw.githubusercontent.com/heavy-duty/rig/main/install.sh | RIG_REF=main bash curl -fsSL https://raw.githubusercontent.com/heavy-duty/rig/main/install.sh | RIG_REF=main bash
# the latest release:
curl -fsSL https://raw.githubusercontent.com/heavy-duty/rig/main/install.sh | bash
# the same two channels from the Forgejo mirror (RIG_HOST picks the forge; #111):
curl -fsSL https://forgejo.heavyduty.builders/heavy-duty/rig/raw/branch/main/install.sh \
| RIG_HOST=https://forgejo.heavyduty.builders RIG_REF=main bash
curl -fsSL https://forgejo.heavyduty.builders/heavy-duty/rig/raw/branch/main/install.sh \
| RIG_HOST=https://forgejo.heavyduty.builders bash
``` ```
This README tracks `main`, so the quick start installs that same development This README tracks `main`, so the quick start installs that same development
@ -42,11 +34,6 @@ curl -fsSL .../install.sh | RIG_REF=main bash # the development tree
A tag outranks a branch of the same name (the pin must win); anything that A tag outranks a branch of the same name (the pin must win); anything that
is not a tag falls back to `refs/heads/<ref>`. is not a tag falls back to `refs/heads/<ref>`.
`RIG_HOST` picks the forge the channel reads — `https://github.com` by
default (#111) — and it is chosen independently of the URL the script itself
came from, so installing *from* the Forgejo mirror means naming the mirror
twice, as the quick start's third line does.
The layout, under the install root (`~/.local/share/rig`): The layout, under the install root (`~/.local/share/rig`):
``` ```
@ -104,7 +91,7 @@ itself is untouched — what bootstrap converged stays converged.
Run as root on the fresh box (over SSH). Convergent — safe to re-run; a Run as root on the fresh box (over SSH). Convergent — safe to re-run; a
second run changes nothing. (The box TENANT roles — `claude-box`, `codex-box`, second run changes nothing. (The box TENANT roles — `claude-box`, `codex-box`,
`grok-box`, `kimi-box`, `staging-box` — share the verb but are their own family; the `grok-box`, `staging-box` — share the verb but are their own family; the
`-box` suffix says so. See *the box tenants* below.) `-box` suffix says so. See *the box tenants* below.)
```sh ```sh
@ -180,7 +167,7 @@ trait), and every other way that step can fail lands in apply's existing
refusal at the end. refusal at the end.
`--users` does **not** reach the box TENANT roles (`claude-box`, `codex-box`, `--users` does **not** reach the box TENANT roles (`claude-box`, `codex-box`,
`grok-box`, `kimi-box`, `staging-box`). A tenant is a box-minted *guest*: box auto-runs its bootstrap at `grok-box`, `staging-box`). A tenant is a box-minted *guest*: box auto-runs its bootstrap at
mint, non-interactively, with no file to hand it; the guest never joins the mint, non-interactively, with no file to hand it; the guest never joins the
tailnet and has no SSH door of its own — you enter with `box shell`, gated by tailnet and has no SSH door of its own — you enter with `box shell`, gated by
the **host's** `incus` grants, which the host's own users file already the **host's** `incus` grants, which the host's own users file already
@ -250,33 +237,11 @@ the only shapes it manages — every other role refuses an effective
`tag:server` after join, one rule instead of per-role exceptions. `tag:server` after join, one rule instead of per-role exceptions.
After the tag verification passes, bootstrap writes `/etc/rig/role` — one After the tag verification passes, bootstrap writes `/etc/rig/role` — one
line, `role=… root-door=… host=… join=… join-by=…` — recording the line, `role=… root-door=… host=… join=…` — recording the **effective** traits,
**effective** traits, overrides and all, plus whether this run performed the overrides and all, so an overridden role never lies to the commands that read
tailnet join. `join-by=rig` means bootstrap called `tailscale up`; the marker later (`rig users` keys root policy off `root-door=`). Written
`join-by=preexisting` means it found the node already joined. Old markers name post-join and cmp-guarded, so a marker never describes a box that failed to
neither and are treated as unknown, never as permission to remove a join. become what it claims.
Written post-join and cmp-guarded, so a marker never describes a box that
failed to become what it claims.
### `rig bootstrap --undo`
```sh
sudo rig bootstrap --undo
```
Leaves the tailnet and then removes `/etc/rig/role`, but only when the marker
says `join-by=rig`. A pre-existing join, an old marker with no provenance, or
no marker at all is refused without calling `tailscale logout`; the refusal
names the manual repair. Re-running bootstrap writes the current marker shape.
Undo also refuses while a GitHub runner is installed and points at
`rig runner remove`, because restoring the local machine while leaving an
off-box runner registration would create a ghost in the repository. If
`tailscale logout` fails, the marker stays in place so the command is retryable.
This is intentionally not a general rollback. It does not uninstall packages,
reverse sshd hardening, remove Docker, Node, agent CLIs, or users. Those changes
are convergent rather than transactional and cannot be safely inferred away.
Immediately after it, bootstrap stamps `/etc/rig/manifest` — **provenance**: Immediately after it, bootstrap stamps `/etc/rig/manifest` — **provenance**:
which rig converged this box and when (see [`rig which rig converged this box and when (see [`rig
@ -393,17 +358,23 @@ box is present) and **opt-out** (`RIG_SKIP_BOX_INSTALL=1`, plus a graceful skip
with a manual-command pointer when curl or the network is missing — box is the with a manual-command pointer when curl or the network is missing — box is the
host *extra*, so a failed box install never aborts a bootstrap that otherwise host *extra*, so a failed box install never aborts a bootstrap that otherwise
succeeded). Source is pinnable with `BOX_REPO` / `BOX_REF` (default succeeded). Source is pinnable with `BOX_REPO` / `BOX_REF` (default
`heavy-duty/box@0.9.0`). If `/dev/kvm` is absent, rig warns (a host that exists to `heavy-duty/box@main`). If `/dev/kvm` is absent, rig warns (a host that exists to
run VMs should have it) but does not fail — the shape is rehearsed in containers, run VMs should have it) but does not fail — the shape is rehearsed in containers,
which legitimately lack it. (The world-readable global install path — box under which legitimately lack it. (The world-readable global install path — box under
`/opt/box` readable by every non-root user — depends on box PR #71; until that `/opt/box` readable by every non-root user — depends on box PR #71; until that
merges box's root install lands in `/root`.) merges box's root install lands in `/root`.)
> **The box install is release-pinned.** A rig release carries one box release > **The box install is unpinned — on purpose, and out loud.** `coolify install`
> pin, so two machines bootstrapped from the same rig install the same box. > demands a version pin; the box step tracks a moving `heavy-duty/box@main`.
> `BOX_REPO` / `BOX_REF` remain explicit overrides for development and > Not because box self-updates (it doesn't — it has Coolify's shape, not the
> pre-release drills; `RIG_SKIP_BOX_INSTALL=1` opts out entirely for a host > runner's) but because there is nothing to pin *to*: box cuts no tags and no
> whose box you manage by hand. > releases, and its installer resolves `refs/heads/<ref>` — branches only — so
> a `BOX_REF=v0.5.0` would 404 even if the tag existed. Issue #12's call was
> that silently tracking `main` on the box that runs the agents is the option
> not to pick — hence this paragraph. `BOX_REPO` / `BOX_REF` are the pin
> points the day box cuts a tag (or you point at a frozen branch of your own
> fork); `RIG_SKIP_BOX_INSTALL=1` opts out entirely for a host whose box you
> manage by hand.
`dev-server` is the closed-door VM-hosting shape — `tag:local`, box CLI installed as `dev-server` is the closed-door VM-hosting shape — `tag:local`, box CLI installed as
above, operators entering as themselves (`--root-door open` turns it into the above, operators entering as themselves (`--root-door open` turns it into the
@ -411,41 +382,13 @@ unattended VM-host appliance) — and `workstation` is the machine at the keyboa
end of all the SSH connections: `root-door=closed`, `join=login`, entering the end of all the SSH connections: `root-door=closed`, `join=login`, entering the
tailnet as *your* device rather than the fleet's. tailnet as *your* device rather than the fleet's.
### Machine-role templates ### `rig bootstrap <claude-box|codex-box|grok-box|staging-box>` — the box tenants
Machine presets can also live in the
[heavy-duty/rig-templates](https://github.com/heavy-duty/rig-templates)
registry. A `*-server` directory is a fleet-machine definition; the exact
name `workstation` is the deliberate suffix-less exception. Its
`template.env` contains exactly the three traits bootstrap's built-in table
uses:
```dotenv
ROOT_DOOR="open" # open|closed
HOST="no" # yes|no
JOIN="authkey" # authkey|login
```
An `install.sh` is optional. When present, bootstrap runs it as root,
non-interactively, from the definition directory with `RIG_ROLE` set, after
the tailnet join, host setup, role marker prerequisites, and operator
convergence. A nonzero exit fails bootstrap and names the role and registry
source. The definition owns idempotence, just as bootstrap does.
Built-in roles and `custom` take precedence over registry names. Any other
non-tenant role is looked up in the resolved registry; the same three source
knobs below apply, including `RIG_TEMPLATES_DIR` for an offline local
definition. Pin reviewed registry content into rig's tree before using it on
fleet machines: an optional machine `install.sh` executes as root on metal,
and an override is the operator explicitly choosing a different trust root.
### `rig bootstrap <role>-box` — the box tenants
Run as root, **inside** a [box](https://github.com/heavy-duty/box)-minted Run as root, **inside** a [box](https://github.com/heavy-duty/box)-minted
guest. Convergent — safe to re-run; a second run changes nothing. guest. Convergent — safe to re-run; a second run changes nothing.
```sh ```sh
rig bootstrap claude-box # or codex-box, grok-box, kimi-box — the agent tenants rig bootstrap claude-box # or codex-box, grok-box — the agent tenants
rig bootstrap staging-box # the server tenant (docker + sshd hardening) rig bootstrap staging-box # the server tenant (docker + sshd hardening)
rig bootstrap claude-box --user dev # when the seed's BOX_USER differs rig bootstrap claude-box --user dev # when the seed's BOX_USER differs
``` ```
@ -460,63 +403,15 @@ same harness as everything else — and re-runnable on an *existing* box to
converge it to a new spec instead of re-minting it. One convergence engine; converge it to a new spec instead of re-minting it. One convergence engine;
the guests were the hole. the guests were the hole.
It is **one mechanism, parameterized per DEFINITION** (#110), not four It is **one mechanism, parameterized per tenant** (`lib/tenant-config.sh`
hand-maintained scripts. The agent-tenant definitions live in the holds the whole per-tenant table), not four hand-maintained scripts:
**[heavy-duty/rig-templates](https://github.com/heavy-duty/rig-templates)
registry** — one directory per role (`template.env`, the allowlisted data
table rig parses and never sources; `install.sh`, the CLI install; `creds.md`,
the per-vendor context paragraph) — so adding an agent tenant is a data PR
there, never a mechanism edit here (#109 was the evidence: adding kimi, pure
data, meant editing six files in this repo). Which `-box` roles exist is the
registry's fact; `rig bootstrap <anything>-box` dispatches on the family
suffix and refuses an unknown role by listing what the resolved registry
actually contains. `staging-box` is the one in-tree tenant — mechanism-adjacent
(sshd hardening through the shared `lib/sshd.sh`, docker, no agent), user
`ops`, box#69's server posture with `root-door=open` acceptance.
**Where the registry comes from — precedence high to low:** | tenant role | user | what lands |
|---------------|----------|------------|
| knob | meaning | | `claude-box` | `claude` | the agent toolbelt (git, gh, tmux, ripgrep, jq, age, unzip, build-essential), docker, node 22, the Claude Code CLI on the system PATH, zsh + oh-my-zsh, and `~/.claude/CLAUDE.md` |
|------|---------| | `codex-box` | `codex` | the toolbelt, docker, node 22, `@openai/codex` on the system PATH, and `~/.codex/AGENTS.md` |
| `RIG_TEMPLATES_DIR` | a local folder — no fetch: the offline-test path, and "try a template before it exists anywhere" | | `grok-box` | `grok` | the toolbelt, docker, the grok CLI on the system PATH, and `~/.grok/AGENTS.md` |
| `RIG_TEMPLATES_REF` | any ref of `RIG_TEMPLATES_REPO` (default `heavy-duty/rig-templates`), fetched as an unauthenticated tarball at bootstrap time | | `staging-box` | `ops` | box#69's server posture: docker + the same sshd hardening the machine roles get (shared `lib/sshd.sh`, `root-door=open` acceptance) |
| `RIG_TEMPLATES_HOST` | which **forge** that repo lives on (default `https://github.com`) — not a mirror knob but a URL-grammar one: GitHub serves three candidate archive paths (`refs/tags`, `refs/heads`, bare) and Forgejo serves exactly one (`/archive/<ref>.tar.gz`), so the host decides what is even worth requesting. `install.sh` reads the same variable for its snapshot, so the two cannot disagree about where the registry is |
| *(neither set; matching snapshot installed)* | **the installed pin snapshot** — `install.sh` best-effort fetches `RIG_TEMPLATES_PIN` once into `templates@<pin-sha>/` inside the versioned rig tree; default converges read it with zero registry network I/O |
| *(snapshot absent, empty, or stale)* | **live fetch of the in-tree pin** — the pre-snapshot fallback: `RIG_TEMPLATES_PIN` in `commands/lib/templates.sh` is fetched at converge time. A failed snapshot download only warns during install, so rig remains usable and retries here |
The pin remains the only source of truth. An older `templates@<sha>/`
directory cannot answer after a pin bump, and an explicit
`RIG_TEMPLATES_REF` always fetches that ref rather than consulting the
snapshot. Logs mark the installed path as `(snapshot)` so drill evidence
records which source actually served the converge.
> **A self-hosted forge must serve public repos anonymously.** The fetch is
> unauthenticated *by contract* — box auto-runs a tenant bootstrap at mint,
> holding nothing — so `RIG_TEMPLATES_HOST` can only point somewhere a
> credential-less `curl` succeeds. Forgejo with `REQUIRE_SIGNIN_VIEW=true`
> answers **404 for a repo it reports as public**, which is exactly what a
> wrong ref looks like; the refusal names this case rather than leaving you
> hunting for a typo. `FORGEJO__service__REQUIRE_SIGNIN_VIEW=false` is what
> makes an instance usable as a registry host.
>
> Verified on `forgejo.heavyduty.builders` (2026-07-27): a credential-less
> `templates_resolve` against `RIG_TEMPLATES_HOST=https://forgejo.heavyduty.builders`
> fetches and extracts a real repository archive. The mint-time path works
> there today.
**The security trade — in bold, not a footnote.** **A main-tracked
rig-templates repo means every merged PR there executes as root inside every
future mint.** This is acceptable — and an improvement — only because of
three facts together: (1) it *narrows* today's surface, where all of rig is
main-tracked-as-root; (2) the repo is small, single-purpose, and
ceremony-governed with a **human merge** as the gate and the review panel
ahead of it; (3) drills pin the SHA they proved. If any of those three
weakens, the default flips to a pinned `RIG_TEMPLATES_REF`. install.sh diffs
in that repo are the highest-trust review surface in the org — the reviewer
doctrine should say so. *(2026-07-24: the flip this paragraph reserves was
taken, before the migration and by the decider — the default IS the pin
above, so a merged template reaches mints only through a reviewed pin bump
here, or an explicit per-mint `RIG_TEMPLATES_REF`.)*
**The role carries the suffix; the user does not.** A tenant user is the **The role carries the suffix; the user does not.** A tenant user is the
account the box *seed* created (`BOX_USER`) and the agent CLI's own dotdir account the box *seed* created (`BOX_USER`) and the agent CLI's own dotdir
@ -550,9 +445,8 @@ disposability facts, and the guard note — **never run `box setup-host`,
`box teardown-host`, or the drill inside a box; the box you are in is not a `box teardown-host`, or the drill inside a box; the box you are in is not a
host you own**. A nested box stack claims the guest's own uplink subnet and host you own**. A nested box stack claims the guest's own uplink subnet and
silently breaks its networking (box#80). The note lives in silently breaks its networking (box#80). The note lives in
`lib/templates.sh`'s renderer exactly once — mechanism, not template data — `lib/tenant-config.sh` exactly once, not copy-pasted per template — that was
never copy-pasted per definition; that was the point of moving it here. Only the point of moving it here.
the creds paragraph is the definition's (`creds.md`).
**Tenants and the role marker.** A tenant run writes `role=<tenant> tenant=yes **Tenants and the role marker.** A tenant run writes `role=<tenant> tenant=yes
host=no` — no `root-door=`, because a guest has no root-door policy of its own host=no` — no `root-door=`, because a guest has no root-door policy of its own
@ -839,7 +733,6 @@ What is this machine — computed at run time, **stored nowhere**:
``` ```
PLATFORM PLATFORM
HOSTNAME hetzner-cp-1 HOSTNAME hetzner-cp-1
ID cd9fb802-1493-2336-d027-7955f328bcd8
OS Debian GNU/Linux 13 (trixie) OS Debian GNU/Linux 13 (trixie)
KERNEL 6.12.95+deb13-amd64 (x86_64) KERNEL 6.12.95+deb13-amd64 (x86_64)
CPU AMD Ryzen 7 3700X 8-Core Processor (16 cores) CPU AMD Ryzen 7 3700X 8-Core Processor (16 cores)
@ -867,33 +760,10 @@ Computing at run time removes the problem instead of managing it: the answer
is correct by construction because there is nothing to go stale. is correct by construction because there is nothing to go stale.
The corollary is deliberate: **`rig platform` works on a machine rig has never The corollary is deliberate: **`rig platform` works on a machine rig has never
converged.** It reads only `/proc`, `uname`, `/etc/os-release`, converged.** It reads only `/proc`, `uname`, `/etc/os-release`, `df` and
`/etc/machine-id`, `df` and `systemd-detect-virt`, so it runs on bare Debian `systemd-detect-virt`, so it runs on bare Debian before bootstrap — useful for
before bootstrap — useful for deciding *what to converge this into*, not just deciding *what to converge this into*, not just for auditing afterwards. It
for auditing afterwards. It needs no root, makes no network call, and writes needs no root, makes no network call, and writes nothing, ever.
nothing, ever.
**`ID` names the machine where `HOSTNAME` names the slot** — that contrast is
why they sit together. rig sets the hostname itself during bootstrap and
reuses it across rebuilds (`hetzner-cp-1` is a role, not hardware), so the
hostname cannot answer "is this the same machine I converged in June, or its
replacement?". `ID` can: it is derived from `/etc/machine-id` as
`sha256("rig-machine-id:<machine-id>")`, first 32 hex chars rendered
8-4-4-4-12 — computed at run time and stored nowhere, like every other fact in
the block, so it exists before bootstrap too. It is deliberately **not** the
raw machine-id: `machine-id(5)` asks that the value not be exposed, and the
namespaced hash is its documented remedy — a reader of `rig platform` output
cannot recover `/etc/machine-id`, nor correlate the id with any other tool's
derivation of it. A missing, empty or `uninitialized` machine-id renders
`ID unavailable (reason)` while every other field still reports; it is never
an empty string and never a hash of nothing, which would hand every such
machine the same identity.
**Two machines reporting the same `ID` means a cloned image** — actionable
information, not a coincidence. A host cloned from a golden image carries the
image's `/etc/machine-id`, and no identity that lives in the filesystem
survives the filesystem being copied. If you hit it, regenerate the clone's
machine-id (`systemd-machine-id-setup`) rather than doubting the field.
The `PROVENANCE` block is the complementary half — which rig, and when, which The `PROVENANCE` block is the complementary half — which rig, and when, which
is *decided* rather than observed, so it is stored. It is **read, never is *decided* rather than observed, so it is stored. It is **read, never
@ -1079,163 +949,6 @@ prints the exact `runner install` line that finishes the job.
Convergent — repointing to the repo it is already on changes nothing, exits 0, Convergent — repointing to the repo it is already on changes nothing, exits 0,
and never asks for a token. and never asks for a token.
### `rig forgejo-runner install --instance <url>`
The other forge's runner, and a different shape of box. Where `rig runner`
converges a fleet **machine** (`runner-server`), this converges a **ci-box
tenant** — a box guest whose whole job is running CI:
```sh
box mint ci-box # box auto-runs: rig bootstrap ci-box
box shell ci-box
sudo rig forgejo-runner install --instance https://forgejo.heavyduty.builders
```
Installs `forgejo-runner` as a systemd service under an unprivileged user
(default: the tenant user `ci`). Like its GitHub sibling the runner is an
agent, not a server — it long-polls the instance outbound and needs **zero
inbound ports**.
**Jobs run in containers, and there is no docker-in-docker.**
`rig bootstrap ci-box` already installed Docker and put the tenant user in the
`docker` group, so the runner drives that daemon directly. The usual dind
sidecar — privileged, with a plaintext `tcp://…:2375` socket — exists to
isolate jobs from a *shared* CI server; inside a box that boundary is already
paid for. The box is network-isolated, has no inbound path, and is thrown away.
That is also why this command allows what `rig runner install` refuses. Docker
group membership is root-equivalent, and on a fleet machine the blast radius is
the machine. Here it is a disposable guest. Same trade, different box, opposite
answer — which is why these are two commands and not one with a `--forge` flag.
- `--version <pin>` — `forgejo-runner` release (default: latest at install
time). **Convergent, and it replaces an existing binary** — including
downward, because a pin is an instruction rather than a floor. Without a
pin, a binary already on the box is left alone: chasing "latest" on every
converge would make a plain re-run an unrequested upgrade.
The published `.sha256` is **verified before the binary is installed**, and
a missing or unreadable checksum **refuses** just as a mismatch does — this
binary runs as root under a systemd unit, so the gate does not fail open.
> Unlike `rig runner install`, presence alone is not enough to skip the
> download here. That command can skip because `actions/runner`
> **self-updates**; `forgejo-runner` does not, so nothing else would ever
> move the version — and a ci-box's template preinstalls the binary at mint,
> which would leave `--version` doing nothing on the exact path this command
> is for.
- `--name <name>` — runner name (default: this host's hostname)
- `--labels <csv>` — replaces the default map:
`ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-22.04,docker:docker://node:22-bookworm`,
so `runs-on: ubuntu-latest` works in a workflow written for GitHub.
**Applied at registration only.** Forgejo owns a runner's labels from the
moment it registers, so passing `--labels` to a re-run cannot change them —
rig says so rather than letting the request evaporate, and changing labels
means `remove` then `install` again
- `--user <name>` — service user (default: `ci` when it exists, else
`forgejo-runner`)
**There is no `--repo`, and that is the substantive difference.** A Forgejo
runner registers to an *instance*; whether it then serves that whole instance,
one organisation, or one repository is a property of the **registration
token**, which you mint in Forgejo's UI at the scope you want:
| scope | where the token comes from |
|---|---|
| instance | Site Administration → Actions → Runners |
| organisation | Org → Settings → Actions → Runners |
| repository | Repo → Settings → Actions → Runners |
Pass it via `FORGEJO_RUNNER_TOKEN` or the interactive prompt. It is consumed at
registration and never written to disk by rig.
Convergent **toward `--instance`** — re-running against the instance the box is
already on re-uses the binary and skips registration. Pointed at a *different*
instance it refuses and names both, for the same reason `rig runner install`
refuses a different repo.
> **`.runner` holds a credential here.** GitHub's names a repository; Forgejo's
> holds the runner's own long-lived token. rig installs it `0600` owned by the
> runner user and **re-asserts that mode on every converge** — a mode that
> drifted leaks the secret silently, since nothing fails and the runner keeps
> working. `status` warns when it finds one that has.
### `rig forgejo-runner status`
Which instance this box's runner is registered to — instance, name, labels,
directory, unit and state. Reads the box only: no token, no network call, and
it never prints the registration secret `.runner` holds. Exits 1 when no runner
is installed.
### `rig forgejo-runner remove`
Stops and disables the service and wipes the local registration. The binary and
the user stay, so a later `install` re-registers without downloading anything.
**Always local-only, and there is no `--local` flag.** Forgejo has no runner
deregistration endpoint — no removal token, nothing to hand back — so the box
is cleaned and the runner stays listed as offline until you delete it under
Actions → Runners. Offering the flag would advertise a server-side alternative
that does not exist. For the same reason there is no `forgejo-runner repoint`:
a move cannot be one atomic act, so it is `remove` then `install`.
### Enabling Actions on the Forgejo side
Two environment variables on the Forgejo service (Coolify → Environment
Variables → redeploy). Forgejo maps `FORGEJO__<section>__<KEY>` onto its
config, so this survives image upgrades in a way an edited `app.ini` does not:
```
FORGEJO__actions__ENABLED=true
FORGEJO__actions__DEFAULT_ACTIONS_URL=https://code.forgejo.org
```
> **`DEFAULT_ACTIONS_URL` is a single fallback, and it only governs *steps*.**
> It decides where a bare `uses: owner/repo@ref` on a **step** resolves — and
> nothing else. Measured on this instance, with a registered runner (#112):
>
> | reference | count | bare resolves to | verdict |
> |---|---|---|---|
> | `actions/checkout@v4` (step) | 3 | `code.forgejo.org` | **200** — mirrored, keep bare |
> | `heavy-duty/ceremony/actions/…` (step) | 6 | `code.forgejo.org` | **404** — must be absolute |
> | `heavy-duty/ceremony/.github/workflows/…` (reusable) | 2 | **this instance** | **200** — keep bare |
>
> The last row is the one that surprises: a **reusable-workflow** `uses:` never
> consults `DEFAULT_ACTIONS_URL` at all. It resolves against the runner's own
> instance, so the `release.yml` and `labels.yml` callers already work bare.
>
> **The ruling is to keep the value above** — `actions/*` are exactly what
> `DEFAULT_ACTIONS_URL` is designed to resolve, and `code.forgejo.org` mirrors
> them — and to make only the **six** first-party *step* references absolute:
>
> ```yaml
> - uses: https://forgejo.heavyduty.builders/heavy-duty/ceremony/actions/docs-sync@0.3.0
> ```
>
> The two reusable callers **stay bare**, and `release.yml`'s must: ceremony's
> `docs-sync` reads rig's pin out of that line with a grep anchored to the bare
> spelling, so absolutising it hides the pin and the guard exits 1. `test/cli.sh`
> pins all three shapes.
> `rig forgejo-runner` exists to run *your repositories'* workflows and does not
> depend on any of it — the value above is correct for those from the start.
> **A registry served from Forgejo needs one more.** `RIG_TEMPLATES_HOST`
> (below) lets the template registry live on any forge, but the mint-time fetch
> is **unauthenticated by contract** — box auto-runs `rig bootstrap <role>-box`
> at mint, holding no credentials. A Forgejo instance with
> `REQUIRE_SIGNIN_VIEW=true` answers **404 for public repos** to anonymous
> callers, which is indistinguishable from a wrong ref. So hosting the registry
> there also needs:
>
> ```
> FORGEJO__service__REQUIRE_SIGNIN_VIEW=false
> ```
>
> This affects only the *registry* fetch. `rig forgejo-runner` itself
> authenticates with a token and works either way.
>
> On `forgejo.heavyduty.builders` this is **already set** — verified
> 2026-07-27 by resolving a registry from it with no credentials at all.
### `rig users apply --file <path>` ### `rig users apply --file <path>`
Converges named operator accounts from a declarative users file — on **every** Converges named operator accounts from a declarative users file — on **every**

2
VERSION
View file

@ -1 +1 @@
0.3.2-dev 0.3.0

90
bin/rig
View file

@ -33,23 +33,15 @@ commands:
tenant roles); custom and workstation take none. join=login tenant roles); custom and workstation take none. join=login
(workstation) needs no key: interactive login, node must come up (workstation) needs no key: interactive login, node must come up
untagged. Run as root. untagged. Run as root.
bootstrap --undo bootstrap <claude-box|codex-box|grok-box|staging-box> [--user <name>]
Leave the tailnet only when /etc/rig/role proves rig performed the
join, then remove that marker. Refuses for pre-existing or old unknown
joins, and while a GitHub runner is installed. Run as root.
bootstrap <role>-box [--user <name>]
The box TENANT roles: converge a box-minted guest. The '-box' suffix The box TENANT roles: converge a box-minted guest. The '-box' suffix
names the family (a guest, vs the '-server' machine roles above). names the family (a guest, vs the '-server' machine roles above).
The agent tenants (claude-box, codex-box, grok-box, kimi-box, …) are claude-box|codex-box|grok-box land the agent toolbelt (git, gh, tmux, docker), the agent's CLI on
DEFINED in the heavy-duty/rig-templates registry — resolved via the system PATH, and the agent-context file (with the box#80 guard:
RIG_TEMPLATES_DIR > RIG_TEMPLATES_REF > the in-tree pin — and land never run box setup-host inside a box). staging-box lands box#69's
the agent toolbelt (git, gh, tmux, docker), the agent's CLI on the server posture — docker + sshd hardening; its tailnet join stays
system PATH, and the agent-context file (with the box#80 guard: operator-run via 'rig bootstrap workload-server'. Creds-free and non-interactive: box
never run box setup-host inside a box). staging-box, in rig's own auto-runs these at mint. Run as root, inside the box.
tree, lands box#69's server posture — docker + sshd hardening; its
tailnet join stays operator-run via 'rig bootstrap workload-server'.
Creds-free and non-interactive: box auto-runs these at mint. Run as
root, inside the box.
coolify install --version <pin> coolify install --version <pin>
Pinned Coolify install (AUTOUPDATE=false). Control-plane box only. Pinned Coolify install (AUTOUPDATE=false). Control-plane box only.
coolify backup install [options] coolify backup install [options]
@ -87,23 +79,6 @@ commands:
re-register, reusing the binary already on the box. Needs a removal re-register, reusing the binary already on the box. Needs a removal
token for the old repo and a registration token for the new one. token for the old repo and a registration token for the new one.
Run as root. Run as root.
forgejo-runner install --instance <url> [options]
Forgejo Actions runner as a systemd service under an unprivileged
user — outbound-only, jobs in containers on this box's own dockerd.
Its home is a ci-box tenant, where 'rig bootstrap ci-box' already
installed that daemon. Registers to an INSTANCE: whether the runner
serves that instance, one org or one repo is a property of the
registration token, so there is no --repo. Prompts for the token
(FORGEJO_RUNNER_TOKEN env overrides). Run as root.
forgejo-runner status [--user <name>]
Which instance this box's Forgejo runner is registered to: instance,
name, labels, unit. Reads the box only — no token, no network call,
and never prints the registration secret .runner holds. Run as root.
forgejo-runner remove [--user <name>]
Take the service down and wipe this box's registration. Always
local-only: Forgejo has no deregistration endpoint, so the runner
stays listed offline until you delete it in the instance. Needs no
token. Run as root.
users apply --file <path> users apply --file <path>
Converge named operator accounts from a declarative users file, on Converge named operator accounts from a declarative users file, on
every class: groups by role (admin/rig/box), passwords locked always, every class: groups by role (admin/rig/box), passwords locked always,
@ -116,12 +91,6 @@ commands:
Shut root SSH on a class=human box once an admin key works. Refuses Shut root SSH on a class=human box once an admin key works. Refuses
on class=server — root there is the control plane's automation door — on class=server — root there is the control plane's automation door —
and while no admin holds a key. Run as root. and while no admin holds a key. Run as root.
template-lint <role-dir>...
Validate tenant-role definitions (the heavy-duty/rig-templates
shape): template.env against the allowlist (data, never sourced),
install.sh present with a shebang, creds.md non-blank. Every refusal
names the failing key. The registry repo's CI runs this on every PR;
no root, no network, no writes.
manifest [<key>] manifest [<key>]
Print /etc/rig/manifest — which rig converged this machine and when Print /etc/rig/manifest — which rig converged this machine and when
(bootstrapped_by/_at pin the FIRST convergence forever; converged_by/_at (bootstrapped_by/_at pin the FIRST convergence forever; converged_by/_at
@ -152,12 +121,10 @@ commands:
install/upgrade: install/upgrade:
curl -fsSL https://raw.githubusercontent.com/heavy-duty/rig/main/install.sh | bash curl -fsSL https://raw.githubusercontent.com/heavy-duty/rig/main/install.sh | bash
Installs the latest RELEASE from GitHub (the default RIG_HOST). Installs the latest RELEASE (RIG_REF=<tag> pins one, RIG_REF=main
RIG_REF=<tag> pins one, RIG_REF=main tracks the development tree. tracks the development tree). Re-run any time: an installed version
From a Forgejo host: set RIG_HOST to that origin and fetch install.sh converges (no-op), a new one installs side by side at
from <host>/heavy-duty/rig/raw/branch/main/install.sh (#111). <root>/versions/<v> and becomes the default.
Re-run any time: an installed version converges (no-op), a new one
installs side by side at <root>/versions/<v> and becomes the default.
EOF EOF
} }
@ -462,37 +429,6 @@ case "$cmd" in
;; ;;
esac esac
;; ;;
forgejo-runner)
shift
sub="${1:-}"
case "$sub" in
install)
shift
exec "$ROOT/commands/forgejo-runner-install.sh" "$@"
;;
status)
shift
exec "$ROOT/commands/forgejo-runner-status.sh" "$@"
;;
remove)
shift
exec "$ROOT/commands/forgejo-runner-remove.sh" "$@"
;;
repoint)
# The GitHub sibling HAS this verb, so an operator will try it. Say why
# it cannot exist here rather than printing usage and leaving them to
# infer it: 'repoint' is atomic because GitHub lets rig deregister from
# the old repo; Forgejo has no such endpoint, so the honest shape is
# two acts, and one of them leaves a stale entry behind.
printf 'rig: forgejo-runner has no repoint: Forgejo has no deregistration endpoint, so a move cannot be one atomic act. Use "rig forgejo-runner remove" then "rig forgejo-runner install --instance <url>", and delete the stale runner in the old instance under Actions > Runners.\n' >&2
exit 2
;;
*)
usage >&2
exit 2
;;
esac
;;
users) users)
shift shift
sub="${1:-}" sub="${1:-}"
@ -515,10 +451,6 @@ case "$cmd" in
;; ;;
esac esac
;; ;;
template-lint)
shift
exec "$ROOT/commands/template-lint.sh" "$@"
;;
manifest) manifest)
shift shift
exec "$ROOT/commands/manifest.sh" "$@" exec "$ROOT/commands/manifest.sh" "$@"

4
changelog.d/109.md
View file

@ -1,4 +0,0 @@
### Added
- `rig forgejo-runner install|status|remove` registers a Forgejo Actions runner against an instance, jobs in containers on the box's own dockerd (#109)
- The template registry can be served from any forge — `RIG_TEMPLATES_HOST` (#109)

3
changelog.d/111.md
View file

@ -1,3 +0,0 @@
### Changed
- `install.sh` and bootstrap's box fetch take `RIG_HOST` / `BOX_HOST` so a Forgejo origin can serve the install channel (#111)

3
changelog.d/112.md
View file

@ -1,3 +0,0 @@
### Fixed
- rig's six ceremony CI guards resolve on Forgejo (#112)

3
changelog.d/116.md
View file

@ -1,3 +0,0 @@
### Fixed
- Ceremony recognizes the Forgejo review panel and both forges' triage actors (#116)

3
changelog.d/129.md
View file

@ -1,3 +0,0 @@
### Added
- The release drill exercises `rig forgejo-runner` beside `rig runner`, so both shipped runner families carry evidence (#129)

3
changelog.d/131.md
View file

@ -1,3 +0,0 @@
### Fixed
- The README's install quick start documents the Forgejo channel — `RIG_HOST` shipped in #111 but was documented nowhere (#131)

3
changelog.d/133.md
View file

@ -1,3 +0,0 @@
### Fixed
- `rig forgejo-runner status` no longer lets a service's `(active)` stand as proof the runner is fetching jobs (#133)

3
changelog.d/135.md
View file

@ -1,3 +0,0 @@
### Fixed
- Forgejo runners installed by rig can start their cache server — `$HOME/.cache` is created and punched through `ProtectHome` (#135)

4
changelog.d/139.md
View file

@ -1,4 +0,0 @@
### Fixed
- `rig forgejo-runner install`, `rig runner install`, `rig users apply` and `rig bootstrap <tenant>` refuse with a named remedy when root's `PATH` carries no `/usr/sbin`, instead of dying on `useradd: command not found` after prompting for a token (#139)
- `rig users apply` no longer reports success having silently skipped the sudoers drop-in when `visudo` is off `PATH` (#139)

8
changelog.d/144.md
View file

@ -1,8 +0,0 @@
### Added
- Default Forgejo runner labels include opt-in `ubuntu-latest-full` for the GitHub-parity image (#144)
### Fixed
- `ci.yml` installs `shellcheck` when the runner image lacks it, so Forgejo's slim `ubuntu-latest` can run `check` (#144)
- Plain `rig forgejo-runner install` warns when recorded labels are a retired rig default, without nagging custom `--labels` (#144)

3
changelog.d/152.md
View file

@ -1,3 +0,0 @@
### Added
- Machine-role templates can declare bootstrap traits and an optional final root install hook (#152)

3
changelog.d/153.md
View file

@ -1,3 +0,0 @@
### Added
- Pinned template registries install with rig and serve default converges offline (#153)

3
changelog.d/160.md
View file

@ -1,3 +0,0 @@
### Fixed
- The netmap tag read is scoped to `Self`: an untagged node next to tagged peers no longer reads a peer's tag, false-refusing `--join login` and false-verifying untagged authkey joins (#160)

3
changelog.d/162.md
View file

@ -1,3 +0,0 @@
### Fixed
- Agent tenant boxes ship cron — binary asserted, service enabled and active — so the duty engine can arm its timer (#162)

10
changelog.d/README.md
View file

@ -1,10 +0,0 @@
# changelog.d/ — the next release's section, one fragment per issue
Machine-assembled by `bin/changelog-assemble` (#112): every PR that changes
behavior writes one file here — `<issue>.md`, the exact prose that will be
published, nothing else — and the release PR folds them all into the next
`## X.Y.Z — DATE` section of `CHANGELOG.md`, consuming them. Distinct
filenames never conflict, which is this directory's whole reason to exist.
This README is the marker that keeps the directory tracked when it holds no
fragments (#112 D1) — `changelog-armed` refuses a tree without it; do not
delete it.

343
commands/bootstrap-tenant.sh
View file

@ -1,38 +1,26 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# rig bootstrap <role>-box — the box TENANT roles ('-box' names the family: a # rig bootstrap <claude-box|codex-box|grok-box|staging-box> — the box TENANT
# guest, vs the '-server' machine roles): what a box-minted guest becomes # roles ('-box' names the family: a guest, vs the '-server' machine roles): what a
# (issue #31). box mints the thin, creds-free seed (base image, user, rig # box-minted guest becomes (issue #31). box mints the thin, creds-free seed
# preinstalled — heavy-duty/box#81); rig converges the tenant content that # (base image, user, rig preinstalled — heavy-duty/box#81); rig converges the
# used to live in the templates' cloud-init, idempotent and effective-state # tenant content that used to live in the templates' cloud-init, idempotent and
# asserted, so an EXISTING box can be re-run to a new spec instead of # effective-state asserted, so an EXISTING box can be re-run to a new spec
# re-minted. # instead of re-minted. One mechanism, parameterized per tenant through
# # lib/tenant-config.sh — never four hand-maintained copies.
# One MECHANISM, parameterized per tenant by a fetched DEFINITION (#110): the
# agent-tenant registry lives in heavy-duty/rig-templates — one directory per
# role (template.env, install.sh, creds.md), resolved through lib/templates.sh
# (RIG_TEMPLATES_DIR > RIG_TEMPLATES_REF > the in-tree pin) — so adding a
# tenant is a data PR there, never an edit here (#109 is the scar: adding
# kimi, pure data, meant editing six files in this repo). staging-box is the
# one in-tree tenant: it is mechanism-adjacent (sshd hardening, docker — no
# agent, no CLI, no context file), so it converges from rig's own tree.
# #
# Creds-free BY CONTRACT: box auto-runs these at mint ('box exec … rig # Creds-free BY CONTRACT: box auto-runs these at mint ('box exec … rig
# bootstrap claude-box'), so every path here is non-interactive and nothing joins # bootstrap claude-box'), so every path here is non-interactive and nothing joins
# or admits — no tailnet, no keys, no prompts. That is also why the registry # or admits — no tailnet, no keys, no prompts. staging-box's tailnet join stays
# fetch is UNAUTHENTICATED: a mint holds nothing to authenticate with. # operator-run ('rig bootstrap workload-server' through 'box shell'), exactly the
# staging-box's tailnet join stays operator-run ('rig bootstrap # creds split box#69 designed.
# workload-server' through 'box shell'), exactly the creds split box#69
# designed.
# Convergent: safe to re-run; a second run changes nothing. # Convergent: safe to re-run; a second run changes nothing.
set -euo pipefail set -euo pipefail
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)" HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
# shellcheck source=SCRIPTDIR/lib/templates.sh # shellcheck source=SCRIPTDIR/lib/tenant-config.sh
. "$HERE/lib/templates.sh" # templates_resolve / template_parse_env / render_tenant_context . "$HERE/lib/tenant-config.sh" # tenant_user / tenant_context_path / render_tenant_context
# shellcheck source=SCRIPTDIR/lib/users-config.sh # shellcheck source=SCRIPTDIR/lib/users-config.sh
. "$HERE/lib/users-config.sh" # read_role_marker / root_door_of . "$HERE/lib/users-config.sh" # read_role_marker / root_door_of
# shellcheck source=SCRIPTDIR/lib/admin-path.sh
. "$HERE/lib/admin-path.sh" # require_admin_bins
# shellcheck source=SCRIPTDIR/lib/sshd.sh # shellcheck source=SCRIPTDIR/lib/sshd.sh
. "$HERE/lib/sshd.sh" # harden_sshd (the staging-box tenant) . "$HERE/lib/sshd.sh" # harden_sshd (the staging-box tenant)
# shellcheck source=SCRIPTDIR/lib/manifest.sh # shellcheck source=SCRIPTDIR/lib/manifest.sh
@ -44,35 +32,26 @@ die() { printf 'rig-bootstrap: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
usage() { usage() {
cat <<'EOF' cat <<'EOF'
usage: rig bootstrap <role>-box [--user <name>] usage: rig bootstrap <claude-box|codex-box|grok-box|staging-box> [--user <name>]
Box TENANT roles — what a box-minted guest becomes. box mints the thin, Box TENANT roles — what a box-minted guest becomes. box mints the thin,
creds-free seed (base image, user, rig preinstalled); this converges the creds-free seed (base image, user, rig preinstalled); this converges the
tenant on top, and re-runs converge an existing box to a new spec. tenant on top, and re-runs converge an existing box to a new spec.
<role>-box an agent tenant DEFINED IN THE REGISTRY claude-box|codex-box|grok-box
(heavy-duty/rig-templates — claude-box, codex-box, the agent tenants: base tooling (git, gh, tmux, …),
grok-box, kimi-box, …): base tooling (git, gh, tmux, …),
docker, the agent's CLI on the system PATH, and the docker, the agent's CLI on the system PATH, and the
agent-context file — including the box#80 guard: never agent-context file — including the box#80 guard: never
run `box setup-host` or the drill inside a box. run `box setup-host` or the drill inside a box.
staging-box the server tenant (box#69's posture), in rig's own staging-box the server tenant (box#69's posture): docker + sshd
tree: docker + sshd hardening. The tailnet workload hardening. The tailnet workload join is deliberately
join is deliberately NOT here — it holds a credential, NOT here — it holds a credential, so it stays
so it stays operator-run: `box shell` → `sudo rig operator-run: `box shell` → `sudo rig bootstrap
bootstrap workload-server` with a tagged pre-auth key. workload-server` with a tagged pre-auth key.
--user <name> the tenant user the box seed created (default: the --user <name> the tenant user the box seed created (default: the
definition's USER; staging-box defaults to `ops`) role's name minus the suffix; staging-box defaults to
`ops`)
The registry source is three knobs, precedence high to low:
RIG_TEMPLATES_DIR a local folder (no fetch — the offline/test path, and
"try a template before it exists anywhere")
RIG_TEMPLATES_REF a ref of RIG_TEMPLATES_REPO (default
heavy-duty/rig-templates), fetched as a tarball
(neither set) the ref pinned in rig's tree (lib/templates.sh
RIG_TEMPLATES_PIN — bumped by ordinary rig PR, so a
rig release freezes the mechanism+registry pair)
Tenant roles are creds-free and non-interactive by contract — box auto-runs Tenant roles are creds-free and non-interactive by contract — box auto-runs
them at mint (`box exec … rig bootstrap claude-box`). They take none of the them at mint (`box exec … rig bootstrap claude-box`). They take none of the
@ -84,35 +63,19 @@ EOF
# --- args (validated before the root check, so errors are testable) --------- # --- args (validated before the root check, so errors are testable) ---------
ROLE="${1:-}" ROLE="${1:-}"
case "$ROLE" in case "$ROLE" in
staging-box) shift ;; claude-box|codex-box|grok-box|staging-box) shift ;;
*-box)
# The family suffix is the whole gate here — WHICH '-box' roles exist is
# the resolved registry's fact, checked below, so a template added to the
# registry is mintable with zero code changes in rig (#110).
shift ;;
-h|--help) usage; exit 0 ;; -h|--help) usage; exit 0 ;;
"") usage >&2; die "tenant role required (a '-box' role from the template registry, or staging-box)" 2 ;; "") usage >&2; die "tenant role required (claude-box|codex-box|grok-box|staging-box)" 2 ;;
*) die "unknown tenant role: $ROLE — tenant roles carry the '-box' family suffix (#76); the machine roles are control-plane-server|workload-server|runner-server|staging-server|dev-server|workstation|custom" 2 ;; *) die "unknown tenant role: $ROLE (want claude-box|codex-box|grok-box|staging-box)" 2 ;;
esac esac
# The suffix rule above admits ANY '-box' name, so the charset is pinned
# before the name is ever used as a path component: a crafted role dies HERE,
# never in a registry lookup (the valid_version discipline, bin/rig).
[[ "$ROLE" =~ ^[a-z][a-z0-9-]*-box$ ]] \
|| die "invalid tenant role name: '$ROLE' — must match ^[a-z][a-z0-9-]*-box\$" 2
TENANT_USER_OVERRIDE="" TENANT_USER="$(tenant_user "$ROLE")"
while [ $# -gt 0 ]; do while [ $# -gt 0 ]; do
case "$1" in case "$1" in
-h|--help) usage; exit 0 ;; -h|--help) usage; exit 0 ;;
--user) --user)
[ $# -ge 2 ] || die "--user needs a value" 2 [ $# -ge 2 ] || die "--user needs a value" 2
TENANT_USER_OVERRIDE="$2"; shift 2 TENANT_USER="$2"; shift 2 ;;
# Same charset the users file enforces, for the same reasons (a leading
# '-' reads as a usermod flag; '|', ':' corrupt things downstream).
# Checked HERE, at parse — the definition's USER is checked by the
# parser — so the refusal needs no registry and no network.
[[ "$TENANT_USER_OVERRIDE" =~ ^[a-z_][a-z0-9_-]{0,31}$ ]] \
|| die "invalid user: '$TENANT_USER_OVERRIDE' — must match ^[a-z_][a-z0-9_-]{0,31}\$" 2 ;;
--hostname|--root-door|--host|--join) --hostname|--root-door|--host|--join)
# The machine-role traits, refused with a story rather than "unknown # The machine-role traits, refused with a story rather than "unknown
# flag": a tenant is a guest, not a tailnet machine — its shape comes # flag": a tenant is a guest, not a tailnet machine — its shape comes
@ -126,6 +89,10 @@ while [ $# -gt 0 ]; do
*) die "unknown flag: $1" 2 ;; *) die "unknown flag: $1" 2 ;;
esac esac
done done
# Same charset the users file enforces, for the same reasons (a leading '-'
# reads as a usermod flag; '|', ':' corrupt things downstream).
[[ "$TENANT_USER" =~ ^[a-z_][a-z0-9_-]{0,31}$ ]] \
|| die "invalid user: '$TENANT_USER' — must match ^[a-z_][a-z0-9_-]{0,31}\$" 2
# --- guards ------------------------------------------------------------------ # --- guards ------------------------------------------------------------------
# A tenant role converges a box GUEST. A box already carrying a machine-role # A tenant role converges a box GUEST. A box already carrying a machine-role
@ -173,44 +140,7 @@ if [ -n "$EXISTING_ROOT_DOOR" ]; then
fi fi
fi fi
# --- the definition ----------------------------------------------------------
# Resolved and parsed BEFORE the root check (but after the marker guards,
# which need no definition and must stay refusable with no registry in
# reach), so the two refusals a definition can earn — unknown role (listing
# what the resolved source actually contains) and malformed data (naming the
# failing key) — are testable non-root, offline, via RIG_TEMPLATES_DIR
# fixtures. The parse is the mint's
# own guard, deliberately duplicating the registry CI's lint: CI protects the
# registry, this protects a mint served through RIG_TEMPLATES_REPO/_DIR that
# CI never saw. template.env is parsed, NEVER sourced — a definition cannot
# execute arbitrary shell through its data file; install.sh is the one
# deliberately executable part, and it runs only after the root check below.
trap '[ -n "$TEMPLATES_TMP" ] && rm -rf "$TEMPLATES_TMP"' EXIT
TPL_DIR=""
if [ "$ROLE" = "staging-box" ]; then
TENANT_USER="${TENANT_USER_OVERRIDE:-ops}" # box#69's ops
else
templates_resolve \
|| die "cannot resolve the template registry ($(templates_source_desc)) — see above" 2
TPL_DIR="$REGISTRY_DIR/$ROLE"
if [ ! -f "$TPL_DIR/template.env" ]; then
die "unknown tenant role: $ROLE — the resolved registry ($(templates_source_desc)) defines: $(templates_roles "$REGISTRY_DIR" | tr '\n' ' ')— and staging-box is in rig's own tree. A misconfigured RIG_TEMPLATES_REPO/_REF/_DIR looks exactly like this; check the source before the spelling." 2
fi
template_parse_env "$TPL_DIR/template.env" \
|| die "invalid definition for $ROLE in $(templates_source_desc) — the failing key is named above. The registry's CI lints every PR ('rig template-lint'); a malformed definition reaching a mint means the source above was never linted." 2
TENANT_USER="${TENANT_USER_OVERRIDE:-$TPL_USER}"
fi
[ "$(id -u)" -eq 0 ] || die "must run as root" [ "$(id -u)" -eq 0 ] || die "must run as root"
# Root is not enough here either (#139). This mint adds the tenant user to the
# docker group with `usermod` far below — AFTER installing docker and node,
# which is what makes it the worst-placed of the four call sites: on a
# PATH-shorn root it dies mid-convergence with a bare `usermod: command not
# found`, having already changed the machine, rather than before touching it.
#
# Unconditional because the docker block below is unconditional — "every tenant
# gets docker" is the stated rule there, so every tenant reaches the usermod.
require_admin_bins usermod
if [ -r /etc/os-release ]; then if [ -r /etc/os-release ]; then
# Sourced in a subshell: os-release defines VERSION, NAME, ID, etc. — # Sourced in a subshell: os-release defines VERSION, NAME, ID, etc. —
# sourcing it in the main shell silently clobbers same-named script vars. # sourcing it in the main shell silently clobbers same-named script vars.
@ -244,46 +174,23 @@ append_line_once() {
chown "$TENANT_USER:$TENANT_GROUP" "$file" chown "$TENANT_USER:$TENANT_GROUP" "$file"
} }
# The binary on PATH is not the effective state — an image can ship crontab
# with cron.service masked or stopped, and an unarmed timer is exactly the
# silent-inert box #162 is about. Converge best-effort, then assert what
# systemd actually reports; the assert is the authority. Enabling an
# already-enabled unit is a no-op and no path here touches any crontab.
# A function so test/cli.sh can lift it verbatim and drive it against a
# stubbed systemctl, the drop_incus precedent.
converge_cron() {
if ! systemctl is-enabled cron >/dev/null 2>&1; then
log "cron.service not enabled — converging"
systemctl unmask cron >/dev/null 2>&1 || true
systemctl enable cron >/dev/null 2>&1 || true
fi
if ! systemctl is-active cron >/dev/null 2>&1; then
log "cron.service not active — starting"
systemctl start cron >/dev/null 2>&1 || true
fi
systemctl is-enabled cron >/dev/null 2>&1 || die "cron.service is not enabled after converge — the duty engine's timer never fires without it (#162)"
systemctl is-active cron >/dev/null 2>&1 || die "cron.service is not active after converge — the duty engine's timer never fires without it (#162)"
}
# --- packages ---------------------------------------------------------------- # --- packages ----------------------------------------------------------------
export DEBIAN_FRONTEND=noninteractive export DEBIAN_FRONTEND=noninteractive
log "installing base packages (tenant ${ROLE})" log "installing base packages (tenant ${ROLE})"
apt-get update -qq apt-get update -qq
if [ "$ROLE" = "staging-box" ]; then case "$ROLE" in
# openssh-server: the hardening drop-in below targets /etc/ssh/sshd_config.d/, claude-box)
# which only exists once the package is installed — pristine container/VM # The claude-box tenant keeps zsh (its shell UX ships with the box); the
# images (and thin seeds) do not ship it. # remaining list is the shared agent toolbelt the templates carried.
apt-get install -y -qq curl ca-certificates tmux openssh-server apt-get install -y -qq git gh curl ca-certificates gnupg ripgrep jq tmux age unzip build-essential zsh ;;
else codex-box|grok-box)
# The shared agent toolbelt the templates carried, plus the definition's apt-get install -y -qq git gh curl ca-certificates gnupg ripgrep jq tmux age unzip build-essential ;;
# APT_EXTRAS (claude-box's zsh rides there). Unquoted on purpose — it is a staging-box)
# word list, every word already vetted by the parser's package-name gate. # openssh-server: the hardening drop-in below targets /etc/ssh/sshd_config.d/,
# cron is toolbelt, not a template flavour: every agent tenant exists to # which only exists once the package is installed — pristine container/VM
# run the cron-driven duty engine, whose unprivileged installer can detect # images (and thin seeds) do not ship it.
# a missing cron but never apt-get it (#162). apt-get install -y -qq curl ca-certificates tmux openssh-server ;;
# shellcheck disable=SC2086 esac
apt-get install -y -qq git gh curl ca-certificates gnupg ripgrep jq tmux age unzip build-essential cron $TPL_APT_EXTRAS
fi
# Assert the effective toolbelt, not apt's exit code — tmux is the box#65 # Assert the effective toolbelt, not apt's exit code — tmux is the box#65
# contract ('box tmux' runs tmux new-session inside every box) and gh is how # contract ('box tmux' runs tmux new-session inside every box) and gh is how
# the operator's git credential lands. # the operator's git credential lands.
@ -291,9 +198,6 @@ command -v tmux >/dev/null 2>&1 || die "tmux missing after package install — '
if [ "$ROLE" != "staging-box" ]; then if [ "$ROLE" != "staging-box" ]; then
command -v gh >/dev/null 2>&1 || die "gh missing after package install" command -v gh >/dev/null 2>&1 || die "gh missing after package install"
command -v git >/dev/null 2>&1 || die "git missing after package install" command -v git >/dev/null 2>&1 || die "git missing after package install"
command -v crontab >/dev/null 2>&1 || die "crontab missing after package install — the duty engine arms itself with cron (#162)"
# staging-box is exempt with the rest of this block: no agent, no engine.
converge_cron
fi fi
# --- docker ------------------------------------------------------------------ # --- docker ------------------------------------------------------------------
@ -328,19 +232,17 @@ else
warn "no docker group after install — skipping the ${TENANT_USER} group add; check docker's install" warn "no docker group after install — skipping the ${TENANT_USER} group add; check docker's install"
fi fi
# --- node (definitions carrying NEEDS_NODE="yes") ---------------------------- # --- node (claude-box, codex-box) ----------------------------------------------------
# An npm-installed CLI needs Node 22+ (codex — the SCOPED @openai/codex, # Codex is an npm global needing Node 22+ (the SCOPED @openai/codex — verified
# verified upstream when the template was written); claude ships node as part # upstream when the template was written); the claude-box tenant ships node as part
# of its toolbelt, same pin. Whether a tenant needs it is the DEFINITION's # of its toolbelt, same pin. grok's CLI is a self-contained binary: no node.
# fact (NEEDS_NODE), never a role list here — grok's CLI is a self-contained
# binary and kimi's is uv-managed Python, so both say no.
node_ok() { node_ok() {
command -v node >/dev/null 2>&1 || return 1 command -v node >/dev/null 2>&1 || return 1
local major local major
major="$(node --version 2>/dev/null | sed -E 's/^v([0-9]+)\..*$/\1/')" major="$(node --version 2>/dev/null | sed -E 's/^v([0-9]+)\..*$/\1/')"
[ "${major:-0}" -ge 22 ] 2>/dev/null [ "${major:-0}" -ge 22 ] 2>/dev/null
} }
if [ "$ROLE" != "staging-box" ] && [ "$TPL_NEEDS_NODE" = "yes" ]; then if [ "$ROLE" = "claude-box" ] || [ "$ROLE" = "codex-box" ]; then
if node_ok; then if node_ok; then
log "node $(node --version) already present" log "node $(node --version) already present"
else else
@ -352,53 +254,46 @@ if [ "$ROLE" != "staging-box" ] && [ "$TPL_NEEDS_NODE" = "yes" ]; then
fi fi
# --- the agent CLI ----------------------------------------------------------- # --- the agent CLI -----------------------------------------------------------
# Per-definition install, shared discipline: install only when the CLI is # Per-agent install, shared discipline: install only when the CLI is absent
# absent (upgrades are the CLI's own business) — presence is CLI_SRC when the # (upgrades are the CLI's own business), then put it on the SYSTEM path —
# definition names one, `command -v` when it does not (an npm global's path # 'box exec <box> -- <cli> …' runs a NON-interactive shell that reads no rc
# is the prefix's fact, not the data file's) — then put it on the SYSTEM # files, so a PATH export alone is invisible to it (the #15 lesson) — and
# path: 'box exec <box> -- <cli> …' runs a NON-interactive shell that reads # assert it ANSWERS as the tenant user: a CLI that exists but cannot run is
# no rc files, so a PATH export alone is invisible to it (the #15 lesson). # what cost the last drill (the grok-box template's scar).
# And assert it ANSWERS as the tenant user: a CLI that exists but cannot run
# is what cost the last drill (the grok-box template's scar).
#
# install.sh — the definition's one executable part — runs AS ROOT with the
# tenant named in its environment (TENANT_USER/TENANT_HOME/TENANT_GROUP/ROLE);
# each definition drops to the tenant user itself (runuser -l) where the
# vendor's layout demands it, because some installs are inherently root's
# (codex's npm -g writes the global prefix). This is the trade #110 states in
# bold — a registry definition executes as root inside every future mint —
# and it is why install.sh diffs there are the highest-trust review surface
# in the org, why the default ref is a reviewed in-tree pin, and why the data
# file beside it is parsed rather than sourced.
CLI="" CLI_SRC="" CLI="" CLI_SRC=""
if [ "$ROLE" != "staging-box" ]; then case "$ROLE" in
CLI="$TPL_CLI_NAME" claude-box)
# '~/' in CLI_SRC is data — expanded to the tenant home HERE, by string CLI=claude CLI_SRC="$TENANT_HOME/.local/bin/claude"
# substitution, never by the shell (hence the literal quoted tilde, SC2088). if [ ! -e "$CLI_SRC" ]; then
# shellcheck disable=SC2088 log "installing the Claude Code CLI as ${TENANT_USER}"
case "$TPL_CLI_SRC" in runuser -l "$TENANT_USER" -c 'curl -fsSL https://claude.ai/install.sh | bash'
'~/'*) CLI_SRC="$TENANT_HOME/${TPL_CLI_SRC#'~/'}" ;; else
*) CLI_SRC="$TPL_CLI_SRC" ;; log "claude CLI already installed"
esac fi ;;
installed="" codex-box)
if [ -n "$CLI_SRC" ]; then CLI=codex
[ -e "$CLI_SRC" ] && installed=1 if ! command -v codex >/dev/null 2>&1; then
elif command -v "$CLI" >/dev/null 2>&1; then log "installing the Codex CLI (npm global)"
installed=1 npm install -g @openai/codex
fi else
if [ -z "$installed" ]; then log "codex CLI already installed"
log "installing the ${CLI} CLI (${ROLE}'s install.sh)" fi
TENANT_USER="$TENANT_USER" TENANT_HOME="$TENANT_HOME" \ CLI_SRC="$(npm prefix -g)/bin/codex" ;;
TENANT_GROUP="$TENANT_GROUP" ROLE="$ROLE" \ grok-box)
bash "$TPL_DIR/install.sh" \ # The OFFICIAL installer (x.ai/cli/install.sh): installs the CLI as `grok`,
|| die "${ROLE}'s install.sh failed — the definition is $(templates_source_desc)" # a SYMLINK under $HOME/.grok/bin pointing into its versioned download dir.
else # Run it AS the tenant user, never root: a symlink into root's 0700 home
log "${CLI} CLI already installed" # would be a CLI that exists and cannot run.
fi CLI=grok CLI_SRC="$TENANT_HOME/.grok/bin/grok"
if [ -z "$CLI_SRC" ]; then if [ ! -e "$CLI_SRC" ]; then
CLI_SRC="$(command -v "$CLI" 2>/dev/null || true)" log "installing the Grok CLI as ${TENANT_USER}"
[ -n "$CLI_SRC" ] || die "the ${CLI} installer put no '${CLI}' on root's PATH and the definition names no CLI_SRC — upstream layout changed?" runuser -l "$TENANT_USER" -c 'curl -fsSL https://x.ai/cli/install.sh | bash'
fi else
log "grok CLI already installed"
fi ;;
staging-box) ;; # no agent lives on the staging-box tenant
esac
if [ -n "$CLI" ]; then
[ -e "$CLI_SRC" ] || die "the ${CLI} installer produced no ${CLI_SRC} — upstream layout changed?" [ -e "$CLI_SRC" ] || die "the ${CLI} installer produced no ${CLI_SRC} — upstream layout changed?"
ln -sf "$CLI_SRC" "/usr/local/bin/$CLI" ln -sf "$CLI_SRC" "/usr/local/bin/$CLI"
# One capture serves both the assert and the log line; emptiness IS the # One capture serves both the assert and the log line; emptiness IS the
@ -407,23 +302,27 @@ if [ "$ROLE" != "staging-box" ]; then
[ -n "$CLI_VER" ] || die "'$CLI --version' does not answer for ${TENANT_USER} — the CLI landed but cannot run; check /usr/local/bin/$CLI and its target" [ -n "$CLI_VER" ] || die "'$CLI --version' does not answer for ${TENANT_USER} — the CLI landed but cannot run; check /usr/local/bin/$CLI and its target"
log "${CLI} CLI on the system PATH and answering (${CLI_VER})" log "${CLI} CLI on the system PATH and answering (${CLI_VER})"
# The interactive-shell PATH export the templates carried, converged as a # The interactive-shell PATH exports the templates carried, converged as
# literal rc line (written once, never duplicated). The definition's # literal rc lines (written once, never duplicated). Single quotes are the
# PATH_LINE is DATA, appended verbatim: it must expand in the USER's # point: the line must expand in the USER's shell, not here.
# shell, not here. # shellcheck disable=SC2016
append_line_once "$TENANT_HOME/.bashrc" "$TPL_PATH_LINE" case "$ROLE" in
claude-box)
append_line_once "$TENANT_HOME/.bashrc" 'export PATH="$HOME/.local/bin:$PATH"' ;;
codex-box)
append_line_once "$TENANT_HOME/.bashrc" 'export PATH="$(npm prefix -g)/bin:$PATH"' ;;
grok-box)
append_line_once "$TENANT_HOME/.bashrc" 'export PATH="$HOME/.grok/bin:$PATH"' ;;
esac
fi fi
# --- the agent-context file -------------------------------------------------- # --- the agent-context file --------------------------------------------------
# The one file every agent reads before touching anything. The skeleton — # The one file every agent reads before touching anything. Rendered from
# including the box#80 guard note ("never run box setup-host or the drill # lib/tenant-config.sh — the box#80 guard note ("never run box setup-host or
# inside a box; the box you are in is not a host you own") — is MECHANISM, # the drill inside a box; the box you are in is not a host you own") lives
# rendered from lib/templates.sh ONCE for all agents, never copy-pasted per # there ONCE, for all agents, instead of copy-pasted per template. cmp-guarded
# template; only the creds paragraph is the definition's (creds.md). # like every file rig converges.
# cmp-guarded like every file rig converges. staging-box has no agent and no if CTX_PATH="$(tenant_context_path "$ROLE" "$TENANT_HOME")"; then
# context file.
if [ "$ROLE" != "staging-box" ]; then
CTX_PATH="$TENANT_HOME/$TPL_CONTEXT_PATH"
CTX_DIR="$(dirname "$CTX_PATH")" CTX_DIR="$(dirname "$CTX_PATH")"
if [ ! -d "$CTX_DIR" ]; then if [ ! -d "$CTX_DIR" ]; then
mkdir -p "$CTX_DIR" mkdir -p "$CTX_DIR"
@ -433,7 +332,7 @@ if [ "$ROLE" != "staging-box" ]; then
# its ownership is converged on every run, not only on creation. # its ownership is converged on every run, not only on creation.
chown "$TENANT_USER:$TENANT_GROUP" "$CTX_DIR" chown "$TENANT_USER:$TENANT_GROUP" "$CTX_DIR"
CTX_TMP="$(mktemp)" CTX_TMP="$(mktemp)"
render_tenant_context "$ROLE" "$TPL_DIR/creds.md" > "$CTX_TMP" render_tenant_context "$ROLE" > "$CTX_TMP"
if ! cmp -s "$CTX_TMP" "$CTX_PATH" 2>/dev/null; then if ! cmp -s "$CTX_TMP" "$CTX_PATH" 2>/dev/null; then
install -m 0644 -o "$TENANT_USER" -g "$TENANT_GROUP" "$CTX_TMP" "$CTX_PATH" install -m 0644 -o "$TENANT_USER" -g "$TENANT_GROUP" "$CTX_TMP" "$CTX_PATH"
log "agent-context file written: ${CTX_PATH}" log "agent-context file written: ${CTX_PATH}"
@ -443,6 +342,32 @@ if [ "$ROLE" != "staging-box" ]; then
rm -f "$CTX_TMP" rm -f "$CTX_TMP"
fi fi
# --- claude-box shell niceties ---------------------------------------------------
# The claude-box template shipped zsh + oh-my-zsh + tmux mouse mode; they move with
# the tenant. oh-my-zsh is a cosmetic EXTRA: its failure warns, never aborts a
# bootstrap whose real work (CLI, context, docker) already converged.
if [ "$ROLE" = "claude-box" ]; then
if [ "$(getent passwd "$TENANT_USER" | cut -d: -f7)" != "/usr/bin/zsh" ]; then
chsh -s /usr/bin/zsh "$TENANT_USER"
log "login shell set to zsh for ${TENANT_USER}"
else
log "login shell already zsh for ${TENANT_USER}"
fi
if [ ! -d "$TENANT_HOME/.oh-my-zsh" ]; then
log "installing oh-my-zsh for ${TENANT_USER}"
# Single quotes on purpose: the $(...) must run in the USER's shell.
# shellcheck disable=SC2016
runuser -l "$TENANT_USER" -c 'RUNZSH=no CHSH=no sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"' \
|| warn "oh-my-zsh install failed — cosmetic only; continuing"
else
log "oh-my-zsh already installed"
fi
# After oh-my-zsh (it rewrites .zshrc on first install).
# shellcheck disable=SC2016
append_line_once "$TENANT_HOME/.zshrc" 'export PATH="$HOME/.local/bin:$PATH"'
append_line_once "$TENANT_HOME/.tmux.conf" 'set -g mouse on'
fi
# --- staging-box server posture -------------------------------------------------- # --- staging-box server posture --------------------------------------------------
# box#69's posture, minus the join: docker (above) + sshd hardening, through # box#69's posture, minus the join: docker (above) + sshd hardening, through
# the SAME code the machine roles use (lib/sshd.sh) — the staging-box guest is a # the SAME code the machine roles use (lib/sshd.sh) — the staging-box guest is a

74
commands/bootstrap-undo.sh
View file

@ -1,74 +0,0 @@
#!/usr/bin/env bash
# rig bootstrap --undo — remove only off-box state rig can prove it created.
set -euo pipefail
log() { printf 'rig-bootstrap: %s\n' "$*"; }
die() { printf 'rig-bootstrap: ERROR: %s\n' "$*" >&2; exit 1; }
MARKER="${RIG_ROLE_MARKER:-/etc/rig/role}"
[ "$(id -u)" -eq 0 ] || die "must run as root"
[ -e "$MARKER" ] || die "no /etc/rig/role marker — refusing to touch the tailnet"
runner_installed=0
if [ -n "${RIG_RUNNER_DIR:-}" ]; then
[ -e "$RIG_RUNNER_DIR/.runner" ] && runner_installed=1
else
for runner_config in /home/*/actions-runner/.runner /root/actions-runner/.runner; do
[ -e "$runner_config" ] && runner_installed=1
done
compgen -G '/etc/systemd/system/actions.runner.*.service' >/dev/null \
&& runner_installed=1
fi
if [ "$runner_installed" -eq 1 ]; then
die "a GitHub runner is installed — run 'rig runner remove' first so undo does not leave a ghost runner in the repository"
fi
# The same hazard, the other forge (#109): leaving the tailnet under a live
# Forgejo runner strands a registration this box can no longer serve, and
# Forgejo has no deregistration endpoint — so the ghost it leaves is one
# somebody has to delete BY HAND in the instance's admin UI. That makes the
# refusal more load-bearing here than for GitHub, not less.
#
# RIG_FORGEJO_RUNNER_DIR mirrors RIG_RUNNER_DIR above so tests can point this
# at a fixture. The glob covers the tenant default (`ci`) and the dedicated
# account alike, because both are reachable defaults of `install --user`.
forgejo_runner_installed=0
if [ -n "${RIG_FORGEJO_RUNNER_DIR:-}" ]; then
[ -e "$RIG_FORGEJO_RUNNER_DIR/.runner" ] && forgejo_runner_installed=1
else
for runner_config in /home/*/forgejo-runner/.runner /root/forgejo-runner/.runner; do
[ -e "$runner_config" ] && forgejo_runner_installed=1
done
[ -e /etc/systemd/system/forgejo-runner.service ] && forgejo_runner_installed=1
fi
if [ "$forgejo_runner_installed" -eq 1 ]; then
die "a Forgejo runner is installed — run 'rig forgejo-runner remove' first so undo does not leave a ghost runner in the instance"
fi
join_by=""
while IFS= read -r field; do
case "$field" in
join-by=*) join_by="${field#join-by=}" ;;
esac
done < <(tr '[:space:]' '\n' < "$MARKER")
case "$join_by" in
rig) ;;
preexisting)
die "the tailnet join predates this bootstrap run (join-by=preexisting), so rig will not remove state it did not create; run 'tailscale logout' by hand if that is intended" ;;
"")
die "the role marker predates join-by provenance, so rig cannot prove it made this tailnet join and will not remove it; re-run bootstrap to write a current marker, or run 'tailscale logout' by hand" ;;
*)
die "the role marker has unknown join-by=$join_by, so rig cannot prove it made this tailnet join and will not remove it; run 'tailscale logout' by hand if that is intended" ;;
esac
# The same back-out/keep law as first-join verification: logout is earned only
# when the marker proves rig performed the join. Preserve the marker on failure
# so the operation remains retryable and never reports a half-undone machine.
if ! tailscale logout; then
die "tailscale logout failed; role marker kept so 'rig bootstrap --undo' can be retried"
fi
rm -f -- "$MARKER"
log "tailnet join removed; role marker removed"

189
commands/bootstrap.sh
View file

@ -12,8 +12,6 @@ HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
. "$HERE/lib/users-config.sh" # parse_users_file — the --users PRE-FLIGHT only . "$HERE/lib/users-config.sh" # parse_users_file — the --users PRE-FLIGHT only
# shellcheck source=SCRIPTDIR/lib/manifest.sh # shellcheck source=SCRIPTDIR/lib/manifest.sh
. "$HERE/lib/manifest.sh" # manifest_stamp — provenance, written beside the marker . "$HERE/lib/manifest.sh" # manifest_stamp — provenance, written beside the marker
# shellcheck source=SCRIPTDIR/lib/templates.sh
. "$HERE/lib/templates.sh" # registry-backed machine-role definitions
# The users lib is sourced for validation, never for convergence: `users apply` # The users lib is sourced for validation, never for convergence: `users apply`
# stays the single owner of what a users file DOES to a box (#51). Bootstrap # stays the single owner of what a users file DOES to a box (#51). Bootstrap
# borrows the parser so a typo'd users file is caught in the same breath as a # borrows the parser so a typo'd users file is caught in the same breath as a
@ -31,14 +29,9 @@ usage: rig bootstrap <control-plane-server|workload-server|runner-server|
(--users <path> | --no-users) (--users <path> | --no-users)
[--hostname <name>] [--root-door <closed|open>] [--hostname <name>] [--root-door <closed|open>]
[--host <yes|no>] [--join <authkey|login>] [--host <yes|no>] [--join <authkey|login>]
rig bootstrap <role>-box [--user <name>] rig bootstrap <claude-box|codex-box|grok-box|staging-box> [--user <name>]
(the box TENANT roles — the agent tenants come from the (the box TENANT roles — see their own --help; they take
heavy-duty/rig-templates registry, staging-box from no --users, see below)
rig's own tree; see their own --help — they take no
--users, see below)
rig bootstrap --undo
leave the tailnet only when the role marker proves rig
performed the join, then remove the role marker
--users the users file this box's operators come from — REQUIRED. It is --users the users file this box's operators come from — REQUIRED. It is
applied as bootstrap's last phase, exactly as `rig users apply applied as bootstrap's last phase, exactly as `rig users apply
@ -63,8 +56,7 @@ and per-human accounts keep attribution intact for the times someone does go
in. So the complete path is the default path and skipping it is a deliberate in. So the complete path is the default path and skipping it is a deliberate
--no-users, not an omission. --no-users, not an omission.
--users does NOT reach the box TENANT roles (any '-box' name, e.g. --users does NOT reach the box TENANT roles (claude-box|codex-box|grok-box|staging-box). A
claude-box, staging-box). A
tenant is a box-minted GUEST: box auto-runs its bootstrap at mint, tenant is a box-minted GUEST: box auto-runs its bootstrap at mint,
non-interactively, with no file to hand it; the guest never joins the tailnet non-interactively, with no file to hand it; the guest never joins the tailnet
and has no SSH door of its own — entry is `box shell`, gated by the HOST's and has no SSH door of its own — entry is `box shell`, gated by the HOST's
@ -121,38 +113,17 @@ EOF
# --- args (validated before the root check, so errors are testable) --------- # --- args (validated before the root check, so errors are testable) ---------
ROLE="${1:-}" ROLE="${1:-}"
MACHINE_TEMPLATE_DIR=""
case "$ROLE" in case "$ROLE" in
--undo)
shift
[ $# -eq 0 ] || die "bootstrap --undo takes no arguments" 2
exec "$HERE/bootstrap-undo.sh" ;;
control-plane-server|workload-server|runner-server|staging-server|dev-server|workstation|custom) shift ;; control-plane-server|workload-server|runner-server|staging-server|dev-server|workstation|custom) shift ;;
*-box) claude-box|codex-box|grok-box|staging-box)
# The box TENANT roles (#31) are a different family — guests a box mints, # The box TENANT roles (#31) are a different family — guests a box mints,
# never tailnet machines — and live in their own mechanism, one script # never tailnet machines — and live in their own mechanism, one script
# parameterized per DEFINITION fetched from the template registry (#110; # parameterized per tenant. Dispatched here so `rig bootstrap <role>`
# staging-box stays in-tree). Dispatched on the FAMILY SUFFIX (#76), not # stays the single entrypoint for both families.
# an enumerated list: which '-box' roles exist is the registry's fact, so
# a template added there is mintable with zero code changes here.
# `rig bootstrap <role>` stays the single entrypoint for both families.
exec "$HERE/bootstrap-tenant.sh" "$@" ;; exec "$HERE/bootstrap-tenant.sh" "$@" ;;
-h|--help) usage; exit 0 ;; -h|--help) usage; exit 0 ;;
"") usage >&2; die "role required (control-plane-server|workload-server|runner-server|staging-server|dev-server|workstation|custom — or a '-box' tenant role from the template registry, e.g. claude-box)" 2 ;; "") usage >&2; die "role required (control-plane-server|workload-server|runner-server|staging-server|dev-server|workstation|custom — or a tenant role: claude-box|codex-box|grok-box|staging-box)" 2 ;;
*) *) die "unknown role: $ROLE (want control-plane-server|workload-server|runner-server|staging-server|dev-server|workstation|custom — or a tenant role: claude-box|codex-box|grok-box|staging-box)" 2 ;;
shift
templates_resolve || exit 2
trap '[ -n "$TEMPLATES_TMP" ] && rm -rf "$TEMPLATES_TMP"' EXIT
MACHINE_TEMPLATE_DIR="$REGISTRY_DIR/$ROLE"
if [[ ! "$ROLE" =~ ^[a-z][a-z0-9-]*-server$ ]] \
|| [ "$(template_family "$ROLE" 2>/dev/null || true)" != "machine" ] \
|| [ ! -f "$MACHINE_TEMPLATE_DIR/template.env" ]; then
MACHINE_ROLES="$(templates_machine_roles "$REGISTRY_DIR" | paste -sd'|' -)"
[ -n "$MACHINE_ROLES" ] || MACHINE_ROLES="none"
die "unknown role: $ROLE (want control-plane-server|workload-server|runner-server|staging-server|dev-server|workstation|custom; machine roles from $(templates_source_desc): $MACHINE_ROLES; or a '-box' tenant role)" 2
fi
machine_template_parse_env "$MACHINE_TEMPLATE_DIR/template.env" \
|| die "invalid machine role $ROLE from $(templates_source_desc)" 2 ;;
esac esac
# Role→traits map — the single place a role's shape is declared (issue #26). # Role→traits map — the single place a role's shape is declared (issue #26).
@ -172,7 +143,6 @@ case "$ROLE" in
dev-server) ROOT_DOOR=closed HOST=yes JOIN=authkey ;; dev-server) ROOT_DOOR=closed HOST=yes JOIN=authkey ;;
workstation) ROOT_DOOR=closed HOST=yes JOIN=login ;; workstation) ROOT_DOOR=closed HOST=yes JOIN=login ;;
custom) ;; custom) ;;
*) ROOT_DOOR="$TPL_ROOT_DOOR" HOST="$TPL_HOST" JOIN="$TPL_JOIN" ;;
esac esac
# custom has no hostname default: a made-up name on a made-up shape helps nobody. # custom has no hostname default: a made-up name on a made-up shape helps nobody.
@ -564,7 +534,6 @@ if ! command -v tailscale >/dev/null 2>&1; then
log "installing tailscale" log "installing tailscale"
curl -fsSL https://tailscale.com/install.sh | sh curl -fsSL https://tailscale.com/install.sh | sh
fi fi
JOIN_BY=preexisting
if tailscale status >/dev/null 2>&1; then if tailscale status >/dev/null 2>&1; then
log "tailnet already joined; skipping tailscale up (no pre-auth key needed)" log "tailnet already joined; skipping tailscale up (no pre-auth key needed)"
# ...but skipping `tailscale up` also skipped --hostname, so the TAILNET name # ...but skipping `tailscale up` also skipped --hostname, so the TAILNET name
@ -605,7 +574,6 @@ elif [ "$JOIN" = "login" ]; then
log "joining tailnet as ${TS_HOSTNAME} (interactive login; follow the URL tailscale prints)" log "joining tailnet as ${TS_HOSTNAME} (interactive login; follow the URL tailscale prints)"
tailscale up --hostname="$TS_HOSTNAME" tailscale up --hostname="$TS_HOSTNAME"
verify_user_owned back-out verify_user_owned back-out
JOIN_BY=rig
else else
# env override, else prompt; never touches disk. The prompt only fires on a # env override, else prompt; never touches disk. The prompt only fires on a
# tty: with no terminal, a bare `read` exits non-zero and `set -e` would end # tty: with no terminal, a bare `read` exits non-zero and `set -e` would end
@ -626,7 +594,6 @@ else
log "joining tailnet as ${TS_HOSTNAME} (tag comes from the pre-auth key)" log "joining tailnet as ${TS_HOSTNAME} (tag comes from the pre-auth key)"
tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME" tailscale up --authkey="$TS_AUTHKEY" --hostname="$TS_HOSTNAME"
verify_effective_tag back-out verify_effective_tag back-out
JOIN_BY=rig
fi fi
# --- role marker -------------------------------------------------------------- # --- role marker --------------------------------------------------------------
@ -644,12 +611,11 @@ fi
# those exist in the field by the thousand and nothing will rewrite them. # those exist in the field by the thousand and nothing will rewrite them.
MARKER=/etc/rig/role MARKER=/etc/rig/role
MARKER_TMP="$(mktemp)" MARKER_TMP="$(mktemp)"
printf 'role=%s root-door=%s host=%s join=%s join-by=%s\n' \ printf 'role=%s root-door=%s host=%s join=%s\n' "$ROLE" "$ROOT_DOOR" "$HOST" "$JOIN" > "$MARKER_TMP"
"$ROLE" "$ROOT_DOOR" "$HOST" "$JOIN" "$JOIN_BY" > "$MARKER_TMP"
if ! cmp -s "$MARKER_TMP" "$MARKER" 2>/dev/null; then if ! cmp -s "$MARKER_TMP" "$MARKER" 2>/dev/null; then
mkdir -p /etc/rig mkdir -p /etc/rig
install -m 0644 "$MARKER_TMP" "$MARKER" install -m 0644 "$MARKER_TMP" "$MARKER"
log "role marker written: role=$ROLE root-door=$ROOT_DOOR host=$HOST join=$JOIN join-by=$JOIN_BY" log "role marker written: role=$ROLE root-door=$ROOT_DOOR host=$HOST join=$JOIN"
else else
log "role marker already current" log "role marker already current"
fi fi
@ -705,9 +671,7 @@ fi
# hardening + the tailnet, and box is the host EXTRA, so a failed box install # hardening + the tailnet, and box is the host EXTRA, so a failed box install
# must never abort a bootstrap that otherwise fully succeeded. # must never abort a bootstrap that otherwise fully succeeded.
# #
# PIN POINTS: BOX_REPO / BOX_REF override the source (default # PIN POINTS: BOX_REPO / BOX_REF override the source (default heavy-duty/box@main).
# heavy-duty/box@0.9.0). BOX_RELEASE is bumped deliberately when rig releases,
# after the pinned combination has passed the release drill.
# BOX_YES=1 makes box's installer non-interactive AND keeps setup-host (so the # BOX_YES=1 makes box's installer non-interactive AND keeps setup-host (so the
# Incus stack is actually built, not just the CLI dropped on PATH). # Incus stack is actually built, not just the CLI dropped on PATH).
# #
@ -720,114 +684,24 @@ fi
# on box PR #71. Until that merges, box's root install lands in /root and non-root # on box PR #71. Until that merges, box's root install lands in /root and non-root
# users cannot reach it, so this step is only fully correct once box#71 is merged. # users cannot reach it, so this step is only fully correct once box#71 is merged.
if [ "$HOST" = "yes" ]; then if [ "$HOST" = "yes" ]; then
BOX_RELEASE=0.9.0
BOX_REPO="${BOX_REPO:-heavy-duty/box}" BOX_REPO="${BOX_REPO:-heavy-duty/box}"
BOX_REF="${BOX_REF:-$BOX_RELEASE}" BOX_REF="${BOX_REF:-main}"
# BOX_HOST: which forge serves box's *installer script* (#111). Parallel to BOX_INSTALL_URL="https://raw.githubusercontent.com/${BOX_REPO}/${BOX_REF}/install.sh"
# RIG_HOST / RIG_TEMPLATES_HOST. Defaults to RIG_HOST when set, else GitHub, BOX_MANUAL="curl -fsSL ${BOX_INSTALL_URL} | BOX_YES=1 bash"
# so a Forgejo-sourced rig stays Forgejo-native for this fetch without a
# second knob — override with BOX_HOST when the two must diverge.
# Raw-file grammar:
# GitHub raw.githubusercontent.com/<repo>/<ref>/install.sh
# Forgejo <host>/<repo>/raw/{tag|branch}/<ref>/install.sh
# Forgejo's bare /raw/<ref>/ is branch-first (opposite of /archive/<ref>),
# so we never guess kind from spelling: try /raw/tag/ then /raw/branch/
# and let the fetch decide (same pin-wins rule as ref_candidate_urls).
# SCOPE: this only moves the script fetch. box@0.9.0's installer still
# hardcodes GitHub for its own archive — zero-GitHub bootstrap needs a
# BOX_HOST knob in heavy-duty/box (tracked separately).
BOX_HOST="${BOX_HOST:-${RIG_HOST:-https://github.com}}"
BOX_HOST="${BOX_HOST%/}"
box_install_urls() {
case "$BOX_HOST" in
https://github.com|http://github.com|*//github.com)
printf 'https://raw.githubusercontent.com/%s/%s/install.sh\n' "$BOX_REPO" "$BOX_REF" ;;
*)
printf '%s/%s/raw/tag/%s/install.sh\n' "$BOX_HOST" "$BOX_REPO" "$BOX_REF"
printf '%s/%s/raw/branch/%s/install.sh\n' "$BOX_HOST" "$BOX_REPO" "$BOX_REF" ;;
esac
}
# One pasteable recovery command per candidate URL. Never join candidates
# with English prose or shell metacharacters — "curl A | bash; if that
# 404s: curl B | bash" is not valid shell (`bash -n` exits 2) and is the
# same class of operator-facing failure #111 exists to remove (#125 /
# codex REQUEST_CHANGES on !114). Multi-candidate display uses separate
# prefixed lines (try: / or:); a single candidate (GitHub default) is a
# bare pasteable command — a try: prefix turns paste into a silent no-op
# (`try:` is not a command; the pipe's bash still exits 0). After a live
# probe succeeds the install loop rewrites BOX_MANUAL to the single URL
# that worked. Consumers MUST emit via box_manual_emit — never interpolate
# ${BOX_MANUAL} into a single log/warn string (multi-line orphans the or:
# line; claude REQUEST_CHANGES on 1c9a245).
box_manual_cmd() { # box_manual_cmd <url> — one pasteable install line
printf 'curl -fsSL %s | BOX_YES=1 BOX_REF=%s bash\n' "$1" "$BOX_REF"
}
box_manual_text() {
local _n=0 _url _cmd _urls=()
while IFS= read -r _url; do
[ -n "$_url" ] && _urls+=("$_url")
done < <(box_install_urls)
# Single candidate: bare command (no try:). Multi: try:/or: lines.
if [ "${#_urls[@]}" -le 1 ]; then
if [ "${#_urls[@]}" -eq 1 ]; then
_cmd="$(box_manual_cmd "${_urls[0]}")"
printf '%s' "$_cmd"
fi
return 0
fi
for _url in "${_urls[@]}"; do
_cmd="$(box_manual_cmd "$_url")"
_cmd="${_cmd%$'\n'}"
_n=$((_n + 1))
if [ "$_n" -eq 1 ]; then
printf 'try: %s\n' "$_cmd"
else
printf 'or: %s\n' "$_cmd"
fi
done
}
# Emit BOX_MANUAL one line at a time through log or warn. Never splice the
# multi-line value into a prose sentence.
box_manual_emit() { # box_manual_emit log|warn
local _fn="$1" _line
while IFS= read -r _line; do
[ -n "$_line" ] && "$_fn" " ${_line}"
done <<EOF
$BOX_MANUAL
EOF
}
BOX_INSTALL_URL="$(box_install_urls | head -n1)"
# Newline-separated recovery lines — each command (after optional try:/or:
# prefix) is independently pasteable.
BOX_MANUAL="$(box_manual_text)"
if [ "${RIG_SKIP_BOX_INSTALL:-}" = "1" ]; then if [ "${RIG_SKIP_BOX_INSTALL:-}" = "1" ]; then
log "RIG_SKIP_BOX_INSTALL=1 — skipping box install; to prepare Incus by hand later:" log "RIG_SKIP_BOX_INSTALL=1 — skipping box install; to prepare Incus by hand later: ${BOX_MANUAL}"
box_manual_emit log
elif ! command -v curl >/dev/null 2>&1; then elif ! command -v curl >/dev/null 2>&1; then
warn "curl not found — skipping box install; once curl is present, prepare Incus with:" warn "curl not found — skipping box install; once curl is present, prepare Incus with: ${BOX_MANUAL}"
box_manual_emit warn
else else
log "installing box (${BOX_REPO}@${BOX_REF}) and running its host setup — box owns Incus, not rig" log "installing box (${BOX_REPO}@${BOX_REF}) and running its host setup — box owns Incus, not rig"
# BOX_YES=1 in the environment: non-interactive AND keeps setup-host, so box # BOX_YES=1 in the environment: non-interactive AND keeps setup-host, so box
# builds the Incus stack rather than only dropping the CLI on PATH. Running as # builds the Incus stack rather than only dropping the CLI on PATH. Running as
# root, box installs globally (/opt/box + /usr/local/bin). No-op if box is # root, box installs globally (/opt/box + /usr/local/bin). No-op if box is
# already installed, so re-running bootstrap converges instead of reinstalling. # already installed, so re-running bootstrap converges instead of reinstalling.
# Download and execute are separate so a 404 on /raw/tag/ can fall through # A curl failure (no network) fails the pipe under pipefail and lands in the
# to /raw/branch/ without running a half-fetched body, and so an installer # else — a warning, never an abort: box is the host extra, the OS+tailnet core
# that runs and fails is NOT retried against the next candidate. # is already done.
# A curl failure (no network) lands in the else — a warning, never an if curl -fsSL "$BOX_INSTALL_URL" | BOX_YES=1 bash; then
# abort: box is the host extra, the OS+tailnet core is already done.
BOX_SCRIPT="$(mktemp)"
BOX_GOT=""
while IFS= read -r _box_url; do
if curl -fsSL "$_box_url" -o "$BOX_SCRIPT"; then
BOX_GOT="$_box_url"
BOX_INSTALL_URL="$_box_url"
BOX_MANUAL="$(box_manual_cmd "$BOX_INSTALL_URL" | tr -d '\n')"
break
fi
done < <(box_install_urls)
if [ -n "$BOX_GOT" ] && BOX_YES=1 BOX_REF="$BOX_REF" bash "$BOX_SCRIPT"; then
# Don't trust the exit code — prove the effective state (issue #12). An # Don't trust the exit code — prove the effective state (issue #12). An
# installer can exit 0 having done less than it claims: box's setup-host # installer can exit 0 having done less than it claims: box's setup-host
# is written for a sudo-capable user, and one of its paths exits 0 after # is written for a sudo-capable user, and one of its paths exits 0 after
@ -847,18 +721,14 @@ EOF
if box doctor >/dev/null 2>&1; then if box doctor >/dev/null 2>&1; then
log "box installed and host set up — 'box doctor' passed; mint guest boxes with 'box new'" log "box installed and host set up — 'box doctor' passed; mint guest boxes with 'box new'"
else else
warn "box is on PATH but 'box doctor' does not pass — the CLI landed, the host stack is unproven. Run 'box doctor' for the verdict, then 'box setup-host' (or finish by hand:)" warn "box is on PATH but 'box doctor' does not pass — the CLI landed, the host stack is unproven. Run 'box doctor' for the verdict, then 'box setup-host' (or finish by hand: ${BOX_MANUAL})"
box_manual_emit warn
fi fi
else else
warn "box's installer reported success but no 'box' is on PATH — the install did not take effect. Finish the host by hand:" warn "box's installer reported success but no 'box' is on PATH — the install did not take effect. Finish the host by hand: ${BOX_MANUAL}"
box_manual_emit warn
fi fi
else else
warn "box install did not complete (no network, or box's installer failed); bootstrap's core work is done. Finish the host by hand:" warn "box install did not complete (no network, or box's installer failed); bootstrap's core work is done. Finish the host by hand: ${BOX_MANUAL}"
box_manual_emit warn
fi fi
rm -f "$BOX_SCRIPT"
fi fi
fi fi
@ -894,17 +764,6 @@ if [ -n "$USERS_FILE" ]; then
"$HERE/users-apply.sh" --file "$USERS_FILE" "$HERE/users-apply.sh" --file "$USERS_FILE"
fi fi
# A registry machine's optional install is the final convergence phase: after
# join, host setup, the marker prerequisites, and operators. It inherits the
# caller environment, adds only the selected role, and runs from its definition
# directory. Definitions own idempotence, like bootstrap itself.
if [ -n "$MACHINE_TEMPLATE_DIR" ] && [ -e "$MACHINE_TEMPLATE_DIR/install.sh" ]; then
log "running install hook for ${ROLE} from $(templates_source_desc)"
if ! (cd "$MACHINE_TEMPLATE_DIR" && RIG_ROLE="$ROLE" bash ./install.sh); then
die "install hook failed for role $ROLE from $(templates_source_desc)"
fi
fi
log "done — role ${ROLE}, hostname ${TS_HOSTNAME}" log "done — role ${ROLE}, hostname ${TS_HOSTNAME}"
if [ "$ROLE" = "control-plane-server" ]; then if [ "$ROLE" = "control-plane-server" ]; then
log "next: rig coolify install --version <pin>" log "next: rig coolify install --version <pin>"

557
commands/forgejo-runner-install.sh
View file

@ -1,557 +0,0 @@
#!/usr/bin/env bash
# rig forgejo-runner install — Forgejo Actions runner as a systemd service
# under an unprivileged user. Outbound-only (long-poll to the instance), no
# inbound ports. Convergent toward --instance: re-running against the instance
# the box is already on leaves it alone; a box registered to a DIFFERENT
# instance is refused, never silently restarted on the old one.
#
# The GitHub sibling (runner-install.sh) refuses Docker outright: it converges
# a fleet MACHINE, where `docker` group membership is root-equivalent and the
# blast radius is the machine. This command's home is a ci-box TENANT, where
# bootstrap-tenant.sh has already installed Docker and added the tenant user to
# the group, and where the blast radius is a disposable guest with no inbound
# path. Same trade, different machine, opposite answer — which is why this is a
# separate command and not a flag on that one.
set -euo pipefail
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
# shellcheck source=SCRIPTDIR/lib/forgejo-runner-config.sh
. "$HERE/lib/forgejo-runner-config.sh"
# shellcheck source=SCRIPTDIR/lib/admin-path.sh
. "$HERE/lib/admin-path.sh"
log() { printf 'rig-forgejo-runner: %s\n' "$*"; }
warn() { printf 'rig-forgejo-runner: WARNING: %s\n' "$*" >&2; }
die() { printf 'rig-forgejo-runner: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
# The default label map. `runs-on: ubuntu-latest` is what a workflow written
# for GitHub says, so it must mean something here or every workflow needs
# editing to migrate; catthehacker's image is the act/Forgejo ecosystem's
# stand-in for GitHub's runner image. `docker` is the lean second option.
#
# Both are `docker://` — jobs run in CONTAINERS on the box's own dockerd, not
# on the box itself. No docker-in-docker: the guide this came from stacks a
# privileged dind sidecar with a plaintext tcp://…:2375 daemon to isolate jobs
# from a shared CI server, and inside a box that boundary is already paid for.
#
# WHY act-22.04 (slim) for ubuntu-latest, not full-22.04 — measured 2026-08-01
# against ghcr manifests (#144):
# act-22.04: ~0.55 GB compressed / ~2.2 GB on disk — no shellcheck
# full-22.04: ~18.67 GB compressed / ~54.5 GB on disk — has shellcheck 0.8.0
# A normal box-class ci tenant cannot hold 54.5 GB (typical free space ~34 GB).
# So ubuntu-latest stays slim, and workflows must not assume GitHub-image tools
# (rig's own ci.yml installs shellcheck when missing). Operators who need the
# full tool surface opt in with runs-on: ubuntu-latest-full — that label is
# inert until matched, so boxes that never ask pay nothing.
DEFAULT_LABELS='ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-22.04,ubuntu-latest-full:docker://ghcr.io/catthehacker/ubuntu:full-22.04,docker:docker://node:22-bookworm'
# labels_are_a_retired_default <recorded>
#
# True when <recorded> is a past DEFAULT_LABELS value rig has shipped — the
# only plain-converge case that should warn about re-registration (#144).
# Custom operator maps (drill's Leg 3, any --labels) must return false so a
# bare re-run stays quiet. One pattern per past default; add when the string
# changes. Extracted and driven by test/cli.sh — a grep pin alone cannot prove
# the match is exact.
labels_are_a_retired_default() {
case "${1:-}" in
'ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-22.04,docker:docker://node:22-bookworm') return 0 ;;
*) return 1 ;;
esac
}
# fetch_and_verify_sha256 <asset-url> <file> <sumfile> <label>
#
# The whole checksum POLICY, in one place: fetch the published .sha256 beside
# an asset and prove the download matches it. Prints the reason on stderr and
# returns 1 on any failure; the caller supplies the refusal in its own voice.
#
# BYTE-IDENTICAL to the copy in docs/templates/ci-box/install.sh, diffed by
# test/cli.sh — the valid_version / templates_archive_urls precedent. The two
# downloaders cannot share a lib: this one sources commands/lib/, and that one
# is a REGISTRY DEFINITION that runs standalone inside a mint from a fetched
# tarball, with rig's tree nowhere in reach. So the pin is the only mechanism
# that keeps one policy from becoming two.
#
# Review !110 is the evidence for why that matters: a fail-open branch lived in
# BOTH copies while a grep for "checksum mismatch" passed against both, because
# the string it looked for sat right beside the branch it could not see. The
# next checksum-policy change must not be able to land in one file only.
#
# AN UNFETCHABLE CHECKSUM REFUSES — it is a gate, not a courtesy. The earlier
# reasoning ("do not let an upstream layout change break installs") reasons
# about the wrong failure: a layout change moves the BINARY url too, so the
# download would already have died. "Binary yes, checksum no" is not what a
# layout change looks like — it is what an interfered fetch looks like, which
# is precisely what a checksum exists to catch. Failing open would hand an
# unverified root install to anyone able to block a single URL. There is
# deliberately no bypass flag: if upstream really does move its assets, that is
# a rig PR editing the URL, not an operator improvising past a security gate.
fetch_and_verify_sha256() {
local url="$1" file="$2" sumfile="$3" label="$4" want got
if ! curl -fsSL "${url}.sha256" -o "$sumfile" 2>/dev/null; then
printf 'no published .sha256 for %s at %s.sha256 — the binary itself downloaded, so this is not an upstream layout change; check what is intercepting the fetch\n' "$label" "$url" >&2
return 1
fi
# The published .sha256 names the asset, not our temp path. Compare the
# digest itself rather than rewriting the file into sha256sum -c's format:
# one comparison, no parsing of a file we did not write.
want="$(tr -d '\r' < "$sumfile" 2>/dev/null | awk '{print $1}' | head -n1)"
got="$(sha256sum "$file" | awk '{print $1}')"
if [ -z "$want" ]; then
printf 'the published checksum for %s is unreadable — a fetch that succeeds but returns nothing usable is not a verified download\n' "$label" >&2
return 1
fi
if [ "$want" != "$got" ]; then
printf 'checksum mismatch for %s: published %s, downloaded %s\n' "$label" "$want" "$got" >&2
return 1
fi
printf 'checksum verified (%s)\n' "$got"
}
usage() {
cat <<'EOF'
usage: rig forgejo-runner install --instance <url> [options]
--instance <url> Forgejo instance the runner registers to (required),
e.g. https://forgejo.example.com
--version <pin> forgejo-runner release to install, e.g. 12.13.2
(default: the latest release, resolved at install
time). Pin it for a deterministic, auditable install.
--name <name> runner name (default: this host's hostname)
--labels <csv> runner labels; replaces the default. The default maps
ubuntu-latest (slim act image), ubuntu-latest-full
(opt-in parity image), and docker onto containers so
a workflow written for GitHub runs; full tools need
runs-on: ubuntu-latest-full or an install step.
--user <name> unprivileged service user (default: the tenant user
`ci` when it exists, else forgejo-runner; created if
absent; never root)
Installs forgejo-runner as a systemd service under an unprivileged user. The
runner is an agent, not a server: it long-polls the instance outbound and
receives jobs down that already-established connection, so it needs ZERO
inbound ports.
Jobs run in Docker containers on this box's own daemon. Inside a ci-box tenant
that daemon is already there — `rig bootstrap ci-box` installs it and puts the
tenant user in the `docker` group.
Provide the runner registration token via the FORGEJO_RUNNER_TOKEN env var or
the interactive prompt. Get one from the scope you want the runner to serve:
instance Site Administration > Actions > Runners > Create new Runner
org Org > Settings > Actions > Runners
repo Repo > Settings > Actions > Runners
The SCOPE IS THE TOKEN'S, not a flag here. It is consumed at registration and
never written to disk by rig.
Convergent toward --instance: re-running against the instance this box is
already on re-uses the binary, skips registration, and never asks for a token.
A box registered to a DIFFERENT instance is refused — take it off the old one
with `rig forgejo-runner remove` first.
EOF
}
# --- args (validated before the root check, so errors are testable) ---------
INSTANCE=""
VERSION=""
RUNNER_NAME="$(hostname)"
LABELS="$DEFAULT_LABELS"
# Whether --labels was ASKED FOR, distinct from what it resolved to. A rerun
# cannot apply labels (Forgejo owns them from registration time), and the
# difference between "operator requested a change" and "operator passed
# nothing" is what separates a warning worth printing from noise on every
# converge.
LABELS_EXPLICIT=0
RUNNER_USER=""
while [ $# -gt 0 ]; do
case "$1" in
--instance)
[ $# -ge 2 ] || die "--instance needs a value" 2
INSTANCE="$2"; shift 2 ;;
--version)
[ $# -ge 2 ] || die "--version needs a value" 2
VERSION="$2"; shift 2 ;;
--name)
[ $# -ge 2 ] || die "--name needs a value" 2
RUNNER_NAME="$2"; shift 2 ;;
--labels)
[ $# -ge 2 ] || die "--labels needs a value" 2
LABELS="$2"; LABELS_EXPLICIT=1; shift 2 ;;
--user)
[ $# -ge 2 ] || die "--user needs a value" 2
RUNNER_USER="$2"; shift 2 ;;
--repo)
# Named, not "unknown flag": everyone arrives here from `rig runner
# install --repo`, and the honest answer is that the argument does not
# exist on this forge rather than that it is misspelled.
[ $# -ge 2 ] && shift
die "--repo does not exist here: a Forgejo runner registers to an INSTANCE, and whether it serves that whole instance, one org, or one repo is a property of the registration TOKEN you mint in Forgejo's UI. Pass --instance <url> and mint the token at the scope you want." 2 ;;
-h|--help) usage; exit 0 ;;
*) die "unknown flag: $1" 2 ;;
esac
done
# --- validation ----------------------------------------------------------
[ -n "$INSTANCE" ] || die "--instance <url> is required" 2
case "$INSTANCE" in
https://*|http://*) ;;
*) die "--instance must be a URL with a scheme, e.g. https://forgejo.example.com (got: $INSTANCE)" 2 ;;
esac
# A path component would be a repo URL — the GitHub habit, and the one mistake
# that produces a runner registered somewhere subtly wrong rather than a clean
# failure. Refuse it by name.
case "${INSTANCE#*://}" in
*/*[!/]*) die "--instance takes the instance ROOT, not a repository URL: got ${INSTANCE}. Scope comes from the token, not the URL." 2 ;;
esac
VERSION="${VERSION#v}"
# The same sane-version charset the resolve-latest path enforces further down.
# An explicit pin skipped it entirely and went straight into the download URL,
# so a value carrying `/` or `..` was interpolated into a URL PATH rather than
# refused. Not a trust boundary — this command is already root, and the operator
# typed the flag — but a pin that cannot name a release should fail BY NAME at
# parse time, not as an opaque 404 forty lines later. install.sh's
# valid_version is the same instinct, and the asymmetry was the whole defect:
# the value rig resolves for itself was checked, the one it is handed was not.
case "$VERSION" in
"") ;; # unset — the latest release is resolved and validated below
*[!0-9.]*|.*|*.)
die "--version must be a release number like 12.13.2 (got: ${VERSION})" 2 ;;
esac
[ -n "$LABELS" ] || die "--labels must not be empty" 2
# The tenant user is the default when it is there: inside a ci-box the runner
# IS the tenant, and inventing a second service account beside it would leave
# the docker-group membership bootstrap-tenant.sh converged on the wrong user.
# Falls back to a dedicated account so this still works on a plain machine.
if [ -z "$RUNNER_USER" ]; then
if id -u ci >/dev/null 2>&1; then RUNNER_USER="ci"; else RUNNER_USER="forgejo-runner"; fi
fi
[ "$RUNNER_USER" != "root" ] || die "runner user must not be root" 2
# --- guards ----------------------------------------------------------------
[ "$(id -u)" -eq 0 ] || die "must run as root"
# Root is not enough: the admin binaries must also be reachable (#139). This
# sits beside the root check so identity and capability are asserted together,
# and BEFORE any prompt or download — a token typed for a doomed run is waste.
#
# BOTH binaries this command goes on to call: useradd at the user-create below,
# usermod at the docker-group add further down. Naming only the first would
# still consume the token on a PATH that happened to resolve useradd but not
# usermod — the same failure one step later, which is the shape #75 exists to
# refuse (a sweep that covers most of its call sites is the hole the next bug
# arrives through).
require_admin_bins useradd usermod
if [ -r /etc/os-release ]; then
# Sourced in a subshell: os-release defines VERSION (e.g. "13 (trixie)"),
# which would clobber this script's $VERSION.
# shellcheck source=/dev/null
OS_FAMILY="$(. /etc/os-release && printf '%s %s' "${ID:-}" "${ID_LIKE:-}")"
case "$OS_FAMILY" in
*debian*) ;;
*) warn "not a Debian-family system (${OS_FAMILY:-unknown}); proceeding anyway" ;;
esac
else
warn "cannot read /etc/os-release; proceeding anyway"
fi
command -v curl >/dev/null || die "curl is required (run rig bootstrap first)"
command -v systemctl >/dev/null || die "systemctl is required — this command installs the runner as a systemd service"
# --- is this box already registered somewhere else? --------------------------
# Before anything is prompted for, downloaded, or started: --instance must
# agree with what is already on the box. Everything below treats an existing
# .runner as "nothing to do" — right for the instance the box is already on,
# silently wrong for any other. See assert_runner_instance.
REG_PENDING=1
if id -u "$RUNNER_USER" >/dev/null 2>&1; then
USER_HOME="$(getent passwd "$RUNNER_USER" | cut -d: -f6)"
RUNNER_DIR="$USER_HOME/forgejo-runner"
assert_runner_instance "$RUNNER_DIR" "$INSTANCE" || exit 1
if [ -e "$RUNNER_DIR/.runner" ]; then
REG_PENDING=0
fi
fi
# --- registration token — only when registration is actually pending -------
if [ "$REG_PENDING" -eq 1 ]; then
FORGEJO_RUNNER_TOKEN="${FORGEJO_RUNNER_TOKEN:-}"
# Prompt only on a tty: headless, a bare `read` dies under set -e with no
# message at all. Refuse loudly, naming the variable.
if [ -z "$FORGEJO_RUNNER_TOKEN" ]; then
[ -t 0 ] || die "FORGEJO_RUNNER_TOKEN is unset and stdin is not a tty — set FORGEJO_RUNNER_TOKEN to run unattended"
read -rsp "forgejo runner registration token: " FORGEJO_RUNNER_TOKEN || { echo; die "no registration token read (EOF) — set FORGEJO_RUNNER_TOKEN to run unattended"; }
echo
fi
[ -n "$FORGEJO_RUNNER_TOKEN" ] || die "empty registration token"
fi
# --- user --------------------------------------------------------------------
if ! id -u "$RUNNER_USER" >/dev/null 2>&1; then
useradd --create-home --shell /bin/bash "$RUNNER_USER"
log "created user ${RUNNER_USER}"
else
log "user exists"
fi
USER_HOME="$(getent passwd "$RUNNER_USER" | cut -d: -f6)"
RUNNER_GROUP="$(id -gn "$RUNNER_USER")"
RUNNER_DIR="$USER_HOME/forgejo-runner"
BIN=/usr/local/bin/forgejo-runner
# The runner talks to dockerd over its socket, so it needs the group. In a
# ci-box bootstrap-tenant.sh already did this for the tenant user; on a plain
# machine, or for a --user that is not the tenant, it has not.
if getent group docker >/dev/null 2>&1; then
if id -nG "$RUNNER_USER" | tr ' ' '\n' | grep -qx docker; then
log "${RUNNER_USER} already in the docker group"
else
usermod -aG docker "$RUNNER_USER"
log "added ${RUNNER_USER} to the docker group"
fi
else
warn "no docker group on this box — jobs using docker:// labels will fail. Inside a ci-box, 'rig bootstrap ci-box' installs docker; elsewhere install it before running jobs."
fi
# --- download ----------------------------------------------------------------
# Forgejo publishes BARE BINARIES (not a tarball) with a .sha256 beside each
# one. Taking that checksum is nearly free and makes the install auditable —
# the same instinct as `coolify install`'s mandatory version pin.
#
# "Already present" is NOT enough to skip here, and this is where the GitHub
# sibling's shape must not be copied. Its skip is justified by "self-update
# owns upgrades" — actions/runner updates itself, and GitHub refuses jobs from
# stale runners, so freezing it would be pointless. **forgejo-runner does not
# self-update.** Nothing else ever moves the version, so a bare presence check
# would mean the binary a box first happened to get is the binary it keeps
# forever.
#
# That lands hardest on the path this command is FOR: a ci-box's template
# install.sh preinstalls /usr/local/bin/forgejo-runner at mint, so the
# executable always exists before an operator ever runs this — and --version,
# documented as the deterministic-pin lever, would silently do nothing on
# every ci-box in the fleet.
#
# So: converge toward --version when it is given, exactly as this command
# converges toward --instance. A pin is not a trust boundary the way an
# instance is (that one refuses), it is an instruction — including downward,
# which is what a pin is for. Absent a pin, an existing binary is left alone:
# chasing "latest" on every converge would make a re-run an unrequested
# upgrade, and convergence must not be a moving target.
# runner_version_of / runner_download_decision live in the lib, so the rule can
# be driven by test/cli.sh without root — see there for the full reasoning.
PRESENT_VER=""
HAVE_BIN=no
if [ -x "$BIN" ]; then
HAVE_BIN=yes
PRESENT_VER="$(runner_version_of "$BIN")"
fi
case "$(runner_download_decision "$HAVE_BIN" "$PRESENT_VER" "$VERSION")" in
skip)
NEED_DOWNLOAD=0
if [ -n "$VERSION" ]; then
log "forgejo-runner ${VERSION} already installed; skipping download"
else
log "forgejo-runner ${PRESENT_VER:-(version unreadable)} already present at ${BIN}; skipping download (pass --version <pin> to converge to a specific release)"
fi ;;
converge)
NEED_DOWNLOAD=1
log "converging ${BIN}: ${PRESENT_VER:-unreadable} -> ${VERSION} (--version)" ;;
*)
NEED_DOWNLOAD=1 ;;
esac
if [ "$NEED_DOWNLOAD" -eq 1 ]; then
case "$(uname -m)" in
x86_64) ARCH="amd64" ;;
aarch64) ARCH="arm64" ;;
*) die "unsupported arch: $(uname -m)" ;;
esac
if [ -z "$VERSION" ]; then
# No pin given: resolve the latest release by following the redirect on
# the /releases/latest page — no API call, no token, no JSON to parse on
# a dependency-free box (install.sh's resolve_latest_tag idiom).
LATEST_URL="$(curl -fsSLI -o /dev/null -w '%{url_effective}' \
https://code.forgejo.org/forgejo/runner/releases/latest)" \
|| die "could not resolve the latest forgejo-runner release"
VERSION="${LATEST_URL##*/}"
VERSION="${VERSION#v}"
case "$VERSION" in
""|*[!0-9.]*) die "could not parse a version from ${LATEST_URL}" ;;
esac
log "resolved latest forgejo-runner: ${VERSION}"
fi
ASSET="forgejo-runner-${VERSION}-linux-${ARCH}"
URL="https://code.forgejo.org/forgejo/runner/releases/download/v${VERSION}/${ASSET}"
WORKDIR="$(mktemp -d)"
cleanup() { rm -rf "$WORKDIR"; }
trap cleanup EXIT
log "downloading forgejo-runner ${VERSION} (${ARCH})"
curl -fsSL "$URL" -o "$WORKDIR/forgejo-runner" \
|| die "could not download ${URL}"
fetch_and_verify_sha256 "$URL" "$WORKDIR/forgejo-runner" "$WORKDIR/forgejo-runner.sha256" "$ASSET" \
|| die "refusing to install an unverified ${ASSET} — it lands as root and runs under a systemd unit. See the checksum failure above."
# Staged beside the target and RENAMED into place, never written over.
# Replacing a running executable in place fails with ETXTBSY, and this path
# now runs on boxes where the daemon is live (a --version converge). A
# rename is atomic and leaves the running process on the old inode until the
# restart below picks up the new one.
install -m 0755 -o root -g root "$WORKDIR/forgejo-runner" "$BIN.rig-new"
mv -f "$BIN.rig-new" "$BIN"
log "installed ${BIN}"
fi
# `|| true` so the refusal BELOW is the one that fires. Under set -euo pipefail
# a bare `VAR="$(cmd | head)"` dies at the assignment when cmd exits non-zero,
# which is precisely the case this line exists to diagnose — see the lib.
INSTALLED_VER="$("$BIN" --version 2>/dev/null | head -n1 || true)"
[ -n "$INSTALLED_VER" ] || die "${BIN} does not answer --version — the download landed but cannot run"
# The converge actually took — asserted, not assumed. A pin that silently did
# not land is exactly the failure --version exists to make impossible.
if [ -n "$VERSION" ]; then
EFFECTIVE_VER="$(runner_version_of "$BIN")"
[ "$EFFECTIVE_VER" = "$VERSION" ] \
|| die "asked for forgejo-runner ${VERSION} but ${BIN} reports ${EFFECTIVE_VER:-nothing} after install"
fi
# --- register ----------------------------------------------------------------
# UPSTREAM MARKS `register` DEPRECATED (measured on v12.13.2: both `register`
# and `create-runner-file` carry "(deprecated)" in their help). It is chosen
# here anyway, deliberately, and this is the reasoning to revisit when it
# finally goes:
#
# - It still works. `daemon` reads the `.runner` this writes, resolves the
# instance from it, and connects — verified against a live instance, where
# a planted `.runner` got as far as "Unauthenticated: unregistered runner".
# The mechanism is intact; only the credential was fake.
# - The successor needs MORE than rig can honestly ask for at this layer:
# `daemon --url --uuid --token-url` requires the runner to already exist on
# the instance, so the operator would have to create it via API/UI and
# carry back a UUID. That is a second, differently-shaped credential dance
# for no gain today.
# - `register` writes a file `status` can read back. The successor's config
# lives in flags on a unit line, where "what is this box registered to" has
# no on-disk answer that is not just rig's own copy of what it was told.
#
# When upstream removes it: the shape becomes `daemon --url/--uuid`, the unit
# gains those flags, and forgejo-runner-config.sh's readers move to whatever
# holds the UUID. assert_runner_instance's contract survives either way — it
# asks about the instance, which both spellings record.
install -d -m 0755 -o "$RUNNER_USER" -g "$RUNNER_GROUP" "$RUNNER_DIR"
# forgejo-runner's cache server writes to $HOME/.cache, which ProtectHome makes
# read-only below. Create it HERE, before the unit can reference it: a
# ReadWritePaths entry naming a path that does not exist makes systemd refuse
# to start the unit at all ("Failed to set up mount namespacing"), which is
# worse than the disabled cache it was meant to fix. Measured on a live runner,
# 2026-07-31 (#135). Root-owned would fail the same way under User=, so it
# carries the runner's own ownership like RUNNER_DIR above.
install -d -m 0755 -o "$RUNNER_USER" -g "$RUNNER_GROUP" "$USER_HOME/.cache"
if [ -e "$RUNNER_DIR/.runner" ]; then
log "already registered; skipping registration"
# Registration was skipped, so the labels on the instance are the ones it was
# registered with — NOT whatever this invocation was passed. Forgejo owns
# labels from registration time; a re-run never rewrites them.
#
# Two warn paths (#144):
# EXPLICIT --labels that differs → operator asked and it was not applied.
# Plain converge whose recorded labels match a known *retired* default →
# rig's default map moved (e.g. added ubuntu-latest-full). Without this
# the operator re-runs install, sees "already registered", and believes
# they have the new default while Forgejo still holds the old set.
#
# Do NOT warn on every RECORDED != current default: that fires forever for
# any runner the operator deliberately gave custom --labels (drill's Leg 3
# registers drill:docker://node:22-bookworm). The old LABELS_EXPLICIT-only
# gate existed to avoid that noise; retired-default matching keeps the
# silence for intentional maps and still catches silent drift off a past
# rig default. Re-register only to pick up labels the old set never had —
# nothing matching the recorded set is broken by the map change alone.
if [ -r "$RUNNER_DIR/.rig-labels" ]; then
RECORDED="$(cat "$RUNNER_DIR/.rig-labels")"
if [ "$LABELS_EXPLICIT" -eq 1 ] && [ "$RECORDED" != "$LABELS" ]; then
warn "--labels was not applied: this runner is already registered, and Forgejo owns its labels from registration time. It still has: ${RECORDED}. Labels are what 'runs-on' matches, so changing them means re-registering: 'rig forgejo-runner remove' then install again with the labels you want."
elif [ "$LABELS_EXPLICIT" -eq 0 ] && labels_are_a_retired_default "$RECORDED"; then
warn "this runner was registered with an older rig default label set. The current default adds ubuntu-latest-full (the GitHub-parity image). Labels are fixed at registration, so picking it up means re-registering: 'rig forgejo-runner remove' then install again. Nothing you run today is affected — re-register only if you want the new label."
fi
fi
else
log "registering runner ${RUNNER_NAME} against ${INSTANCE}"
(cd "$RUNNER_DIR" && runuser -u "$RUNNER_USER" -- env HOME="$USER_HOME" \
"$BIN" register --no-interactive \
--instance "$INSTANCE" --token "$FORGEJO_RUNNER_TOKEN" \
--name "$RUNNER_NAME" --labels "$LABELS") \
|| die "registration failed — check the token is a RUNNER registration token from ${INSTANCE} and has not been used already"
[ -e "$RUNNER_DIR/.runner" ] \
|| die "register reported success but wrote no ${RUNNER_DIR}/.runner"
# INSIDE the registration branch, where runner-install.sh keeps its copy and
# for the same reason: this file records what rig ACTUALLY registered with,
# so `status` has something to read back. Writing it unconditionally — as an
# earlier draft did — makes a plain re-run stamp this invocation's labels
# over a registration that used different ones, and `status` then reports
# confidently wrong labels while Forgejo still holds the originals. A
# metadata file that can disagree with the thing it describes is worse than
# no metadata file.
printf '%s\n' "$LABELS" > "$RUNNER_DIR/.rig-labels"
chown "$RUNNER_USER:$RUNNER_GROUP" "$RUNNER_DIR/.rig-labels"
fi
# EVERY run, registration or not: .runner holds the runner's own long-lived
# token, and a mode that drifted leaks it silently. See the lib.
forgejo_runner_secure "$RUNNER_DIR" "$RUNNER_USER" "$RUNNER_GROUP"
# --- service -------------------------------------------------------------
# Written by rig rather than shipped by upstream: forgejo-runner has no
# svc.sh, so there is no vendor unit to defer to (the GitHub sibling defers to
# actions/runner's). Converged like every file rig writes — cmp-guarded, so a
# re-run that changes nothing reloads nothing.
UNIT=/etc/systemd/system/forgejo-runner.service
UNIT_TMP="$(mktemp)"
cat > "$UNIT_TMP" <<EOF
[Unit]
Description=Forgejo Actions runner
Documentation=https://forgejo.org/docs/latest/admin/actions/
After=network-online.target docker.service
Wants=network-online.target
[Service]
Type=simple
User=${RUNNER_USER}
WorkingDirectory=${RUNNER_DIR}
ExecStart=${BIN} daemon
Restart=on-failure
RestartSec=10
# The runner supervises job containers on this box's docker socket; it is not
# a sandbox for them. These keep the DAEMON from being a soft target.
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=full
ProtectHome=read-only
ReadWritePaths=${RUNNER_DIR} ${USER_HOME}/.cache
[Install]
WantedBy=multi-user.target
EOF
if ! cmp -s "$UNIT_TMP" "$UNIT" 2>/dev/null; then
install -m 0644 "$UNIT_TMP" "$UNIT"
systemctl daemon-reload
log "systemd unit written: ${UNIT}"
else
log "systemd unit already current"
fi
rm -f "$UNIT_TMP"
systemctl enable forgejo-runner >/dev/null 2>&1 || die "could not enable forgejo-runner.service"
systemctl restart forgejo-runner || die "could not start forgejo-runner.service — see 'journalctl -u forgejo-runner'"
# Assert the EFFECTIVE state, not systemctl's exit code: a unit that starts and
# immediately dies (bad token, unreachable instance) leaves restart succeeding
# and the runner absent. Settle briefly, then ask.
active=""
for _ in 1 2 3 4 5 6; do
if systemctl is-active forgejo-runner >/dev/null 2>&1; then active=1; break; fi
sleep 2
done
[ -n "$active" ] || die "forgejo-runner.service is not active after 12s — see 'journalctl -u forgejo-runner' (a bad token or an unreachable instance both land here)"
log "runner ${RUNNER_NAME} (${INSTALLED_VER}) installed and running"
log "labels: ${LABELS}"
log "verify it shows Idle under ${INSTANCE} > Site Administration > Actions > Runners"
log "this box needs no inbound ports — the runner polls the instance outbound"

139
commands/forgejo-runner-remove.sh
View file

@ -1,139 +0,0 @@
#!/usr/bin/env bash
# rig forgejo-runner remove — take the service down and wipe this box's
# registration. Convergent: a box with nothing installed exits 0.
#
# There is no --local flag here, and its absence is the design. The GitHub
# sibling offers --local as an ESCAPE HATCH from a real deregistration
# handshake (config.sh remove --token, against an endpoint that mints removal
# tokens). Forgejo has no such handshake and no such endpoint: local is the
# only thing removal can ever be. Shipping the flag would advertise a
# server-side alternative that does not exist, and an operator would spend the
# afternoon hunting for the token that turns it off.
set -euo pipefail
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
# shellcheck source=SCRIPTDIR/lib/forgejo-runner-config.sh
. "$HERE/lib/forgejo-runner-config.sh"
log() { printf 'rig-forgejo-runner: %s\n' "$*"; }
warn() { printf 'rig-forgejo-runner: WARNING: %s\n' "$*" >&2; }
die() { printf 'rig-forgejo-runner: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
usage() {
cat <<'EOF'
usage: rig forgejo-runner remove [--user <name>]
--user <name> unprivileged service user (default: the tenant user `ci`
when it exists, else forgejo-runner)
Stops and disables the systemd service, then wipes this box's registration.
The binary and the user stay put, so a later `rig forgejo-runner install`
re-registers without downloading anything.
Forgejo has no runner deregistration endpoint, so this is always local-only:
the box is cleaned, and the runner stays listed as offline in the instance
until you delete it under Actions > Runners. No token is needed or asked for.
Convergent: safe to re-run; a box with no runner installed exits 0.
EOF
}
# --- args (validated before the root check, so errors are testable) ---------
RUNNER_USER=""
while [ $# -gt 0 ]; do
case "$1" in
--user)
[ $# -ge 2 ] || die "--user needs a value" 2
RUNNER_USER="$2"; shift 2 ;;
--local)
# Named rather than "unknown flag": it is the GitHub sibling's spelling,
# and the answer is that removal here is ALWAYS what --local means.
die "--local is not a flag here: Forgejo has no deregistration endpoint, so 'rig forgejo-runner remove' is always local-only. Re-run it without the flag, then delete the offline runner in the instance's Actions > Runners." 2 ;;
-h|--help) usage; exit 0 ;;
*) die "unknown flag: $1" 2 ;;
esac
done
if [ -z "$RUNNER_USER" ]; then
if id -u ci >/dev/null 2>&1; then RUNNER_USER="ci"; else RUNNER_USER="forgejo-runner"; fi
fi
# --- validation ------------------------------------------------------------
[ "$RUNNER_USER" != "root" ] || die "runner user must not be root" 2
# --- guards ----------------------------------------------------------------
[ "$(id -u)" -eq 0 ] || die "must run as root"
UNIT=/etc/systemd/system/forgejo-runner.service
# --- nothing to remove? -----------------------------------------------------
# The unit is checked INDEPENDENTLY of the user, and that ordering is the whole
# point. A missing user used to exit 0 here before the unit was ever looked at,
# so a deleted account with a leftover forgejo-runner.service reported "nothing
# to remove" and left the unit behind — while the absence-assert at the end,
# which never ran, implied removal had been complete. `bootstrap --undo`'s own
# unit check would still have caught it, but a verb that claims to have removed
# everything must not be the thing that lies about it.
RUNNER_DIR=""
if id -u "$RUNNER_USER" >/dev/null 2>&1; then
USER_HOME="$(getent passwd "$RUNNER_USER" | cut -d: -f6)"
RUNNER_DIR="$USER_HOME/forgejo-runner"
else
log "no ${RUNNER_USER} user on this box"
fi
if [ -z "$RUNNER_DIR" ] && [ ! -e "$UNIT" ]; then
log "no runner user and no unit on this box; nothing to remove"
exit 0
fi
if [ -n "$RUNNER_DIR" ] && [ ! -e "$RUNNER_DIR/.runner" ] && [ ! -e "$UNIT" ]; then
log "no runner registered in ${RUNNER_DIR}; nothing to remove"
exit 0
fi
[ -n "$RUNNER_DIR" ] || warn "the ${RUNNER_USER} user is gone but ${UNIT} is still here — removing the orphaned unit"
INSTANCE=""
RUNNER_NAME=""
if [ -n "$RUNNER_DIR" ]; then
INSTANCE="$(forgejo_runner_instance "$RUNNER_DIR")"
RUNNER_NAME="$(forgejo_runner_name "$RUNNER_DIR")"
fi
# --- service ---------------------------------------------------------------
# First, in both paths: stopping after the registration is wiped would strand a
# running daemon polling with credentials that no longer exist on disk.
if [ -e "$UNIT" ]; then
log "stopping and disabling forgejo-runner.service"
systemctl stop forgejo-runner >/dev/null 2>&1 || true
systemctl disable forgejo-runner >/dev/null 2>&1 || true
rm -f "$UNIT"
systemctl daemon-reload
else
log "no service installed; skipping"
fi
# --- registration -----------------------------------------------------------
# Every path below is gated on RUNNER_DIR being non-empty. With the user gone
# it is "", and an unguarded "$RUNNER_DIR/.rig-labels" would expand to
# "/.rig-labels" — an rm at the filesystem root, as root. The repo already
# treats this class of expansion as a hazard worth spelling out (`rm -rf
# "${ir:?}/versions/$ver"` in bin/rig); same discipline here.
if [ -n "$RUNNER_DIR" ]; then
if [ -e "$RUNNER_DIR/.runner" ]; then
rm -f "$RUNNER_DIR/.runner"
log "wiped the local registration"
fi
rm -f "$RUNNER_DIR/.rig-labels"
fi
# END WITH THE ABSENCE ASSERT: "removed" is a claim, and claims get verified
# (the `rig uninstall` precedent).
leftover=""
[ -n "$RUNNER_DIR" ] && [ -e "$RUNNER_DIR/.runner" ] && leftover="$leftover $RUNNER_DIR/.runner"
[ -e "$UNIT" ] && leftover="$leftover $UNIT"
if [ -n "$leftover" ]; then
printf 'rig-forgejo-runner: remove INCOMPLETE — still present:%s\n' "$leftover" >&2
exit 1
fi
log "runner removed; the binary stays for a future rig forgejo-runner install"
warn "the runner${RUNNER_NAME:+ ${RUNNER_NAME}} is still listed as offline in ${INSTANCE:-the instance} — delete it under Actions > Runners. Forgejo has no deregistration endpoint, so rig cannot do this for you."

118
commands/forgejo-runner-status.sh
View file

@ -1,118 +0,0 @@
#!/usr/bin/env bash
# rig forgejo-runner status — what is this box's Forgejo runner registered to?
# Read-only: reports what is already on the box. No credential, no network call.
set -euo pipefail
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
# shellcheck source=SCRIPTDIR/lib/forgejo-runner-config.sh
. "$HERE/lib/forgejo-runner-config.sh"
log() { printf 'rig-forgejo-runner: %s\n' "$*"; }
warn() { printf 'rig-forgejo-runner: WARNING: %s\n' "$*" >&2; }
die() { printf 'rig-forgejo-runner: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
# forgejo_runner_liveness_note <systemctl-state> — what `active` does not cover.
#
# `active` is the strongest health signal this command has, and it proves only
# that a process exists — not that the runner is still asking Forgejo for work.
# A poller can go quiet while the daemon stays up: measured 2026-07-30 (#129,
# #133), a daemon logged "[poller] launched" and never fetched a job dispatched
# four minutes later, while a daemon started fresh claimed that same queued task
# in one second. Both times it read as a label-mapping bug on the forge, which
# is the wrong place to look.
#
# A FUNCTION rather than an inline `if`, because the state boundary is the part
# worth pinning: an absent or inactive unit must say nothing, and a grep over
# the source cannot tell the difference (codex/kimi, !134).
#
# log, not warn: nothing has been DETECTED here. An idle runner with no queued
# jobs is silent in exactly the same way a stalled one is, so there is no signal
# separating them — a warning on every status run would be crying wolf, and warn
# in this file means a drift actually measured (the .runner mode below).
forgejo_runner_liveness_note() {
[ "${1:-}" = active ] || return 0
log " note: 'active' is not proof the runner is fetching jobs — only that the process is up."
log " If a job stays queued and its run page says it never started, run"
log " 'systemctl restart forgejo-runner' and re-read before suspecting the labels."
}
usage() {
cat <<'EOF'
usage: rig forgejo-runner status [--user <name>]
--user <name> unprivileged service user (default: the tenant user `ci`
when it exists, else forgejo-runner)
Prints the Forgejo instance this box's runner is registered to, its runner
name, the labels rig recorded when it registered, the install directory, and
the systemd unit and its state.
Reads only the runner's own on-disk config — no token, no network call. The
registration secret that config holds is never printed. Exits 1 when no runner
is installed.
EOF
}
# --- args (validated before the root check, so errors are testable) ---------
RUNNER_USER=""
while [ $# -gt 0 ]; do
case "$1" in
--user)
[ $# -ge 2 ] || die "--user needs a value" 2
RUNNER_USER="$2"; shift 2 ;;
-h|--help) usage; exit 0 ;;
*) die "unknown flag: $1" 2 ;;
esac
done
if [ -z "$RUNNER_USER" ]; then
if id -u ci >/dev/null 2>&1; then RUNNER_USER="ci"; else RUNNER_USER="forgejo-runner"; fi
fi
# --- validation ------------------------------------------------------------
[ "$RUNNER_USER" != "root" ] || die "runner user must not be root" 2
# --- guards ----------------------------------------------------------------
[ "$(id -u)" -eq 0 ] || die "must run as root"
id -u "$RUNNER_USER" >/dev/null 2>&1 \
|| die "no runner installed (no ${RUNNER_USER} user on this box)"
USER_HOME="$(getent passwd "$RUNNER_USER" | cut -d: -f6)"
RUNNER_DIR="$USER_HOME/forgejo-runner"
[ -e "$RUNNER_DIR/.runner" ] \
|| die "no runner registered in ${RUNNER_DIR}"
# --- read the runner's own config -------------------------------------------
INSTANCE="$(forgejo_runner_instance "$RUNNER_DIR")"
RUNNER_NAME="$(forgejo_runner_name "$RUNNER_DIR")"
if [ -r "$RUNNER_DIR/.rig-labels" ]; then
LABELS="$(cat "$RUNNER_DIR/.rig-labels")"
else
LABELS="(not recorded on this box — see the instance's Actions > Runners)"
fi
UNIT=/etc/systemd/system/forgejo-runner.service
if [ -e "$UNIT" ]; then
STATE="$(systemctl is-active forgejo-runner 2>/dev/null || true)"
SERVICE="forgejo-runner.service (${STATE:-unknown})"
else
SERVICE="(not installed as a service)"
fi
log "instance: ${INSTANCE:-unknown}"
log "name: ${RUNNER_NAME:-unknown}"
log "labels: ${LABELS}"
log "dir: ${RUNNER_DIR}"
log "service: ${SERVICE}"
forgejo_runner_liveness_note "${STATE:-}"
# status is the only command an operator runs when nothing is obviously wrong,
# which makes it the right place to notice a mode that drifted. It reports and
# does not fix: converging state is `install`'s job, and a read-only verb that
# quietly writes is a worse surprise than a loud warning.
MODE="$(stat -c '%a' "$RUNNER_DIR/.runner" 2>/dev/null || true)"
if [ -n "$MODE" ] && [ "$MODE" != "$FORGEJO_RUNNER_FILE_MODE" ]; then
warn ".runner is mode ${MODE}, not ${FORGEJO_RUNNER_FILE_MODE} — it holds this runner's registration secret, and every account on this box can read it. Re-run 'rig forgejo-runner install --instance ${INSTANCE:-<url>}' to converge the mode."
fi

47
commands/lib/admin-path.sh
View file

@ -1,47 +0,0 @@
#!/usr/bin/env bash
# admin-path.sh — assert the admin binaries are REACHABLE, not merely that we
# are root.
#
# Being uid 0 and being able to find useradd are different facts, and rig
# asserted only the first. `su` without `-`, sudo with a sanitised secure_path,
# and several container images all hand you a root shell whose PATH carries no
# /usr/sbin — which is where useradd, usermod and groupadd live on Debian. The
# result was a bare `useradd: command not found` naming a line number inside a
# versioned install root, emitted AFTER a registration token had been read off
# the operator's terminal (#139).
#
# Which binaries this covers is measured, not assumed (Debian 13, 2026-08-01):
#
# useradd usermod groupadd userdel groupdel /usr/sbin package: passwd
# visudo /usr/sbin package: sudo
# gpasswd /usr/bin package: passwd
#
# Two consequences worth keeping written down. `gpasswd` is in the same PACKAGE
# as useradd but a different DIRECTORY, so it is reachable on a PATH-shorn root
# and does not belong in any of these preflights — do not add it for symmetry.
# And `visudo` shares the directory but not the package, so its absence has a
# second, innocent cause (sudo simply not installed) that the others do not, so
# `rig users apply` checks it separately, after the point where that cause is
# ruled out. See the comment there.
#
# (Spelled without the `.sh` on purpose: test/cli.sh pins that exactly one file
# under commands/ names that script, to catch a second caller appearing. A
# comment is not a caller, but the pin is deliberately blunt and cheap.)
#
# It REFUSES rather than repairing PATH itself. A command that quietly prepends
# /usr/sbin teaches the operator nothing and leaves a misconfigured host
# misconfigured; the same reason bootstrap refuses rather than guessing. The
# message carries the fix so the refusal costs one paste, not an investigation.
# require_admin_bins <bin>... — die unless every one resolves on PATH.
require_admin_bins() {
local missing=() b
for b in "$@"; do
command -v "$b" >/dev/null 2>&1 || missing+=("$b")
done
[ "${#missing[@]}" -eq 0 ] && return 0
# Names the REMEDY, not this script: the operator typed a `rig ...` command,
# and echoing the internal path back at them is the unhelpful half of the
# original `useradd: command not found`.
die "cannot find ${missing[*]} on PATH — it lives in /usr/sbin, which this root shell does not carry (a 'su' without '-' does this, and so do some container images). Re-run the same rig command with: PATH=/usr/sbin:/sbin:\$PATH"
}

163
commands/lib/forgejo-runner-config.sh
View file

@ -1,163 +0,0 @@
#!/usr/bin/env bash
# Shared reader for the Forgejo runner's own on-disk config ($RUNNER_DIR/.runner).
# Sourced by the forgejo-runner-* commands; never executed on its own.
#
# WHY A SECOND LIB, not an arm inside lib/runner-config.sh: the two files are
# different documents making different claims, and the sibling's helpers answer
# questions this one cannot ask. GitHub's .runner names a REPOSITORY
# (gitHubUrl), so `runner install` converges toward --repo. Forgejo's names an
# INSTANCE (address) and nothing else about scope — whether a registration is
# instance-wide, org, or single-repo is a property of the TOKEN, decided in
# Forgejo's UI before rig ever sees it. There is no repo here to converge
# toward, and no way to read one back. Sharing a reader would mean a
# gitHubUrl accessor that returns empty forever on one of the two forges.
#
# json_field is deliberately re-used FROM the sibling rather than copied: a
# rig-bootstrapped box has no jq, both files are flat JSON, and one grep/sed
# reader for both is the same trade lib/runner-config.sh already argued.
HERE_FJ="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
# shellcheck source=SCRIPTDIR/runner-config.sh
. "$HERE_FJ/runner-config.sh" # json_field
# THE CREDENTIAL FACT that shapes this whole family: Forgejo's .runner holds
# the runner's own long-lived token — the secret it authenticates every poll
# with — alongside address/name/labels. GitHub's holds no such thing.
#
# So the mode is part of the contract, not hygiene: a registration secret
# readable by every account on the box is a quiet, permanent credential leak,
# and it leaks silently — nothing fails, the runner keeps working. Converge is
# the only moment rig can notice a mode that drifted (an operator's editor, a
# restore from a tarball that lost modes, a hand-edit to add a label).
FORGEJO_RUNNER_FILE_MODE=600
# forgejo_runner_instance <runner_dir> — the Forgejo instance this box's runner
# is registered to, empty when nothing is registered there.
forgejo_runner_instance() {
[ -e "$1/.runner" ] || return 0
json_field "$1/.runner" address
}
# forgejo_runner_name <runner_dir> — the runner's name, empty when unregistered.
forgejo_runner_name() {
[ -e "$1/.runner" ] || return 0
json_field "$1/.runner" name
}
# forgejo_runner_secure <runner_dir> <user> <group> — converge .runner to 0600
# owned by the runner user. Called on every install, not only at registration.
# Silent on success: this is a mode that should always already be right, and a
# line saying so on every converge would train the reader to skip it.
forgejo_runner_secure() {
local dir="$1" user="$2" group="$3"
[ -e "$dir/.runner" ] || return 0
chmod "$FORGEJO_RUNNER_FILE_MODE" "$dir/.runner"
chown "$user:$group" "$dir/.runner"
}
# runner_version_of <bin> — the bare version number ("12.13.2") the binary
# reports, empty when it cannot answer. `forgejo-runner --version` prints
# "forgejo-runner version v12.13.2"; the leading v is stripped so this compares
# against a --version argument, which has its own v stripped at parse.
#
# `|| true` for json_field's reason, which bites harder here. Callers run under
# `set -euo pipefail`, where a pipeline whose FIRST stage exits non-zero fails
# the whole pipeline — and `PRESENT_VER="$(runner_version_of "$BIN")"` is an
# assignment, so the script dies AT THAT LINE, with no message. "Empty when it
# cannot answer" is only true if this says so out loud.
#
# That is not a hypothetical shape: the binary is `[ -x ]` but unrunnable
# exactly when a ci-box's template preinstall landed a truncated or wrong-arch
# download — the one path this command family exists for. Without this, install
# exits 1 in silence and the refusal written for that case ("the download
# landed but cannot run") is unreachable code.
runner_version_of() {
"$1" --version 2>/dev/null | head -n1 \
| sed -nE 's/.*[Vv]ersion[[:space:]]+v?([0-9][0-9A-Za-z.+-]*).*/\1/p' || true
}
# runner_download_decision <have-binary yes|no> <present-ver> <wanted-ver>
# -> "install" | "skip" | "converge"
#
# A PURE function, and pure on purpose: this is the decision review !110 caught
# being wrong, and it was wrong in a way no grep could see. Lifting it out of
# the root-only install path is what makes "a pre-existing binary plus
# --version" a real test rather than a string match.
#
# The rule, and why it is not the GitHub sibling's:
#
# no binary -> install. Nothing to reason about.
# binary, no --version -> skip. Chasing "latest" on every converge would make
# a plain re-run an unrequested upgrade, and a
# convergent verb must not be a moving target.
# binary, pin matches -> skip.
# binary, pin differs -> CONVERGE, including downward. A pin is an
# instruction, not a floor.
#
# runner-install.sh skips on mere presence because actions/runner SELF-UPDATES,
# so its version moves regardless and freezing it would only make GitHub refuse
# the runner's jobs. forgejo-runner does not self-update: nothing else ever
# moves this version, and a ci-box's template preinstalls the binary at mint —
# so mere-presence here would leave --version dead on the one path this whole
# command exists to serve.
#
# An unreadable present version (empty) with a pin asked for falls to
# "converge", which is the right direction: a binary that cannot say what it is
# should be replaced by one that can.
runner_download_decision() {
local have="$1" present="$2" want="$3"
[ "$have" = yes ] || { printf 'install\n'; return 0; }
[ -n "$want" ] || { printf 'skip\n'; return 0; }
[ "$present" = "$want" ] && { printf 'skip\n'; return 0; }
printf 'converge\n'
}
# assert_runner_instance <runner_dir> <instance-url>
#
# Returns 0 when the box has no runner, or has one already registered to
# <instance-url>: re-running `install` against the instance the box is already
# on is real convergence — it re-uses the binary, skips registration, exits 0.
#
# Returns 1, explaining itself on stderr, when the runner is registered to a
# DIFFERENT instance. Skipping *that* is not convergence, it is ignoring the
# argument: `install` would skip its registration step, restart the service
# against the OLD instance, and report success — leaving the instance you asked
# for with no runner and its jobs queued against one that will never come.
#
# This is assert_runner_repo's reasoning, asked about the axis Forgejo actually
# has. There is deliberately no `repoint` sibling: Forgejo has no
# deregistration handshake to perform against the old instance, so moving a
# runner is `remove` then `install` — two acts that are already honest about
# leaving a stale entry behind, rather than one verb pretending to be atomic.
assert_runner_instance() {
local dir="$1" wanted="$2" current
[ -e "$dir/.runner" ] || return 0
current="$(forgejo_runner_instance "$dir")"
if [ -z "$current" ]; then
printf 'rig-forgejo-runner: ERROR: %s\n' \
"${dir}/.runner exists but names no instance — this box's registration cannot
be read, so rig cannot tell whether it is already on ${wanted}.
Wipe the local registration and install again:
rig forgejo-runner remove" >&2
return 1
fi
# Trailing slashes are a spelling difference, not a different instance:
# forgejo-runner records the URL as given, so `--instance https://f.example/`
# and `--instance https://f.example` would otherwise read as a move.
if [ "${current%/}" = "${wanted%/}" ]; then
return 0
fi
printf 'rig-forgejo-runner: ERROR: %s\n' \
"this box's runner is already registered to ${current}, not ${wanted}.
install will not move a runner between instances: it would leave the service
running against the OLD instance and report success. To move it, take it off
the old instance first:
rig forgejo-runner remove
then install against the new one. Forgejo has no deregistration handshake, so
the old entry stays listed until you delete it in that instance's admin UI." >&2
return 1
}

53
commands/lib/runner-config.sh
View file

@ -14,8 +14,8 @@ json_field() {
| head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//' || true | head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//' || true
} }
# json_string_array <file> <key> — the elements of the array named <key> inside # json_string_array <file> <key> — the elements of the FIRST array named <key>,
# the netmap's `Self` object, one per line; empty when Self or the key is absent. # one per line, empty when the key is absent or the array is empty.
# #
# json_field's sibling for the one shape it cannot read: `.Self.Tags` from # json_field's sibling for the one shape it cannot read: `.Self.Tags` from
# `tailscale status --json` is a JSON array, and bootstrap must assert on it to # `tailscale status --json` is a JSON array, and bootstrap must assert on it to
@ -23,44 +23,21 @@ json_field() {
# not the tag rig requested. Same grep/sed spirit, same jq-free reason: a # not the tag rig requested. Same grep/sed spirit, same jq-free reason: a
# rig-bootstrapped box has no jq and we will not install one to read one field. # rig-bootstrapped box has no jq and we will not install one to read one field.
# #
# Scoped to Self, NOT document-global. The previous body took the first "Tags" # `tr -d '\n'` first, because tailscale pretty-prints its JSON and an array
# array anywhere in the file and justified it with Self-before-Peer field order. # spans lines — grep is line-oriented and would never see `[ ... ]` whole
# That holds only when Self HAS tags: an untagged Self omits the key entirely # otherwise. `\[[^]]*\]` then captures the first flat array body for <key>
# (Go omitempty), so the match fell through into Peer and returned a PEER's tag # (tag strings never contain `]`, so this is safe); the inner `grep -o` pulls
# — silently inverting both callers on any tailnet with a tagged node (#160). # every quoted token out of it, and `sed 1d` drops the key's own name — which
# `"key":[...]` leads with — leaving just the elements.
# #
# Self is brace-counted rather than sliced to the next key: PeerStatus carries a # FIRST array wins by design, and the caller leans on it: `tailscale status
# nested object (Location, a pointer with omitempty), which would end a naive # --json` emits Self before Peer (Go struct field order, stable), so the first
# slice early whenever it is present. Known limit of staying jq-free: a `{` or # "Tags" is the node's OWN, never a peer's. An absent key omits itself entirely
# `}` inside a STRING value within Self would miscount — no PeerStatus string # (Go's omitempty) rather than emitting `[]` — which is exactly the untagged,
# field (hostnames, DNS names, OS, key strings) can contain one, so this is # user-owned node bootstrap must catch. Never fails under `set -e`+pipefail: a
# sound in practice, but it is a real assumption, written down on purpose. # non-match is a fact to test for, like json_field, not a reason to die.
#
# `tr -d '\n'` first, because tailscale pretty-prints its JSON and the object
# spans lines — awk and grep are line-oriented and would never see it whole
# otherwise. `\[[^]]*\]` then captures the flat array body for <key> (tag
# strings never contain `]`, so this is safe); the inner `grep -o` pulls every
# quoted token out of it, and `sed 1d` drops the key's own name — which
# `"key":[...]` leads with — leaving just the elements. Never fails under
# `set -e`+pipefail: a non-match is a fact to test for, not a reason to die.
json_string_array() { json_string_array() {
local self tr -d '\n' < "$1" 2>/dev/null \
self="$(tr -d '\n' < "$1" 2>/dev/null | awk '
{
i = index($0, "\"Self\"")
if (i == 0) exit
s = substr($0, i)
j = index(s, "{")
if (j == 0) exit
depth = 0
for (k = j; k <= length(s); k++) {
c = substr(s, k, 1)
if (c == "{") depth++
else if (c == "}") { depth--; if (depth == 0) { print substr(s, j, k - j + 1); exit } }
}
}')" || true
[ -n "$self" ] || return 0
printf '%s' "$self" \
| grep -o "\"$2\"[[:space:]]*:[[:space:]]*\[[^]]*\]" \ | grep -o "\"$2\"[[:space:]]*:[[:space:]]*\[[^]]*\]" \
| head -n1 | grep -o '"[^"]*"' | sed '1d; s/^"//; s/"$//' || true | head -n1 | grep -o '"[^"]*"' | sed '1d; s/^"//; s/"$//' || true
} }

437
commands/lib/templates.sh
View file

@ -1,437 +0,0 @@
#!/usr/bin/env bash
# The tenant-template REGISTRY (#110): resolve where role definitions come
# from, parse a definition's template.env against an allowlist, and lint a
# whole definition. Sourced by bootstrap-tenant.sh (the mint-time consumer)
# and template-lint.sh (the registry repo's CI gate) — pure functions plus
# one pin, no side effects at source time (repo precedent: runner-config,
# and the tenant-config table this lib replaces).
#
# The registry moved out of rig's tree so mechanism and data can move at
# different cadences (#109 is the evidence: adding kimi — pure data — meant
# editing six files here). rig keeps the mechanism and this schema; the
# definitions live in heavy-duty/rig-templates, one directory per role:
#
# <role>/template.env KEY="value" data, parsed against the allowlist
# below and NEVER sourced — a definition cannot
# execute shell through its data file
# <role>/install.sh the CLI install (the one inherently executable part)
# <role>/creds.md the per-vendor creds-free paragraph the context
# renderer splices in
#
# THE SOURCE IS FOUR KNOBS plus the installed pin snapshot, precedence
# _DIR > _REF > snapshot > pin fetch:
# RIG_TEMPLATES_DIR a local folder — bypasses the fetch entirely (the
# offline-test path, and "try a template before it
# exists anywhere")
# RIG_TEMPLATES_REF a ref in the registry repo, fetched as a tarball at
# bootstrap time (the same shape as the rig preinstall)
# RIG_TEMPLATES_REPO which repo that ref lives in (default
# heavy-duty/rig-templates)
# RIG_TEMPLATES_HOST which FORGE that repo lives on (default
# https://github.com) — see templates_archive_urls
# and, absent both overrides, the snapshot installed beside this file when it
# matches the PIN below, then a live fetch of that pin as the fallback.
# The default registry ref a mint converges — the BOX_RELEASE discipline
# (#103): one line, bumped deliberately by ordinary rig PR after review, so a
# rig release freezes the mechanism+registry pair and a newer rig matches
# newer templates by default (ruled 2026-07-24 on #110: pinned, not
# main-tracked). RIG_TEMPLATES_REF overrides it per mint.
#
# Currently the seed tree (rig-templates#1's head — fetchable from the
# upstream archive already, an ancestor of its main once merged): the four
# agent tenants ported byte-equivalent from the case arms this PR cut.
RIG_TEMPLATES_PIN=be749f7fd1ff8dd7c2359bbce7fd6abd3f403eb0
# The forge the registry lives on. GitHub by default, so every existing caller
# is byte-unchanged; overridable because a self-hosted Forgejo is a different
# origin AND a different URL grammar (#109).
RIG_TEMPLATES_HOST_DEFAULT="https://github.com"
# templates_archive_urls <host> <repo> <ref> — the source-tarball candidates
# for <ref>, in order, one per line. A PURE function (no network, no globals):
# test/cli.sh lifts it and drives it against both forges, the resolve_latest_tag
# precedent in install.sh.
#
# The two forges are not URL-compatible, and the difference is not cosmetic:
#
# GitHub three forms, refs/tags FIRST so a tag always outranks a branch
# sharing its name (a pin must win), refs/heads as the fallback
# that keeps a branch ref working, then the bare form a commit SHA
# downloads through.
# Forgejo ONE form. /archive/<ref>.tar.gz resolves tags, branches and SHAs
# alike, and the refs/{tags,heads}/ paths are not served at all —
# emitting them would mean two guaranteed 404s ahead of every fetch
# and a failure message listing URLs that never could have worked.
#
# Measured against forgejo.heavyduty.builders, not inferred from the docs.
templates_archive_urls() {
local host="${1%/}" repo="$2" ref="$3"
case "$host" in
https://github.com|http://github.com|*//github.com)
printf '%s/%s/archive/refs/tags/%s.tar.gz\n' "$host" "$repo" "$ref"
printf '%s/%s/archive/refs/heads/%s.tar.gz\n' "$host" "$repo" "$ref"
printf '%s/%s/archive/%s.tar.gz\n' "$host" "$repo" "$ref" ;;
*)
printf '%s/%s/archive/%s.tar.gz\n' "$host" "$repo" "$ref" ;;
esac
}
# The template.env schema. Grammar: blank lines, '#' comments, and
# KEY="value" — nothing else. Parsed by regex, never sourced.
TEMPLATE_KEYS_REQUIRED=(USER CONTEXT_PATH CLI_NAME PATH_LINE)
TEMPLATE_KEYS_OPTIONAL=(CLI_SRC NEEDS_NODE APT_EXTRAS)
MACHINE_KEYS_REQUIRED=(ROOT_DOOR HOST JOIN)
# templates_source_desc — where the resolved registry came from, for error
# messages and logs: a misconfigured RIG_TEMPLATES_REPO must be visible in
# the unknown-role refusal rather than looking like a typo.
# The host rides every non-snapshot description: a registry served from the
# wrong FORGE fails exactly like a misspelled repo, and naming only the repo
# would send the reader hunting for a typo that is not there (#109).
templates_source_desc() {
local host="${RIG_TEMPLATES_HOST:-$RIG_TEMPLATES_HOST_DEFAULT}"
if [ -n "${RIG_TEMPLATES_DIR:-}" ]; then
printf 'local dir %s (RIG_TEMPLATES_DIR)' "$RIG_TEMPLATES_DIR"
elif [ -z "${RIG_TEMPLATES_REF:-}" ] && templates_snapshot_usable; then
printf '%s@%s (snapshot)' \
"${RIG_TEMPLATES_REPO:-heavy-duty/rig-templates}" \
"$RIG_TEMPLATES_PIN"
else
printf '%s/%s@%s%s' \
"${host%/}" \
"${RIG_TEMPLATES_REPO:-heavy-duty/rig-templates}" \
"${RIG_TEMPLATES_REF:-$RIG_TEMPLATES_PIN}" \
"$([ -n "${RIG_TEMPLATES_REF:-}" ] && printf ' (RIG_TEMPLATES_REF)' || printf ' (the in-tree pin)')"
fi
}
# The snapshot path is derived from this library's installed tree. Its
# pin-bearing directory name is the staleness guard: an older snapshot is
# invisible after a pin bump. A usable registry has at least one definition;
# an empty directory means an interrupted extraction and falls through to the
# same live fetch as an absent snapshot.
templates_snapshot_dir() {
local lib_dir
lib_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
printf '%s/templates@%s' "$(cd "$lib_dir/../.." && pwd)" "$RIG_TEMPLATES_PIN"
}
templates_snapshot_usable() {
local snapshot role_env
snapshot="$(templates_snapshot_dir)"
[ -d "$snapshot" ] || return 1
role_env="$(find "$snapshot" -mindepth 2 -maxdepth 2 -type f -name template.env -print -quit 2>/dev/null)"
[ -n "$role_env" ]
}
# templates_resolve — resolve the knobs to a LOCAL directory holding
# the registry, left in the REGISTRY_DIR global (a global, not stdout: a
# $(…) call site would run the fetch in a subshell and lose TEMPLATES_TMP,
# the path the caller's cleanup trap must rm). RIG_TEMPLATES_DIR wins and is
# used as-is; otherwise the repo@ref tarball is fetched and extracted under
# a temp dir, recorded in TEMPLATES_TMP. Candidate URLs come from
# templates_archive_urls, which is forge-aware — see there for why the two
# forges cannot share one list.
#
# Failure lists every URL tried: the fetch is unauthenticated by contract
# (box auto-runs bootstrap at mint, holding nothing), so "is the repo public
# and the ref real" is the whole diagnosis.
#
# On a SELF-HOSTED forge there is a third way to fail that reads exactly like
# the other two, so the refusal names it: an instance with
# REQUIRE_SIGNIN_VIEW=true serves 404 for public repos to anonymous callers —
# the same status a wrong ref gets. A mint holds no credentials and never
# will, so such an instance cannot host a registry until it serves public
# repos anonymously (#109).
TEMPLATES_TMP=""
# shellcheck disable=SC2034 # REGISTRY_DIR is this function's OUTPUT, read by the sourcing script
REGISTRY_DIR=""
templates_resolve() {
local repo ref host url got=""
if [ -n "${RIG_TEMPLATES_DIR:-}" ]; then
[ -d "$RIG_TEMPLATES_DIR" ] || {
printf 'RIG_TEMPLATES_DIR is not a directory: %s\n' "$RIG_TEMPLATES_DIR" >&2
return 1
}
REGISTRY_DIR="$RIG_TEMPLATES_DIR"
return 0
fi
if [ -z "${RIG_TEMPLATES_REF:-}" ] && templates_snapshot_usable; then
REGISTRY_DIR="$(templates_snapshot_dir)"
return 0
fi
repo="${RIG_TEMPLATES_REPO:-heavy-duty/rig-templates}"
ref="${RIG_TEMPLATES_REF:-$RIG_TEMPLATES_PIN}"
host="${RIG_TEMPLATES_HOST:-$RIG_TEMPLATES_HOST_DEFAULT}"
command -v curl >/dev/null 2>&1 || { printf 'curl is required to fetch the template registry\n' >&2; return 1; }
command -v tar >/dev/null 2>&1 || { printf 'tar is required to extract the template registry\n' >&2; return 1; }
TEMPLATES_TMP="$(mktemp -d)"
while IFS= read -r url; do
if curl -fsSL "$url" -o "$TEMPLATES_TMP/templates.tar.gz" 2>/dev/null; then got="$url"; break; fi
done < <(templates_archive_urls "$host" "$repo" "$ref")
if [ -z "$got" ]; then
printf 'cannot fetch the template registry %s/%s@%s — tried:\n' "${host%/}" "$repo" "$ref" >&2
templates_archive_urls "$host" "$repo" "$ref" | sed 's/^/ /' >&2
printf 'the fetch is unauthenticated by contract (a mint holds no credentials): the repo must be public and the ref must exist. RIG_TEMPLATES_DIR=<dir> bypasses the fetch.\n' >&2
case "${host%/}" in
https://github.com|http://github.com) ;;
*) printf 'on a self-hosted forge, check the instance serves PUBLIC repos to anonymous callers too: Forgejo with REQUIRE_SIGNIN_VIEW=true answers 404 for a public repo, which is indistinguishable from a wrong ref above (set FORGEJO__service__REQUIRE_SIGNIN_VIEW=false).\n' >&2 ;;
esac
return 1
fi
tar -xzf "$TEMPLATES_TMP/templates.tar.gz" -C "$TEMPLATES_TMP" || {
printf 'cannot extract the registry tarball from %s\n' "$got" >&2
return 1
}
# A source archive holds exactly one top-level directory — assert that
# SHAPE, never the name, because the name is the forge's choice and the two
# disagree: GitHub writes <repo>-<ref> (slashes flattened), Forgejo writes
# bare <repo>. Globbing for the shape is what makes this line survive a
# forge swap untouched.
set -- "$TEMPLATES_TMP"/*/
{ [ $# -eq 1 ] && [ -d "$1" ]; } || {
printf 'the registry tarball from %s does not hold exactly one top-level directory\n' "$got" >&2
return 1
}
# shellcheck disable=SC2034 # the function's output global, read by the sourcing script
REGISTRY_DIR="${1%/}"
}
# templates_roles <registry-dir> — the roles a registry defines: its
# immediate subdirectories that carry a template.env. This list IS the
# unknown-role refusal's body, so it reflects what the resolved source
# actually contains — never a hardcoded set.
templates_roles() {
local d
for d in "$1"/*/; do
[ -f "$d/template.env" ] || continue
basename "$d"
done
}
# template_family <role> — directory names are the registry's family tag.
# workstation is the one intentional suffix-less machine role (#152 / epic D5).
template_family() {
case "$1" in
*-box) printf 'tenant\n' ;;
*-server|workstation) printf 'machine\n' ;;
*) return 1 ;;
esac
}
# templates_machine_roles <registry-dir> — only machine definitions, for the
# machine bootstrap's unknown-role refusal.
templates_machine_roles() {
local role
while IFS= read -r role; do
[ "$(template_family "$role" 2>/dev/null || true)" = "machine" ] || continue
printf '%s\n' "$role"
done < <(templates_roles "$1")
}
# template_parse_env <template.env> — parse against the allowlist. Sets
# TPL_USER, TPL_CONTEXT_PATH, TPL_CLI_NAME, TPL_CLI_SRC, TPL_PATH_LINE,
# TPL_NEEDS_NODE (default no), TPL_APT_EXTRAS. Every refusal names the
# failing key (or line): the box.env discipline — a definition is data, and
# bad data is refused loudly, never executed to find out.
# shellcheck disable=SC2034 # the TPL_* globals are this function's OUTPUT, read by the sourcing script
template_parse_env() {
local file="$1" line key val n=0 seen=" " k ok
TPL_USER="" TPL_CONTEXT_PATH="" TPL_CLI_NAME="" TPL_CLI_SRC=""
TPL_PATH_LINE="" TPL_NEEDS_NODE="no" TPL_APT_EXTRAS=""
[ -f "$file" ] || { printf 'template.env missing: %s\n' "$file" >&2; return 1; }
while IFS= read -r line || [ -n "$line" ]; do
n=$((n+1))
case "$line" in ''|'#'*) continue ;; esac
if [[ ! "$line" =~ ^([A-Z_]+)=\"(.*)\"$ ]]; then
printf 'template.env:%d: not KEY="value": %s\n' "$n" "$line" >&2
return 1
fi
key="${BASH_REMATCH[1]}" val="${BASH_REMATCH[2]}"
ok=""
for k in "${TEMPLATE_KEYS_REQUIRED[@]}" "${TEMPLATE_KEYS_OPTIONAL[@]}"; do
[ "$key" = "$k" ] && ok=1
done
[ -n "$ok" ] || { printf 'template.env:%d: unknown key: %s (allowed: %s %s)\n' \
"$n" "$key" "${TEMPLATE_KEYS_REQUIRED[*]}" "${TEMPLATE_KEYS_OPTIONAL[*]}" >&2; return 1; }
case "$seen" in *" $key "*)
printf 'template.env:%d: duplicate key: %s\n' "$n" "$key" >&2; return 1 ;;
esac
seen="$seen$key "
case "$key" in
USER) TPL_USER="$val" ;;
CONTEXT_PATH) TPL_CONTEXT_PATH="$val" ;;
CLI_NAME) TPL_CLI_NAME="$val" ;;
CLI_SRC) TPL_CLI_SRC="$val" ;;
PATH_LINE) TPL_PATH_LINE="$val" ;;
NEEDS_NODE) TPL_NEEDS_NODE="$val" ;;
APT_EXTRAS) TPL_APT_EXTRAS="$val" ;;
esac
done < "$file"
for k in "${TEMPLATE_KEYS_REQUIRED[@]}"; do
case "$seen" in *" $k "*) ;; *)
printf 'template.env: missing required key: %s\n' "$k" >&2; return 1 ;;
esac
done
# Value shapes — each refusal names its key. USER shares the charset the
# users file enforces (a leading '-' reads as a usermod flag; '|', ':'
# corrupt things downstream).
[[ "$TPL_USER" =~ ^[a-z_][a-z0-9_-]{0,31}$ ]] \
|| { printf 'template.env: USER: invalid user name: %s (want ^[a-z_][a-z0-9_-]{0,31}$)\n' "$TPL_USER" >&2; return 1; }
case "$TPL_CONTEXT_PATH" in
/*|*..*|'') printf 'template.env: CONTEXT_PATH: must be relative to the tenant home, without "..": %s\n' "$TPL_CONTEXT_PATH" >&2; return 1 ;;
esac
[[ "$TPL_CLI_NAME" =~ ^[a-z0-9][a-z0-9._-]*$ ]] \
|| { printf 'template.env: CLI_NAME: not a sane command name: %s\n' "$TPL_CLI_NAME" >&2; return 1; }
# A literal '~/' on purpose (SC2088): the value is DATA — the mechanism
# expands it to the tenant home by string substitution, never the shell.
# shellcheck disable=SC2088
case "$TPL_CLI_SRC" in
*..*) printf 'template.env: CLI_SRC: must not contain "..": %s\n' "$TPL_CLI_SRC" >&2; return 1 ;;
''|'~/'*|/*) ;;
*) printf 'template.env: CLI_SRC: must be absolute or ~/-relative: %s\n' "$TPL_CLI_SRC" >&2; return 1 ;;
esac
case "$TPL_NEEDS_NODE" in
yes|no) ;;
*) printf 'template.env: NEEDS_NODE: want yes or no, got: %s\n' "$TPL_NEEDS_NODE" >&2; return 1 ;;
esac
[ -n "$TPL_PATH_LINE" ] \
|| { printf 'template.env: PATH_LINE: must not be empty\n' >&2; return 1; }
# Every word must be a sane package name — the list is handed to apt-get
# unquoted by design, and this is what keeps an option ('-o …') or a path
# from riding in through the data file.
local pkg
for pkg in $TPL_APT_EXTRAS; do
[[ "$pkg" =~ ^[a-z0-9][a-z0-9.+-]*$ ]] \
|| { printf 'template.env: APT_EXTRAS: not a sane package name: %s\n' "$pkg" >&2; return 1; }
done
}
# machine_template_parse_env <template.env> — the fleet-machine traits schema.
# The globals match bootstrap's table columns so a definition becomes a table
# row without changing any downstream trait behavior.
# shellcheck disable=SC2034
machine_template_parse_env() {
local file="$1" line key val n=0 seen=" " k ok
TPL_ROOT_DOOR="" TPL_HOST="" TPL_JOIN=""
[ -f "$file" ] || { printf 'template.env missing: %s\n' "$file" >&2; return 1; }
while IFS= read -r line || [ -n "$line" ]; do
n=$((n+1))
case "$line" in ''|'#'*) continue ;; esac
if [[ ! "$line" =~ ^([A-Z_]+)=\"(.*)\"$ ]]; then
printf 'template.env:%d: not KEY="value": %s\n' "$n" "$line" >&2
return 1
fi
key="${BASH_REMATCH[1]}" val="${BASH_REMATCH[2]}"
ok=""
for k in "${MACHINE_KEYS_REQUIRED[@]}"; do
[ "$key" = "$k" ] && ok=1
done
[ -n "$ok" ] || {
printf 'template.env:%d: unknown key: %s (allowed: %s)\n' \
"$n" "$key" "${MACHINE_KEYS_REQUIRED[*]}" >&2
return 1
}
case "$seen" in *" $key "*)
printf 'template.env:%d: duplicate key: %s\n' "$n" "$key" >&2
return 1 ;;
esac
seen="$seen$key "
case "$key" in
ROOT_DOOR) TPL_ROOT_DOOR="$val" ;;
HOST) TPL_HOST="$val" ;;
JOIN) TPL_JOIN="$val" ;;
esac
done < "$file"
for k in "${MACHINE_KEYS_REQUIRED[@]}"; do
case "$seen" in *" $k "*) ;; *)
printf 'template.env: missing required key: %s\n' "$k" >&2
return 1 ;;
esac
done
case "$TPL_ROOT_DOOR" in
open|closed) ;;
*) printf 'template.env: ROOT_DOOR: want open or closed, got: %s\n' "$TPL_ROOT_DOOR" >&2; return 1 ;;
esac
case "$TPL_HOST" in
yes|no) ;;
*) printf 'template.env: HOST: want yes or no, got: %s\n' "$TPL_HOST" >&2; return 1 ;;
esac
case "$TPL_JOIN" in
authkey|login) ;;
*) printf 'template.env: JOIN: want authkey or login, got: %s\n' "$TPL_JOIN" >&2; return 1 ;;
esac
}
# render_tenant_context <role> <creds.md> — the agent-context file's
# content, on stdout: the one file every agent reads before touching
# anything. The skeleton is MECHANISM and lives here once — the box#80 guard
# note ("never run box setup-host or the drill inside a box; the box you are
# in is not a host you own") must never be copy-pasted per template again —
# and only the creds paragraph is per-vendor DATA, spliced in from the
# definition's creds.md.
render_tenant_context() {
local role="$1" creds_file="$2"
cat <<EOF
# You are running inside a box (tenant: ${role})
A box is a trust-less, network-isolated, ephemeral VM created by the
\`box\` CLI. Keep this context in mind:
$(cat "$creds_file")
- **Isolated.** The box reaches the public internet but nothing on the host or
local network. There is no inbound path.
- **Disposable.** Nothing here is backed up. State is discarded when the box is
removed; the operator persists work via git push and via \`box snapshot\`.
- **Not a host you own.** Never run \`box setup-host\`, \`box teardown-host\`,
or the drill inside a box. The box you are in is not a host you own: a
nested box stack claims the guest's own uplink subnet and gateway, and
silently breaks this box's networking with intermittent egress blackouts
(heavy-duty/box#80). Working ON the box repo from in here is fine — editing
and testing never needs the host stack; host setup belongs to the operator's
machine, never this one.
- **Bootstrap runbook.** If the repository you are working in contains a
\`.box/\` folder (older repos may use \`.claudebox/\`), read it as your setup
runbook — how to install dependencies, start services, template environment
files, seed data, and smoke-test — and follow it. It is documentation for
you, not a script the host runs.
EOF
}
# template_lint <role-dir> — the whole-definition check the registry repo's
# CI runs on every PR (rig defines what a valid template is; rig-templates
# CI enforces it, so a broken definition is refused before it can reach a
# mint). Same parser the mint runs — the two gates are not redundant: CI
# protects the registry, the mint-time parse protects a mint served through
# RIG_TEMPLATES_REPO/_DIR that CI never saw.
template_lint() {
local dir="${1%/}" role family
role="$(basename "$dir")"
[ -d "$dir" ] || { printf '%s: not a directory\n' "$dir" >&2; return 1; }
family="$(template_family "$role" 2>/dev/null || true)"
[ -n "$family" ] || {
printf '%s: role directories carry a family suffix (-box for box tenants, -server for fleet machines — rig#76; workstation is #152 machine carve-out)\n' "$role" >&2
return 1
}
if [ "$family" = "tenant" ]; then
template_parse_env "$dir/template.env" || return 1
[ -s "$dir/install.sh" ] \
|| { printf '%s: install.sh missing or empty\n' "$role" >&2; return 1; }
head -n1 "$dir/install.sh" | grep -q '^#!' \
|| { printf '%s: install.sh has no shebang\n' "$role" >&2; return 1; }
grep -q '[^[:space:]]' "$dir/creds.md" 2>/dev/null \
|| { printf '%s: creds.md missing or blank (the context renderer splices it in — a blank paragraph would ship a context file with a hole)\n' "$role" >&2; return 1; }
else
machine_template_parse_env "$dir/template.env" || return 1
[ ! -e "$dir/creds.md" ] \
|| { printf '%s: creds.md is not allowed for machine roles (machines render no tenant context)\n' "$role" >&2; return 1; }
if [ -e "$dir/install.sh" ]; then
[ -s "$dir/install.sh" ] \
|| { printf '%s: install.sh is empty\n' "$role" >&2; return 1; }
head -n1 "$dir/install.sh" | grep -q '^#!' \
|| { printf '%s: install.sh has no shebang\n' "$role" >&2; return 1; }
fi
fi
return 0
}

92
commands/lib/tenant-config.sh Normal file
View file

@ -0,0 +1,92 @@
#!/usr/bin/env bash
# Shared parameters for the box TENANT roles (claude-box, codex-box, grok-box,
# staging-box) — the '-box' suffix names the FAMILY (a guest a box mints, vs the
# '-server' machine roles rig converges); see #76.
# sourced by bootstrap-tenant.sh and by the test harness. Pure text→text, no
# side effects: the per-tenant differences live HERE, in one table, so the
# mechanism stays one script parameterized per tenant instead of four
# hand-maintained copies (repo precedent: parse_users_file, runner-config).
# tenant_user <role> — the user the box seed creates (box.env BOX_USER). The
# agent tenants are named after their agent (minus the suffix — the USER is not
# the role); staging-box keeps box#69's `ops`.
tenant_user() {
case "$1" in
claude-box) printf 'claude' ;;
codex-box) printf 'codex' ;;
grok-box) printf 'grok' ;;
staging-box) printf 'ops' ;;
*) return 1 ;;
esac
}
# tenant_context_path <role> <home> — where the agent-context file lands. Each
# agent CLI reads its own instructions file from its own dotdir (named for the
# agent, not the role — the dotdir is the CLI's, and the suffix is rig's);
# staging-box has no agent and no context file (return 1).
tenant_context_path() {
case "$1" in
claude-box) printf '%s/.claude/CLAUDE.md' "$2" ;;
codex-box) printf '%s/.codex/AGENTS.md' "$2" ;;
grok-box) printf '%s/.grok/AGENTS.md' "$2" ;;
*) return 1 ;;
esac
}
# render_tenant_context <role> — the agent-context file's content, on stdout.
# One renderer for all three agents: only the creds paragraph is per-vendor,
# and the box#80 guard note lives HERE once — never copy-pasted per template.
# staging-box renders nothing (return 1): no agent lives there.
render_tenant_context() {
local role="$1" creds
# The single-quoted markdown below carries literal `$`-free backtick prose;
# single quotes are deliberate — nothing in it may expand here.
# shellcheck disable=SC2016
case "$role" in
claude-box)
creds='- **Creds-free by default.** The box starts with no Claude and no git
credentials. If you need to authenticate Claude, the operator runs `/login`
interactively. For git, the operator adds their own credentials (a PAT or
`gh auth login`). Never assume credentials are present; never ask for or
store secrets on disk beyond what the operator sets up.' ;;
codex-box)
creds='- **Creds-free by default.** The box starts with no OpenAI and no git
credentials. If you need to authenticate Codex, the operator runs the
login flow (`codex`) interactively. For git, the operator adds their own
credentials (a PAT or `gh auth login`). Never assume credentials are
present; never ask for or store secrets on disk beyond what the operator
sets up.' ;;
grok-box)
creds='- **Creds-free by default.** The box starts with no xAI and no git
credentials. If you need to authenticate, the operator runs
`grok login` interactively (SuperGrok / X Premium+). For git, the
operator adds their own credentials (a PAT or `gh auth login`). Never
assume credentials are present; never ask for or store secrets on disk
beyond what the operator sets up.' ;;
*) return 1 ;;
esac
cat <<EOF
# You are running inside a box (tenant: ${role})
A box is a trust-less, network-isolated, ephemeral VM created by the
\`box\` CLI. Keep this context in mind:
${creds}
- **Isolated.** The box reaches the public internet but nothing on the host or
local network. There is no inbound path.
- **Disposable.** Nothing here is backed up. State is discarded when the box is
removed; the operator persists work via git push and via \`box snapshot\`.
- **Not a host you own.** Never run \`box setup-host\`, \`box teardown-host\`,
or the drill inside a box. The box you are in is not a host you own: a
nested box stack claims the guest's own uplink subnet and gateway, and
silently breaks this box's networking with intermittent egress blackouts
(heavy-duty/box#80). Working ON the box repo from in here is fine — editing
and testing never needs the host stack; host setup belongs to the operator's
machine, never this one.
- **Bootstrap runbook.** If the repository you are working in contains a
\`.box/\` folder (older repos may use \`.claudebox/\`), read it as your setup
runbook — how to install dependencies, start services, template environment
files, seed data, and smoke-test — and follow it. It is documentation for
you, not a script the host runs.
EOF
}

77
commands/platform.sh
View file

@ -2,8 +2,7 @@
# rig platform — what is this machine? Calculated at run time, stored nowhere. # rig platform — what is this machine? Calculated at run time, stored nowhere.
# #
# Read-only in the strongest sense rig has: it reads /proc, uname, # Read-only in the strongest sense rig has: it reads /proc, uname,
# /etc/os-release, /etc/machine-id, df and systemd-detect-virt, and writes # /etc/os-release, df and systemd-detect-virt, and writes NOTHING, ever. That
# NOTHING, ever. That
# is the design, not an implementation detail — specs change without rig doing # is the design, not an implementation detail — specs change without rig doing
# anything (RAM added, root disk resized, unattended-upgrades patching the # anything (RAM added, root disk resized, unattended-upgrades patching the
# kernel), so a stored spec is stale the moment the machine changes, and # kernel), so a stored spec is stale the moment the machine changes, and
@ -25,19 +24,13 @@ usage() {
cat <<'EOF' cat <<'EOF'
usage: rig platform usage: rig platform
Describes the machine you are on: hostname, a stable machine ID, OS, kernel, Describes the machine you are on: hostname, OS, kernel, CPU, memory, disk
CPU, memory, disk and virtualization, then rig's own provenance (which rig, and virtualization, then rig's own provenance (which rig, when, and the role
when, and the role marker bootstrap wrote). marker bootstrap wrote).
ID names the machine where HOSTNAME names the slot: it is derived from Computed at run time from /proc, uname, /etc/os-release, df and
/etc/machine-id (a namespaced sha256, never the raw value, which machine-id(5) systemd-detect-virt. Writes nothing, needs no root, makes no network call —
asks tools not to expose). Two machines reporting the same ID were cloned so it also works on a pristine Debian box rig has never bootstrapped, where
from one image — actionable information, not a coincidence: no identity that
lives in the filesystem survives the filesystem being copied.
Computed at run time from /proc, uname, /etc/os-release, /etc/machine-id, df
and systemd-detect-virt. Writes nothing, needs no root, makes no network call
— so it also works on a pristine Debian box rig has never bootstrapped, where
the provenance block reads 'not bootstrapped'. the provenance block reads 'not bootstrapped'.
EOF EOF
} }
@ -60,61 +53,6 @@ field() { printf '%-10s %s\n' "$1" "$2"; }
# before anything has been installed. # before anything has been installed.
HOSTNAME_V="$(hostname 2>/dev/null || uname -n)" HOSTNAME_V="$(hostname 2>/dev/null || uname -n)"
# --- identity (#95) -----------------------------------------------------------
# HOSTNAME names the slot; ID names the machine. rig itself sets the hostname
# during bootstrap and reuses it across rebuilds ('hetzner-cp-1' is a role, not
# hardware), so nothing above answers "is this the same machine I converged in
# June, or its replacement?". /etc/machine-id does — but machine-id(5) asks
# that the raw value not be exposed (it is a stable correlator across every
# tool that leaks it), and its documented remedy is an application-specific
# derivation. So: THE PINNED DERIVATION, fixed by #95 so two implementations
# can never disagree —
#
# printf 'rig-machine-id:%s' "$(cat /etc/machine-id)" | sha256sum
# → first 32 hex chars, rendered 8-4-4-4-12
#
# The 'rig-machine-id:' prefix is the contract, not decoration: it is what
# keeps this id uncorrelatable with any other tool's derivation of the same
# machine-id. sha256sum is coreutils, which this command is restricted to.
# Derived, computed here, stored nowhere — #64's thesis — so it exists before
# bootstrap and needs no write path.
#
# What this deliberately does NOT fix: a host cloned from a golden image
# carries the clone's /etc/machine-id, so two machines reporting the same ID
# means a cloned image. That is surfaced (help text, README) rather than
# defended against — no identity that lives in the filesystem survives the
# filesystem being copied.
#
# RIG_MACHINE_ID overrides the path so the harness can drive the present,
# absent, empty and uninitialized cases against fixtures (repo precedent:
# RIG_MANIFEST / RIG_ROLE_MARKER below).
MID_FILE="${RIG_MACHINE_ID:-/etc/machine-id}"
ID_V=""
if [ ! -r "$MID_FILE" ]; then
# Never an empty string: an ID field that renders blank looks like a bug,
# and a missing file is a fact worth naming.
ID_V="unavailable (no $MID_FILE)"
else
# $(...) strips the trailing newline — that is part of the pinned derivation
# above, not an accident of shell.
MID="$(cat "$MID_FILE")"
if [ -z "$MID" ]; then
# NEVER a hash of nothing: hashing the empty string would hand every such
# machine the SAME id — the worst possible failure for an identity field.
# Images do ship the file empty (that is first-boot semantics per
# machine-id(5)), so this path is real, not defensive.
ID_V="unavailable ($MID_FILE is empty)"
elif [ "$MID" = "uninitialized" ]; then
# machine-id(5)'s other not-yet-set sentinel — same collision failure as
# empty if hashed, so same loud degradation.
ID_V="unavailable ($MID_FILE is uninitialized)"
else
MID_HASH="$(printf 'rig-machine-id:%s' "$MID" | sha256sum)"
MID_HASH="${MID_HASH%% *}"
ID_V="${MID_HASH:0:8}-${MID_HASH:8:4}-${MID_HASH:12:4}-${MID_HASH:16:4}-${MID_HASH:20:12}"
fi
fi
# --- OS --------------------------------------------------------------------- # --- OS ---------------------------------------------------------------------
# THE os-release TRAP: /etc/os-release defines VERSION, NAME and ID, so # THE os-release TRAP: /etc/os-release defines VERSION, NAME and ID, so
# sourcing it in the MAIN shell silently clobbers same-named script variables. # sourcing it in the MAIN shell silently clobbers same-named script variables.
@ -174,7 +112,6 @@ VIRT="$(systemd-detect-virt 2>/dev/null || true)"
printf '%s\n' "PLATFORM" printf '%s\n' "PLATFORM"
field HOSTNAME "$HOSTNAME_V" field HOSTNAME "$HOSTNAME_V"
field ID "$ID_V"
field OS "${OS:-unknown}" field OS "${OS:-unknown}"
field KERNEL "$KERNEL" field KERNEL "$KERNEL"
field CPU "${CPU_MODEL:-unknown}${CORES:+ ($CORES cores)}" field CPU "${CPU_MODEL:-unknown}${CORES:+ ($CORES cores)}"

6
commands/runner-install.sh
View file

@ -9,8 +9,6 @@ set -euo pipefail
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)" HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
# shellcheck source=SCRIPTDIR/lib/runner-config.sh # shellcheck source=SCRIPTDIR/lib/runner-config.sh
. "$HERE/lib/runner-config.sh" . "$HERE/lib/runner-config.sh"
# shellcheck source=SCRIPTDIR/lib/admin-path.sh
. "$HERE/lib/admin-path.sh"
log() { printf 'rig-runner: %s\n' "$*"; } log() { printf 'rig-runner: %s\n' "$*"; }
warn() { printf 'rig-runner: WARNING: %s\n' "$*" >&2; } warn() { printf 'rig-runner: WARNING: %s\n' "$*" >&2; }
@ -87,10 +85,6 @@ VERSION="${VERSION#v}"
# --- guards ---------------------------------------------------------------- # --- guards ----------------------------------------------------------------
[ "$(id -u)" -eq 0 ] || die "must run as root" [ "$(id -u)" -eq 0 ] || die "must run as root"
# Root is not enough: the admin binaries must also be reachable (#139). This
# sits beside the root check so identity and capability are asserted together,
# and BEFORE any prompt or download — a token typed for a doomed run is waste.
require_admin_bins useradd
if [ -r /etc/os-release ]; then if [ -r /etc/os-release ]; then
# Sourced in a subshell: os-release defines VERSION (e.g. "13 (trixie)"), # Sourced in a subshell: os-release defines VERSION (e.g. "13 (trixie)"),
# which would clobber this script's $VERSION. # which would clobber this script's $VERSION.

55
commands/template-lint.sh
View file

@ -1,55 +0,0 @@
#!/usr/bin/env bash
# rig template-lint <role-dir>... — is this a valid role definition?
#
# rig defines what a valid template is (the schema lives in
# lib/templates.sh, beside the mint-time parser that enforces it); the
# heavy-duty/rig-templates repo's CI runs this on every definition on every
# PR, so a broken definition is refused before it can ever reach a mint
# (#110). The two gates are deliberate: CI protects the registry, the
# mint-time parse protects a mint served through RIG_TEMPLATES_REPO/_DIR
# that CI never saw.
#
# Pure read: no root, no network, no writes — lintable anywhere, including
# the registry repo's checkout, where rig's tree is only a fetched tool.
set -euo pipefail
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
# shellcheck source=SCRIPTDIR/lib/templates.sh
. "$HERE/lib/templates.sh" # template_lint (and the schema it enforces)
die() { printf 'rig-template-lint: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
usage() {
cat <<'EOF'
usage: rig template-lint <role-dir>...
Validate role definitions (the heavy-duty/rig-templates shape).
Tenant roles use a *-box directory, tenant template.env schema, a shebang
install.sh, and non-blank creds.md. Machine roles use a *-server directory
(or exact name workstation), the ROOT_DOOR/HOST/JOIN schema, no creds.md,
and an optional install.sh which must be non-empty and carry a shebang.
template.env is parsed as KEY="value" data and never sourced. Every refusal
names the failing key or file. Exits non-zero if any definition fails;
nothing is written.
EOF
}
case "${1:-}" in
-h|--help) usage; exit 0 ;;
"") usage >&2; die "at least one role directory required" 2 ;;
esac
fail=0
for dir in "$@"; do
case "$dir" in
-*) usage >&2; die "unknown flag: $dir" 2 ;;
esac
if template_lint "$dir"; then
printf 'rig-template-lint: OK: %s\n' "$dir"
else
printf 'rig-template-lint: FAIL: %s\n' "$dir" >&2
fail=1
fi
done
exit "$fail"

26
commands/users-apply.sh
View file

@ -11,8 +11,6 @@ set -euo pipefail
HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)" HERE="$(cd "$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")" && pwd)"
# shellcheck source=SCRIPTDIR/lib/users-config.sh # shellcheck source=SCRIPTDIR/lib/users-config.sh
. "$HERE/lib/users-config.sh" . "$HERE/lib/users-config.sh"
# shellcheck source=SCRIPTDIR/lib/admin-path.sh
. "$HERE/lib/admin-path.sh"
log() { printf 'rig-users: %s\n' "$*"; } log() { printf 'rig-users: %s\n' "$*"; }
warn() { printf 'rig-users: WARNING: %s\n' "$*" >&2; } warn() { printf 'rig-users: WARNING: %s\n' "$*" >&2; }
@ -138,10 +136,6 @@ done <<< "$PARSED"
# --- guards ------------------------------------------------------------------ # --- guards ------------------------------------------------------------------
[ "$(id -u)" -eq 0 ] || die "must run as root" [ "$(id -u)" -eq 0 ] || die "must run as root"
# Root is not enough: the admin binaries must also be reachable (#139). This
# sits beside the root check so identity and capability are asserted together,
# and BEFORE any prompt or download — a token typed for a doomed run is waste.
require_admin_bins useradd usermod groupadd
# Identity management gates its INVOKER, not just its uid: %rig's sudoers rule # Identity management gates its INVOKER, not just its uid: %rig's sudoers rule
# is binary-scoped but not argument-scoped, so without this gate a rig-role # is binary-scoped but not argument-scoped, so without this gate a rig-role
@ -201,26 +195,6 @@ if [ "$NEED_SUDO" -eq 1 ] && ! command -v sudo >/dev/null 2>&1; then
DEBIAN_FRONTEND=noninteractive apt-get install -y -qq sudo DEBIAN_FRONTEND=noninteractive apt-get install -y -qq sudo
CHANGED=1 CHANGED=1
fi fi
# visudo is checked HERE and not beside the root check, because until the block
# above has run there is a legitimate reason for it to be absent: sudo is not
# installed yet, and apply is what installs it. Above, a missing visudo would
# be indistinguishable from that, so the refusal would fire on a healthy box.
#
# Below, it is unambiguous. sudo is present, so `visudo` missing means only one
# thing: /usr/sbin is off PATH. And it MUST refuse here rather than be left to
# the sudoers block further down, because that block asks `command -v visudo`
# and treats false as "no sudo on the box means no role needed it" — which on a
# PATH-shorn root is FALSE TWICE. A role does need it, sudo is installed, and
# apply would finish reporting success having silently never written the
# sudoers drop-in: the users get their roles and not the escalation the roles
# are FOR. That is the failure this whole issue is about (a wrong effective
# state reported as success, #12), in its quietest form — the other three sites
# at least crash. Refusing before the first mutation is what keeps it loud.
#
# It sits before `groupadd` below, so nothing has been converged when it fires.
if [ "$NEED_SUDO" -eq 1 ]; then
require_admin_bins visudo
fi
# --- groups ------------------------------------------------------------------ # --- groups ------------------------------------------------------------------
groupadd -f rig-admin groupadd -f rig-admin

319
docs/plans/2026-07-27-forgejo-ci-box.md
View file

@ -1,319 +0,0 @@
# Forgejo-native CI: a `ci-box` tenant and a `forgejo-runner` command family
Status: proposed (#109)
Date: 2026-07-27
## The shape of the thing
A fleet machine hosts boxes. One of those boxes is the CI box. The Forgejo
runner lives inside it, and CI jobs run in containers on that box's own
dockerd.
```
+-----------------------------+ outbound HTTPS (long-poll)
| Forgejo (Coolify) | <---------------------------------+
| forgejo.heavyduty.builders | |
+-----------------------------+ |
+------------------------------+---+
No inbound path to the box. | staging-server (host=yes) |
The runner polls out. | +---------------------------+ |
| | ci-box (tenant) | |
| | forgejo-runner (systemd)| |
| | dockerd -> job containers| |
| +---------------------------+ |
+----------------------------------+
```
Three pieces, in two repos.
## 1. A forge-agnostic registry fetch (`commands/lib/templates.sh`)
`templates_resolve` builds three candidate URLs, all on `github.com`. A
registry hosted anywhere else cannot be fetched.
The fix is a fourth knob beside `_DIR` / `_REF` / `_REPO`:
```sh
RIG_TEMPLATES_HOST the forge origin (default https://github.com)
```
The candidate list becomes the host's fact rather than a constant, because the
two forges genuinely differ:
| | GitHub | Forgejo |
|---|---|---|
| Candidates | `archive/refs/tags/<ref>.tar.gz`, `archive/refs/heads/<ref>.tar.gz`, `archive/<ref>.tar.gz` | `archive/<ref>.tar.gz` — one form, which resolves tags, branches and SHAs alike |
| Archive top-level dir | `<repo>-<ref>` | `<repo>` |
Both were measured against `forgejo.heavyduty.builders`, not assumed.
The existing "exactly one top-level directory" assert survives untouched — it
globs `*/` rather than reconstructing the name, so the differing directory name
costs nothing. Its **comment** was wrong for Forgejo and is corrected.
`templates_source_desc` grows the host, so a misconfigured origin shows up in
the unknown-role refusal instead of reading like a typo. The GitHub default
means every existing caller behaves exactly as before.
`install.sh`'s `snapshot_templates` duplicates the same candidate list for its
install-time cache. It gets the same knob, from the same environment variable,
so the snapshot and the live fetch cannot disagree about where the registry is.
### The blocker this exposed — since cleared
**Resolved 2026-07-27, after this was written.** The operator set
`REQUIRE_SIGNIN_VIEW=false`, and the mint-time path was then verified live: a
credential-less `templates_resolve` with
`RIG_TEMPLATES_HOST=https://forgejo.heavyduty.builders` fetches and extracts a
real archive. Piece 1 is proven end to end on the target instance, not merely
argued.
The requirement below stands for *any* instance hosting a registry, and the
refusal text still names it — it is a property of the mint's creds-free
contract, not of one server's configuration. What follows is the original
analysis, kept because it is why the knob exists.
### The blocker, as originally measured
`templates_resolve` documents a hard contract:
> the fetch is unauthenticated by contract (box auto-runs bootstrap at mint,
> holding nothing)
Measured: on `forgejo.heavyduty.builders`, anonymous requests for
`heavy-duty/rig` — reported by the API as `private: false` — return **404** for
the API, the web page, the git remote and the archive endpoint. Only an
authenticated request succeeds. The instance requires sign-in to view.
A mint holds no credentials, so **a Forgejo-hosted registry is unreachable at
mint time** until the instance serves public repos anonymously:
```
FORGEJO__service__REQUIRE_SIGNIN_VIEW=false
```
This is a Coolify env-var change on the Forgejo service, next to the
`FORGEJO__actions__ENABLED=true` that Actions already needs. It is recorded
here and in the README as a prerequisite. Nothing in this change silently
assumes it: with the gate up, the fetch fails the way any unreachable ref
fails, listing every URL tried.
## 2. The `ci-box` tenant definition
Data, not mechanism — it belongs in the registry repo. It is staged in this PR
under `docs/templates/ci-box/` so it can be reviewed and linted here, and moves
to the registry verbatim once that repo exists on Forgejo.
```
USER="ci"
CONTEXT_PATH=".ci/CONTEXT.md"
CLI_NAME="forgejo-runner"
CLI_SRC="/usr/local/bin/forgejo-runner"
PATH_LINE="export PATH="$HOME/.local/bin:$PATH""
NEEDS_NODE="no"
```
`CLI_SRC` is absolute rather than `~/`-relative, which is the one place this
definition departs from the agent tenants. Their CLI lives in the tenant's
home; this binary is executed by a systemd unit, and a tenant-writable binary
that a root-installed unit runs is a trivial path to root inside the box. So it
lands root-owned under `/usr/local/bin`. `test/cli.sh` pins `CLI_SRC` against
the path `install.sh` actually writes — a drifted pair converges to a CLI that
exists and cannot run, which is the scar `grok-box` left.
`NEEDS_NODE="no"` because the runner is a static Go binary. Jobs get their Node
from the container image, which is the whole point of the label mapping below —
installing a second Node on the host would be a toolchain nobody reads.
`install.sh` fetches the release binary for the architecture and **verifies the
published `.sha256` before installing it**. Forgejo ships bare binaries with
`.sha256` and `.asc` beside them rather than a tarball, so a checksum is
available for free; taking it makes the install auditable in the way
`coolify install`'s version pin is.
This satisfies the tenant schema honestly rather than by paperwork:
`bootstrap-tenant.sh` asserts `<CLI> --version` answers **as the tenant user**,
and `forgejo-runner --version` does.
`creds.md` states the box holds no Forgejo credential and that registration is
a separate, operator-run act — which is true, and is what the rendered context
file needs to say.
### Why the box replaces docker-in-docker
The setup guide this design came from builds a `docker:dind` sidecar with
`privileged: true` and a plaintext `tcp://…:2375` daemon socket. Inside a
tenant that is redundant: `bootstrap-tenant.sh` already installs Docker on
every tenant and adds the tenant user to the `docker` group. The runner talks
to that daemon over its own socket.
The isolation argument that justifies dind on a shared CI server is already
paid for here by the box: it is network-isolated, disposable, and has no
inbound path. Stacking dind inside it would add a privileged container to buy
a boundary that already exists.
Note the trade this makes explicit: `docker` group membership is
root-equivalent *within the box*. `rig runner install` refuses Docker for
exactly that reason — but it converges a fleet **machine**, where the blast
radius is the machine. Here the blast radius is a guest that is thrown away.
That is the difference that makes the same trade correct in one place and wrong
in the other.
## 3. `rig forgejo-runner install|status|remove`
A new family beside `rig runner`, which is left untouched.
```sh
box shell ci-box
sudo rig forgejo-runner install \
--instance https://forgejo.heavyduty.builders \
--name ci-runner-1
```
Default labels:
```
ubuntu-latest:docker://ghcr.io/catthehacker/ubuntu:act-22.04,docker:docker://node:22-bookworm
```
so `runs-on: ubuntu-latest` works in a workflow written for GitHub.
### Why not `rig runner --forge forgejo`
GitHub's runner registers against a repository URL, and
`assert_runner_repo` converges toward `--repo`: re-running against the same
repo is a no-op, a different repo is refused because silently restarting on the
old one would leave the requested repo with jobs queued forever.
Forgejo has no such argument. The runner registers against an **instance**, and
whether the registration is instance-wide, org, or single-repo is a property of
the *token*, decided in Forgejo's UI before rig ever sees it. There is no repo
to converge toward and nothing to compare.
Folding that into one command would make every guard bimodal to share a flag
name, while the contract underneath differs. Two families is the smaller lie.
### `register` is deprecated upstream, and is still the right call
Measured on v12.13.2: both `forgejo-runner register` and `create-runner-file`
carry `(deprecated)` in their help. The successor is
`daemon --url <instance> --uuid <uuid> --token-url <url>`, which requires the
runner to **already exist on the instance** — the operator creates it via the
API or UI and carries a UUID back.
`register` is chosen anyway:
- It still works, and this was verified rather than assumed. A planted
`.runner` made `daemon` resolve the instance from the file, connect to a live
Forgejo, and fail with `Unauthenticated: unregistered runner` — the transport
and the file format are intact; only the credential was fake.
- The successor asks the operator for a second, differently-shaped credential
dance for no gain today.
- `register` writes a file `status` can read back. Under the successor, "what
is this box registered to" has no on-disk answer that is not merely rig's own
copy of what it was told.
When upstream removes it: the unit line gains `--url`/`--uuid`, and the readers
in `forgejo-runner-config.sh` move to whatever holds the UUID.
`assert_runner_instance`'s contract survives either spelling, because it asks
about the instance — which both record.
Not verified here, and left to the drill: that `register` writes `.runner` on a
live instance. That needs a real registration token, which this design cannot
mint.
### The convergence guard that does apply
`assert_runner_instance`: re-running against the instance this box is already
registered to re-uses the binary and skips registration; a **different**
instance refuses and names both. Same trust-boundary reasoning as
`assert_runner_repo`, asked about the axis Forgejo actually has.
### Three corrections from review !110
The first draft carried three defects that review caught, all of the same
family — a stated contract that the code did not actually keep:
1. **`--version` was swallowed on the primary path.** The download block
skipped on mere presence, copying `rig runner install`'s shape without its
justification: `actions/runner` self-updates, `forgejo-runner` does not. And
a ci-box's template *preinstalls* the binary at mint, so the flag documented
as the deterministic-pin lever could never fire on a ci-box. The decision is
now `runner_download_decision` in the lib — a pure function, so the rule is
driven by tests instead of asserted by grep — and the binary is renamed into
place rather than written over, since the converge path now runs while the
daemon is live (in-place would be `ETXTBSY`).
2. **`.rig-labels` outlived the registration it described.** The write had
escaped the registration branch, where `runner-install.sh` correctly keeps
its copy. A plain re-run stamped that invocation's labels over a
registration made with different ones, and `status` then reported
confidently wrong labels while Forgejo held the originals. It is scoped
again, and an *explicit* `--labels` on a re-run now warns that Forgejo owns
labels from registration time.
3. **The checksum gate failed open.** A missing `.sha256` warned and installed
anyway — contradicting both the README and this file's own comment about
unverified root downloads. The original reasoning (do not let an upstream
layout change break installs) reasons about the wrong failure: a layout
change breaks the *binary* URL too, so "binary yes, checksum no" is the
shape of an interfered fetch, which is what the checksum is for. Both paths
refuse now, with no bypass flag.
### `.runner` holds a credential here
GitHub's `.runner` names a repo. Forgejo's holds `address`, `name`, `labels`
**and the runner's own long-lived token** — the credential it authenticates
with on every poll.
So this family does something its GitHub sibling never had to: it installs
`.runner` as `0600` owned by the runner user, and **re-asserts that mode on
every converge**. A registration secret readable by every account on the box
would be a quiet, permanent credential leak, and convergence is the only moment
rig can notice a mode that drifted.
`status` therefore reads `address`, `name` and `labels` and never prints the
token.
### Removal
Forgejo's runner has no deregistration handshake — no removal-token endpoint,
no `config.sh remove`. `remove` stops and disables the unit and wipes the local
registration, then says plainly that the entry must be deleted in Forgejo's
admin UI. `rig runner remove`'s `--local` escape hatch is the *only* mode here,
so it is not offered as a flag that suggests a server-side alternative exists.
## Guard: `bootstrap --undo`
`bootstrap-undo.sh` refuses to leave the tailnet while a GitHub runner is
installed, so undo cannot strand a ghost runner in a repository. The same
hazard exists for a Forgejo runner on a machine, so the guard learns the
`forgejo-runner.service` unit and the `.runner` under the runner user's home.
## Testing
`test/cli.sh` is dependency-free, non-root, offline. What it can prove:
- Every new command's arg validation, `--help`, and unknown-flag refusals.
- `bin/rig` dispatch, including bare `rig forgejo-runner` showing usage.
- The forge-aware URL builder as a **pure function**, lifted and driven
directly: GitHub host yields the three-candidate list in tag-first order,
a Forgejo host yields the single `/archive/<ref>.tar.gz`.
- `templates_source_desc` names a non-default host.
- The staged `ci-box` definition passes `rig template-lint` — the same parser
a mint runs, so the definition cannot ship malformed.
- Grep-pins that the `.runner` 0600 assert and the `bootstrap --undo`
forgejo-runner guard are present, so a deleted guard cannot ship green
(the repo's existing precedent for guards that need a real machine).
What it cannot prove, and is left to the drill: a real registration against a
live Forgejo, and a job actually executing in a container.
## Out of scope
`install.sh` hardcodes `github.com` in `resolve_latest_tag` and
`ref_candidate_urls` (rig's own source), and `commands/bootstrap.sh` fetches
box from `raw.githubusercontent.com`. Hosting rig itself on Forgejo needs those
too. They are follow-ups, not this change — `snapshot_templates` is included
here only because it fetches *the registry*, and leaving it behind would let
the snapshot and the live fetch disagree about where the registry lives.

25
docs/templates/README.md vendored
View file

@ -1,25 +0,0 @@
# Staged template definitions
Definitions here are **not** the registry. They are role definitions destined
for `heavy-duty/rig-templates`, staged in rig's tree only while the registry
repo does not yet exist on the forge that will serve them (#109).
This is deliberately a waiting room, not a second registry:
- `bootstrap-tenant.sh` does not look here. Nothing in this directory is
reachable by a mint, and adding a lookup would recreate exactly the coupling
the registry split was written to remove — where adding a tenant meant
editing rig.
- `test/cli.sh` lints each one with `rig template-lint`, the same parser a mint
runs. A definition that cannot pass the schema never reaches the registry.
- When the registry repo exists, a definition moves there **verbatim** and is
deleted from here in the same PR.
To try one before it is anywhere, point a mint at it directly:
```sh
RIG_TEMPLATES_DIR=docs/templates rig bootstrap ci-box
```
That is `RIG_TEMPLATES_DIR`'s stated purpose — "try a template before it exists
anywhere" — and it is the supported path, not a workaround.

13
docs/templates/ci-box/creds.md vendored
View file

@ -1,13 +0,0 @@
- **Creds-free by default.** The box starts with no Forgejo credentials and no
git credentials. The runner binary is installed but **not registered**:
registration needs a token the operator mints in Forgejo (Site
Administration, org, or repo → Actions → Runners) and hands to
`sudo rig forgejo-runner install --instance <url>`. rig never writes that
token to disk — but it does **not expire, and registering does not spend
it**. A Forgejo registration token stays valid until somebody mints a
replacement at that same scope, and it will register as many runners as it is
shown to. Treat a leaked one as live until it has been replaced. (GitHub's
equivalent expires in about an hour; do not carry that habit across.) After
registration the runner's own credential lives in `~/forgejo-runner/.runner`,
mode 0600 — never copy it, print it, or commit it. Secrets that CI jobs need
belong in Forgejo's repo or org secrets, injected per job, not on this box.

129
docs/templates/ci-box/install.sh vendored
View file

@ -1,129 +0,0 @@
#!/usr/bin/env bash
# ci-box — the forgejo-runner binary. Run BY THE MECHANISM as root, with
# TENANT_USER/TENANT_HOME/TENANT_GROUP/ROLE exported.
#
# This lands the BINARY ONLY. Registration is deliberately not here: it needs a
# registration token from the Forgejo instance, and a tenant install is
# creds-free by contract — box auto-runs it at mint, holding nothing. The
# operator registers afterwards, out loud:
#
# Deliberately NOT described the way the GitHub sibling describes its own
# registration token, which really does expire in about an hour. That adjective
# must not cross this forge boundary — a test pins its absence from this file
# and from creds.md, so do not reintroduce it by copying from `rig runner`.
# Forgejo's ActionRunnerToken carries no expiry field at all;
# NewRunnerToken invalidates prior tokens only when a replacement is minted at
# the same scope, and Register leaves the one it was handed active. It is
# reusable until replaced, so a leak stays live. See creds.md, which is the
# copy an agent inside the box actually reads.
#
# box shell ci-box
# sudo rig forgejo-runner install --instance https://forgejo.example.com
#
# Same split as staging-box's tailnet join, for the same reason.
#
# Root-owned under /usr/local/bin rather than the tenant's home: unlike an
# agent CLI, this binary is run by a systemd unit as the tenant user, and a
# tenant-writable binary that root's unit executes is a trivial path to root
# inside the box.
set -euo pipefail
# fetch_and_verify_sha256 <asset-url> <file> <sumfile> <label>
#
# The whole checksum POLICY, in one place: fetch the published .sha256 beside
# an asset and prove the download matches it. Prints the reason on stderr and
# returns 1 on any failure; the caller supplies the refusal in its own voice.
#
# BYTE-IDENTICAL to the copy in commands/forgejo-runner-install.sh, diffed by
# test/cli.sh — the valid_version / templates_archive_urls precedent. The two
# downloaders cannot share a lib: that one sources commands/lib/, and this one
# is a REGISTRY DEFINITION that runs standalone inside a mint from a fetched
# tarball, with rig's tree nowhere in reach. So the pin is the only mechanism
# that keeps one policy from becoming two.
#
# Review !110 is the evidence for why that matters: a fail-open branch lived in
# BOTH copies while a grep for "checksum mismatch" passed against both, because
# the string it looked for sat right beside the branch it could not see. The
# next checksum-policy change must not be able to land in one file only.
#
# AN UNFETCHABLE CHECKSUM REFUSES — it is a gate, not a courtesy. The earlier
# reasoning ("do not let an upstream layout change break installs") reasons
# about the wrong failure: a layout change moves the BINARY url too, so the
# download would already have died. "Binary yes, checksum no" is not what a
# layout change looks like — it is what an interfered fetch looks like, which
# is precisely what a checksum exists to catch. Failing open would hand an
# unverified root install to anyone able to block a single URL. There is
# deliberately no bypass flag: if upstream really does move its assets, that is
# a rig PR editing the URL, not an operator improvising past a security gate.
fetch_and_verify_sha256() {
local url="$1" file="$2" sumfile="$3" label="$4" want got
if ! curl -fsSL "${url}.sha256" -o "$sumfile" 2>/dev/null; then
printf 'no published .sha256 for %s at %s.sha256 — the binary itself downloaded, so this is not an upstream layout change; check what is intercepting the fetch\n' "$label" "$url" >&2
return 1
fi
# The published .sha256 names the asset, not our temp path. Compare the
# digest itself rather than rewriting the file into sha256sum -c's format:
# one comparison, no parsing of a file we did not write.
want="$(tr -d '\r' < "$sumfile" 2>/dev/null | awk '{print $1}' | head -n1)"
got="$(sha256sum "$file" | awk '{print $1}')"
if [ -z "$want" ]; then
printf 'the published checksum for %s is unreadable — a fetch that succeeds but returns nothing usable is not a verified download\n' "$label" >&2
return 1
fi
if [ "$want" != "$got" ]; then
printf 'checksum mismatch for %s: published %s, downloaded %s\n' "$label" "$want" "$got" >&2
return 1
fi
printf 'checksum verified (%s)\n' "$got"
}
# CIBOX_BIN is a TEST-ONLY override, in the same spirit as bootstrap-undo.sh's
# RIG_FORGEJO_RUNNER_DIR: the production default is the only path the mechanism
# ever uses, but test/cli.sh must be able to drive this script on a box that
# already has a real runner installed. Without it the early-exit below fires
# against the host and the checksum checks silently test nothing (#136).
BIN="${CIBOX_BIN:-/usr/local/bin/forgejo-runner}"
if [ -x "$BIN" ]; then
exit 0
fi
case "$(uname -m)" in
x86_64) ARCH="amd64" ;;
aarch64) ARCH="arm64" ;;
*) echo "ci-box install: unsupported arch: $(uname -m)" >&2; exit 1 ;;
esac
# The latest release, resolved by following the releases/latest redirect — no
# API call, no token, no JSON to parse on a dependency-free guest. A pinned
# version belongs to `rig forgejo-runner install --version`, which is where an
# operator who needs a deterministic install already is; a pin baked into the
# registry would go stale in a repo nobody watches.
LATEST_URL="$(curl -fsSLI -o /dev/null -w '%{url_effective}' \
https://code.forgejo.org/forgejo/runner/releases/latest)" \
|| { echo "ci-box install: could not resolve the latest forgejo-runner release" >&2; exit 1; }
VERSION="${LATEST_URL##*/}"
VERSION="${VERSION#v}"
case "$VERSION" in
""|*[!0-9.]*) echo "ci-box install: could not parse a version from ${LATEST_URL}" >&2; exit 1 ;;
esac
ASSET="forgejo-runner-${VERSION}-linux-${ARCH}"
URL="https://code.forgejo.org/forgejo/runner/releases/download/v${VERSION}/${ASSET}"
WORKDIR="$(mktemp -d)"
cleanup() { rm -rf "$WORKDIR"; }
trap cleanup EXIT
echo "ci-box install: downloading forgejo-runner ${VERSION} (${ARCH})"
curl -fsSL "$URL" -o "$WORKDIR/forgejo-runner" \
|| { echo "ci-box install: could not download ${URL}" >&2; exit 1; }
# Forgejo publishes a .sha256 beside each binary. Verifying it costs one
# request and makes the install auditable; this file executes as root inside
# every future mint, so an unverified download is the last thing it should do.
fetch_and_verify_sha256 "$URL" "$WORKDIR/forgejo-runner" "$WORKDIR/forgejo-runner.sha256" "$ASSET" \
|| { echo "ci-box install: refusing to install an unverified ${ASSET} — it lands as root inside every mint. See the checksum failure above." >&2; exit 1; }
install -m 0755 -o root -g root "$WORKDIR/forgejo-runner" "$BIN"
echo "ci-box install: installed ${BIN}"

16
docs/templates/ci-box/template.env vendored
View file

@ -1,16 +0,0 @@
# ci-box — the Forgejo CI tenant (#109). The box guest a fleet machine mints to
# run CI jobs: forgejo-runner polls the instance outbound, jobs run in
# containers on the dockerd bootstrap-tenant.sh already installed.
#
# The schema is rig's commands/lib/templates.sh; this is data, parsed and never
# sourced.
USER="ci"
CONTEXT_PATH=".ci/CONTEXT.md"
CLI_NAME="forgejo-runner"
CLI_SRC="/usr/local/bin/forgejo-runner"
PATH_LINE="export PATH="$HOME/.local/bin:$PATH""
# NEEDS_NODE is a question about the CLI, not about the jobs. forgejo-runner is
# a static Go binary, and every job gets its own toolchain from its container
# image — a Node on the host would be a second, invisible toolchain that no
# workflow reads.
NEEDS_NODE="no"

114
drill/README.md
View file

@ -1,114 +0,0 @@
# The drill — running it
`drill/drill.sh` is the instrument; `drills/` is the record it feeds
(see [drills/README.md](../drills/README.md) for what a record means and
how the three repos' drills relate). rig's drill asserts **convergence**:
a machine reaches its role, idempotently. This file is the procedure —
written down so a run is repeatable, not reconstructed from memory each
release (#105, and #107's debt).
## What you need
- **A throwaway Debian 13 machine** you can format, reached as root. The
drill hardens its sshd, renames it, joins it to a tailnet, and installs
box/Incus, Coolify and Actions runners on it. It is not coming back.
The machine is its own reset — there is no teardown script and no need
for one.
- **The pinned candidate refs, both of them.** `--rig-ref` and
`--box-ref` are required; the harness refuses to run without them and
refuses to continue if what installed disagrees with what was asked
(`INSTALLED_FROM`, both trees). Since heavy-duty/rig#103 landed, both
installers have sane defaults when unpinned — box installs the
`BOX_RELEASE` pin (currently `0.9.0`), rig's `install.sh` resolves the
latest release — and a sane default is exactly why the drill will not
let a ref go unstated: an unpinned run silently drills a shipping pair
that is not the candidate, and the record it leaves looks clean.
- **A single-use, tagged tailscale pre-auth key** in `TS_AUTHKEY`
(`tag:local` for the default `staging-server` role — bootstrap refuses
`tag:server` outside the control-plane shapes).
- **A users file** (`--users`) naming at least one operator — leg 1
asserts the accounts and keys actually converged.
- **For leg 3** (runner lifecycle): a fork to register against
(`--runner-repo you/rig`) carrying a `workflow_dispatch` workflow —
default name `drill.yml` — whose job has `runs-on: [self-hosted, drill]`
and does something trivial (`echo drilled`). Tokens come from an
authenticated `gh`, or from `RUNNER_TOKEN` / `RUNNER_REMOVE_TOKEN`.
Without a fork the leg **skips, loudly, into the record**.
- **For leg 3's Forgejo half** (#129): `--forgejo-instance <url>` and
`--forgejo-runner-repo <owner>/<repo>`, where that repo carries the same
`workflow_dispatch` workflow — but with `runs-on: drill`, because a
Forgejo runner matches the bare label it registered with. Tokens come from
`FORGEJO_RUNNER_TOKEN` (a registration token) or `FORGEJO_API_TOKEN`, which
mints one and is also what dispatches the job. `--forgejo-ref` names the
branch to dispatch (default `main`): Forgejo's dispatch endpoint requires a
ref in the body, where GitHub's defaults to the repo's default branch.
Without an instance and a repo this half **skips, loudly and separately**.
Note there is no removal token — Forgejo has no deregistration endpoint, so
the leg removes locally and the record tells you to delete the stale runner
row by hand.
- **For leg 4** (coolify): a version pin, `--coolify-version 4.1.2`.
No pin, no leg — rig's own `coolify install` refuses to default a
version and so does its drill. The skip is recorded.
- **A run ID** (`--run-id`) when this drill shares a substrate with
box's or cast's — the shared ID is what lets the per-repo records be
joined afterwards. Defaults to `drill-<date>`.
## Running it
From a checkout of this repo on the throwaway machine (the record lands
in the checkout's `drills/`):
```sh
TS_AUTHKEY=tskey-... bash drill/drill.sh \
--rig-ref release/0.4.0 --box-ref 0.9.0 \
--users ./drill-users --run-id drill-2026-07-24-a \
--coolify-version 4.1.2 --runner-repo you/rig --yes
```
`--box-ref` is a tag on purpose: since #103 the box that ships is the
`BOX_RELEASE` tag, so a `release/…` branch is the wrong thing to pin for
box — while a release branch stays exactly right for rig's own candidate.
It runs unattended from there. Legs execute as 1, 4, 2, 3 — Coolify's
installer is what puts Docker on the box and the db leg needs a daemon —
and the record lists them as they ran. A failing check never aborts the
run (`set -u`, no `-e`: a failing check is data), and the summary counts
passes, failures and skips separately.
## What it asserts
1. **Convergence, and idempotence.** `rig bootstrap <role> --users …`
reaches the declared role, asserted on *effective* state — the marker,
`sshd -T`, the granted tailnet tag, the operators' accounts and keys.
Then bootstrap runs **again**, and the state captured before and after
the re-run must diff **empty**. The diff is mechanical; "watched it
not obviously break" is exactly what this leg exists to replace.
Riding along, the `--host yes` assertions: the **pinned** box
installed (`INSTALLED_FROM` matches `--box-ref`, fatal if not),
`box doctor` passes. It stops there and says so in the output — the
isolation boundary is **box's** drill's assertion, never rig's.
2. **db** — `test/db-integration.sh` from the *installed* tree: a real
dump/restore round-trip. Its clean-skip contract (no Docker → loud
skip, exit 0) survives into the record as a SKIP, never a pass.
3. **Runner lifecycle** — register against the fork, dispatch the drill
workflow and watch the runner take it, deregister, and assert the
box's registration is actually gone. Runs **once per forge**: `rig runner`
against GitHub, then `rig forgejo-runner` against a Forgejo instance
(#129). Both families ship, so a release that evidences only one
evidences half of what it ships; each half skips separately, so a record
can honestly show one forge drilled and the other not.
4. **Coolify** — installed at the pin, `AUTOUPDATE=false` landed in the
effective `.env`, container running.
## The record
The run always ends by writing `drills/<version>.md` (the version is the
installed tree's own `VERSION`) — on failures too: **a failed drill is a
valid record**; the gate wants evidence, not success. Skipped legs are
named as not-run so the record can never read as a clean sweep. Commit
the file on the release branch; the `drill-recorded` guard reads that
file and nothing else.
The instrument's own honesty — the refusals, the skip accounting, the
capture-and-diff, the emitter — is `test/drill.sh`'s job, and CI runs it
on every PR. The live four-leg run is a release's job, once per cycle.

983
drill/drill.sh
View file

@ -1,983 +0,0 @@
#!/usr/bin/env bash
# drill/drill.sh — rig's release drill: the instrument behind drills/README.md.
#
# ⚠ DESTRUCTIVE, AND MEANT TO BE. Run it on a THROWAWAY Debian machine you
# can format. It wipes any installed rig and reinstalls from the pinned
# ref, hardens sshd, sets the hostname, joins the tailnet, installs box
# and its Incus stack, installs Coolify and Actions runners (GitHub, and
# Forgejo when --forgejo-instance is given).
# Never run it on a machine you care about.
#
# TS_AUTHKEY=tskey-... bash drill/drill.sh \
# --rig-ref release/0.4.0 --box-ref 0.9.0 \
# --users ./drill-users --run-id drill-2026-07-24-a \
# --coolify-version 4.1.2 --runner-repo you/rig \
# --forgejo-instance https://forgejo.example.com \
# --forgejo-runner-repo you/drill-probe --yes
# (--box-ref is a tag: since #103 the box that ships is the BOX_RELEASE tag.)
# rig's drill asserts CONVERGENCE — a machine reaches its role, idempotently.
# The legs (drills/README.md, issue #105):
#
# 1. convergence + idempotence — `rig bootstrap <role> --users <path>`
# reaches the declared role; a re-run produces an EMPTY state diff,
# mechanically, never by eye. Rides along: the --host yes assertions
# (the pinned box installed, its host stack stands — and it STOPS there;
# the isolation boundary is box's drill's assertion, not this one's).
# 2. db — the real dump/restore round-trip, test/db-integration.sh.
# 3. runner lifecycle — register, take a job, deregister, against a fork.
# Runs once per forge: `rig runner` against GitHub (--runner-repo), and
# `rig forgejo-runner` against a Forgejo instance (--forgejo-instance +
# --forgejo-runner-repo). Both forges ship, so both need evidence; each
# skips loudly and separately when its inputs are absent (#129).
# 4. coolify install — at a pinned version, AUTOUPDATE=false.
#
# Execution order is 1, 4, 2, 3 — coolify's installer is what puts Docker on
# the box, and leg 2 needs a daemon; running db before coolify would skip a
# leg this same run makes runnable. The record lists legs as they ran.
#
# Exit 0 = no check failed. A FAILED drill still emits a complete record —
# the gate wants evidence, not success — and skipped legs are counted and
# named, never folded into the passes (heavy-duty/box#153's defect class).
#
# The file is one long 'probe && ok "…" || no "…"'. ok/no always return 0, so
# the C-may-run-when-A-is-true trap SC2015 warns about cannot fire here.
# shellcheck disable=SC2015
#
# NOT -e: a failing check is data, not a crash — a drill that aborts on its
# first failure reports one problem per afternoon. NOT pipefail: checks of the
# 'refusal 2>&1 | grep -q text' shape have a left side that exits non-zero BY
# DESIGN, and 'grep -q' SIGPIPEs the left side on early match — box's first
# live run turned both into false FAILs. The pipeline verdict must be grep's
# alone. (box drill/drill.sh's header, the discipline #105 prescribes.)
set -u
SELF="$(readlink -f "$0")"
ROOT="$(cd "$(dirname "$SELF")/.." && pwd)"
REPO="${RIG_REPO:-heavy-duty/rig}"
REF="${RIG_REF:-}"
BOXREPO="${BOX_REPO:-heavy-duty/box}"
BOXREF="${BOX_REF:-}"
# The template registry the converge will read (#110). No explicitness
# demand here, unlike the two refs above: the DEFAULT is already a pin — the
# candidate tree's RIG_TEMPLATES_PIN, read after install from what actually
# landed — so an unset override means "the ref the release will really use",
# not "whatever main was that afternoon".
TPLREPO="${RIG_TEMPLATES_REPO:-heavy-duty/rig-templates}"
TPLREF="${RIG_TEMPLATES_REF:-}"
TPL_SHA=""
TPL_SOURCE="fetched"
ROLE=staging-server
USERS_FILE="${DRILL_USERS_FILE:-}"
RUN_ID="${DRILL_RUN_ID:-drill-$(date -u +%F)}"
RECORD="${DRILL_RECORD:-}"
COOLIFY_VERSION="${DRILL_COOLIFY_VERSION:-}"
RUNNER_REPO="${DRILL_RUNNER_REPO:-}"
RUNNER_WORKFLOW="${DRILL_RUNNER_WORKFLOW:-drill.yml}"
FJ_INSTANCE="${DRILL_FORGEJO_INSTANCE:-}"
FJ_RUNNER_REPO="${DRILL_FORGEJO_RUNNER_REPO:-}"
# The branch the dispatch names. Forgejo's dispatch endpoint requires a ref in
# the body — unlike GitHub's, which defaults to the repo's default branch.
FJ_REF="${DRILL_FORGEJO_REF:-main}"
YES=0
while [ $# -gt 0 ]; do
case "$1" in
--yes|-y) YES=1; shift ;;
--rig-repo) REPO="$2"; shift 2 ;;
--rig-ref) REF="$2"; shift 2 ;;
--box-repo) BOXREPO="$2"; shift 2 ;;
--box-ref) BOXREF="$2"; shift 2 ;;
--role) ROLE="$2"; shift 2 ;;
--users) USERS_FILE="$2"; shift 2 ;;
--run-id) RUN_ID="$2"; shift 2 ;;
--record) RECORD="$2"; shift 2 ;;
--coolify-version) COOLIFY_VERSION="$2"; shift 2 ;;
--runner-repo) RUNNER_REPO="$2"; shift 2 ;;
--runner-workflow) RUNNER_WORKFLOW="$2"; shift 2 ;;
--forgejo-instance) FJ_INSTANCE="$2"; shift 2 ;;
--forgejo-runner-repo) FJ_RUNNER_REPO="$2"; shift 2 ;;
--forgejo-ref) FJ_REF="$2"; shift 2 ;;
-h|--help) sed -n '2,40p' "$0" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) echo "drill: unknown option: $1 (see --help)" >&2; exit 2 ;;
esac
done
# --- the reporting verbs (box drill/drill.sh:52-58, the parts worth copying) --
# ok/no/skip/note always return 0: the body stays one long sequence of
# 'probe && ok || no' without fighting the shell. SKIP is its own verb and its
# own counter — a leg that did not run must be visually and arithmetically
# distinct from one that passed (box#153's defect class: a silent skip reads
# as a pass in the record, months later).
pass=0; fail=0; skipped=0; findings=()
ok() { printf ' \033[32mPASS\033[0m %s\n' "$*"; pass=$((pass + 1)); }
no() { printf ' \033[31mFAIL\033[0m %s\n' "$*"; fail=$((fail + 1)); findings+=("FAIL: $*"); }
skip() { printf ' \033[35mSKIP\033[0m %s\n' "$*"; skipped=$((skipped + 1)); findings+=("SKIP: $*"); }
note() { printf ' \033[33mNOTE\033[0m %s\n' "$*"; findings+=("NOTE: $*"); }
inf() { printf ' %s\n' "$*"; }
phase(){ printf '\n\033[1m══ %s\033[0m\n' "$*"; }
# The record's leg table, appended as legs run. One row per leg, result text
# written at the moment the leg's verdict is known — never reconstructed from
# memory at the end (an invented number is worse than no number).
LEG_NAMES=(); LEG_RESULTS=()
leg() { LEG_NAMES+=("$1"); LEG_RESULTS+=("$2"); }
# forgejo_run_verdict <pre_id> <tasks-json> — the verdict for OUR dispatch:
# success | failed | pending. Reads GET /repos/{o}/{r}/actions/tasks, whose
# shape is NOT GitHub's and was measured against forgejo.heavyduty.builders
# (8.0.3+gitea-1.22.0) on 2026-07-30 rather than read from the docs:
#
# * There is no `conclusion` field. `status` carries the terminal outcome
# directly ("success"), where GitHub splits status:completed +
# conclusion:success. Reading `conclusion` here gets an empty string on
# every run, which would grade a green job as failed.
# * `id` is a GLOBAL task id; the run's own URL ends in `run_number`. The
# pre-dispatch guard therefore compares `id`, exactly as the GitHub leg
# compares databaseId — an old run must never be read as this one.
# * The payload lists ASSIGNED tasks only. A run sitting queued is simply
# absent (measured: 200s of total_count 0 while the web UI showed the run
# as "job is not started"). So "no new id" is the ONLY signal that the
# runner never took the job — which is the verdict this leg exists for.
# * A task appears when ASSIGNED, so it can be seen mid-flight. A
# non-terminal status is pending, not failed; grading a running job as a
# failure would make the leg flaky inside its own watch window.
#
# EVERY entry is inspected, and the NEWEST id above pre_id decides — never
# entry[0]. actions/tasks accumulates, so the moment a repo is drilled twice
# our run shares the payload with older ones, and nothing documents the sort
# order. Reading the first entry made a green job report as a timeout, a false
# FAILURE on the gate this leg exists to provide (grok/kimi on !130).
#
# grep-and-sed, not jq: a throwaway drill machine has neither jq nor an
# authenticated forge CLI, the same constraint json_field() carries in
# commands/lib/runner-config.sh. `id` is a bare number, which json_field's
# quoted-value shape cannot read, so this reads both forms itself. Newlines are
# stripped first so a pretty-printed payload parses identically to a compact
# one — the instance documents neither.
forgejo_run_verdict() {
local pre="$1" file="$2" obj id best_id="" best_st=""
[ -r "$file" ] || { echo pending; return 0; }
while IFS= read -r obj; do
[ -n "$obj" ] || continue
id="$(printf '%s' "$obj" | grep -o '"id"[[:space:]]*:[[:space:]]*[0-9][0-9]*' \
| head -n1 | sed 's/.*:[[:space:]]*//')"
[ -n "$id" ] || continue
# Strictly newer than the pre-dispatch id. Equal is the run that was
# already there; lower is older still.
if [ -n "$pre" ]; then
[ "$id" -gt "$pre" ] 2>/dev/null || continue
fi
if [ -z "$best_id" ] || [ "$id" -gt "$best_id" ] 2>/dev/null; then
best_id="$id"
best_st="$(printf '%s' "$obj" | grep -o '"status"[[:space:]]*:[[:space:]]*"[^"]*"' \
| head -n1 | sed 's/.*:[[:space:]]*"//; s/"$//')"
fi
done <<EOF
$(tr -d '\n' < "$file" 2>/dev/null | grep -o '{[^{}]*}')
EOF
[ -n "$best_id" ] || { echo pending; return 0; }
case "$best_st" in
success) echo success ;;
failure | cancelled | skipped | timedout) echo failed ;;
*) echo pending ;;
esac
}
# forgejo_leg_row <install_ok> <status_ok> <took> <remove_ok> <absent_ok>
# The record row for the Forgejo runner leg. PASS requires the WHOLE lifecycle,
# not just the take-a-job outcome.
#
# Keying the row on <took> alone let it read "PASS — registered, took a job,
# removed" when install had failed, because the dispatched job only needs
# SOMETHING answering runs-on: drill — and this leg removes locally, telling the
# operator to delete the stale runner by hand, so a leftover drill-labeled
# runner from the previous drill is the designed-for aftermath rather than a
# contrived case (codex/grok/kimi on !130). drills/<v>.md is the release's
# durable evidence; a row claiming a lifecycle that did not happen is exactly
# what the gate exists to refuse.
#
# The drill's exit code was never wrong here — every one of those failures also
# called `no`. What was wrong is the row, and the row is what outlives the run.
forgejo_leg_row() {
local install_ok="$1" status_ok="$2" took="$3" remove_ok="$4" absent_ok="$5"
if [ "$install_ok" != 1 ] || [ "$status_ok" != 1 ] \
|| [ "$remove_ok" != 1 ] || [ "$absent_ok" != 1 ]; then
echo "FAIL — see Failed below"
return 0
fi
case "$took" in
success) echo "PASS — registered, took a job, removed (stale row needs deleting by hand)" ;;
none) echo "PARTIAL — registered and removed; took a job: not attempted (no FORGEJO_API_TOKEN)" ;;
*) echo "FAIL — see Failed below" ;;
esac
}
# forgejo_max_task_id <tasks-json> — the highest numeric task id in the payload,
# empty when there is none. This is the PRE-DISPATCH baseline, and it must fold
# max exactly as forgejo_run_verdict does: taking the first id instead names an
# OLD run as the baseline whenever the payload is not newest-first (the order is
# undocumented). A later poll that finds the same body then reads the PREVIOUS
# drill's run as this dispatch's result — a false PASS on the take-a-job
# assertion, which is worse than the false FAIL the same mistake caused inside
# the verdict (grok/kimi, !130). The GitHub leg is safe from this only because
# `gh run list --limit 1` contracts newest-first; this API contracts nothing.
forgejo_max_task_id() {
local file="$1" id best=""
[ -r "$file" ] || return 0
while IFS= read -r id; do
[ -n "$id" ] || continue
if [ -z "$best" ] || [ "$id" -gt "$best" ] 2>/dev/null; then best="$id"; fi
done <<EOF
$(tr -d '\n' < "$file" 2>/dev/null \
| grep -o '"id"[[:space:]]*:[[:space:]]*[0-9][0-9]*' | sed 's/.*:[[:space:]]*//')
EOF
printf '%s\n' "$best"
}
# forgejo_token_verdict <resolved_token> <api_token> — ok | mint-failed | no-source.
# #129's acceptance: "Token source present but the instance is unreachable ->
# the leg FAILS; it must not skip and must not pass". A mint that yields
# nothing — unreachable instance, under-scoped token, wrong repo — is a
# CONFIGURED leg failing, and reporting it as "no token source" both writes
# SKIPPED where the record owes a FAIL and sends the operator to check an env
# var they already set. Absent inputs are the only honest skip.
forgejo_token_verdict() {
if [ -n "$1" ]; then echo ok
elif [ -n "$2" ]; then echo mint-failed
else echo no-source
fi
}
# run_logged <log> <cmd...> — run a long command with its narration in a file
# and a dot every 5s on the terminal: a silent multi-minute apt/install run is
# indistinguishable from a wedge, and that ambiguity has cost box whole
# evenings. Returns the command's exit code.
run_logged() {
local log="$1"; shift
inf "watch it live in another terminal: tail -f $log"
"$@" >"$log" 2>&1 </dev/null &
local pid=$!
while kill -0 "$pid" 2>/dev/null; do printf '.'; sleep 5; done
printf '\n'
wait "$pid"
}
# tree_of <cli-path> — the versioned install tree a CLI's symlink chain lands
# in. Both rig and box install as <root>/versions/<v>/bin/<cli> behind a
# 'current' link, so the tree is two dirnames above the resolved binary —
# derived from the chain itself, never from a hardcoded install root (root vs
# user installs put the root in different places).
tree_of() {
local real
real="$(readlink -f "$1" 2>/dev/null)"
# -e as well as -n: GNU readlink -f resolves a path whose LAST component
# does not exist (exit 0), so a dangling link would hand back a tree that
# is not there.
{ [ -n "$real" ] && [ -e "$real" ]; } || return 1
dirname "$(dirname "$real")"
}
# assert_installed_from <what> <tree> <want> — ASSERT WHAT LANDED, never trust
# that the install obeyed. An installer invoked with stale env vars silently
# falls back to its defaults — sane ones since rig#103 landed (box: the
# BOX_RELEASE pin, rig: the latest release), which is what makes the fallback
# invisible — and a drill that thinks it exercised release/X but actually got
# whatever the defaults resolve to has proven nothing about the combination
# that ships — worse than one that fails, because the record it leaves LOOKS
# like evidence. Refusal names both refs, per #105's acceptance criteria.
assert_installed_from() {
local what="$1" tree="$2" want="$3" got
got="$(cat "$tree/INSTALLED_FROM" 2>/dev/null || echo '<unreadable>')"
if [ "$got" != "$want" ]; then
printf 'drill: FATAL — asked to install %s from %s, but the installed tree says %s.\n' "$what" "$want" "$got" >&2
printf ' (tree: %s)\n' "$tree" >&2
printf ' A drill that silently drills the wrong code is worse than one that fails:\n' >&2
printf ' every result below would describe a tree that is not the candidate. Check\n' >&2
printf ' the env this drill inherited (a stale RIG_REF/BOX_REF export), fix the\n' >&2
printf ' pin, and re-run.\n' >&2
return 1
fi
return 0
}
# classify_leg <rc> <outfile> — pass | skip | fail. The skip contract is
# test/db-integration.sh's, copied carefully: it skips CLEANLY (exit 0) with a
# 'skip: <reason>' line when it cannot run, so exit code alone reads a
# not-run leg as a pass. The reason line is the verdict's tiebreaker; a
# non-zero exit is a fail whatever the output says (a die after a skip line
# would be a broken harness, not a skip).
classify_leg() {
local rc="$1" out="$2"
if [ "$rc" -eq 0 ] && grep -q '^skip:' "$out" 2>/dev/null; then
printf 'skip'
elif [ "$rc" -eq 0 ]; then
printf 'pass'
else
printf 'fail'
fi
}
# capture_state <outfile> — the convergent surface bootstrap owns, as one
# diffable text file. Leg 1's idempotence claim is decided by capturing this
# BEFORE and AFTER the re-run and diffing — mechanically, because idempotence
# is the single easiest property to convince yourself of by eye (#105).
#
# What is captured is what bootstrap CONVERGES, nothing that legitimately
# moves between two back-to-back runs: no package lists (unattended-upgrades
# may act between captures), no clocks. The manifest is included WHOLE on
# purpose — lib/manifest.sh's contract is that a same-version re-run renders
# byte-identical content (converged_at tracks the version, not the run), so
# the diff ENFORCES that contract instead of exempting it.
#
# Every path is overridable so test/drill.sh proves the capture-and-diff
# machinery against fixtures, without root (repo precedent: RIG_ROLE_MARKER,
# RIG_MANIFEST). Absent files and commands degrade to a deterministic
# '(absent)' — a capture must never fail, only describe.
capture_state() {
local out="$1" marker manifest ledger autoup hosts u state home keys
marker="${RIG_ROLE_MARKER:-/etc/rig/role}"
manifest="${RIG_MANIFEST:-/etc/rig/manifest}"
ledger="${DRILL_LEDGER:-/etc/rig/users}"
autoup="${DRILL_AUTOUPGRADES:-/etc/apt/apt.conf.d/20auto-upgrades}"
hosts="${DRILL_ETC_HOSTS:-/etc/hosts}"
{
printf 'hostname: %s\n' "$(hostname 2>/dev/null || echo '(absent)')"
printf 'hosts.127.0.1.1: %s\n' "$(grep -E '^127\.0\.1\.1[[:space:]]' "$hosts" 2>/dev/null || echo '(absent)')"
printf 'role-marker: %s\n' "$(cat "$marker" 2>/dev/null || echo '(absent)')"
printf 'manifest:\n'
sed 's/^/ /' "$manifest" 2>/dev/null || printf ' (absent)\n'
printf 'auto-upgrades:\n'
sed 's/^/ /' "$autoup" 2>/dev/null || printf ' (absent)\n'
printf 'sshd-effective:\n'
if command -v sshd >/dev/null 2>&1; then
sshd -T 2>/dev/null | sort | sed 's/^/ /' || printf ' (sshd -T failed)\n'
else
printf ' (sshd absent)\n'
fi
# Self's Tags is the FIRST occurrence in the status JSON (Self serializes
# before Peer). Tags only, nothing livelier: peers joining, IPs renewing
# or a backend-state flap between two captures is not a convergence diff
# on this box, and a capture that can move on its own poisons the
# idempotence verdict with noise.
printf 'tailscale.self.tags: %s\n' "$(tailscale status --json 2>/dev/null | tr -d '\n ' | grep -o '"Tags":\[[^]]*\]' | head -n1 || true)"
printf 'users-ledger:\n'
sed 's/^/ /' "$ledger" 2>/dev/null || printf ' (absent)\n'
# Per-operator effective state: the account, its groups, its lock state,
# its keys. sha256 of authorized_keys, not the keys themselves — the
# capture may end up quoted in a record and keys are long, not secret.
while read -r u state; do
[ -n "${u:-}" ] || continue
if ! id -u "$u" >/dev/null 2>&1; then
printf 'user.%s: (no account)\n' "$u"
continue
fi
printf 'user.%s: state=%s groups=%s lock=%s\n' "$u" "${state:-active}" \
"$(id -Gn "$u" 2>/dev/null | tr ' ' ',')" \
"$(passwd -S "$u" 2>/dev/null | awk '{print $2}' || echo '?')"
home="$(getent passwd "$u" | cut -d: -f6)"
keys="$home/.ssh/authorized_keys"
printf 'user.%s.authorized_keys: %s\n' "$u" \
"$(sha256sum "$keys" 2>/dev/null | cut -d' ' -f1 || echo '(none)')"
done < <(cat "$ledger" 2>/dev/null)
printf 'sudoers.d:\n'
find "${DRILL_SUDOERS_DIR:-/etc/sudoers.d}" -maxdepth 1 -type f 2>/dev/null | sort \
| while read -r u; do printf ' %s %s\n' "$(sha256sum "$u" | cut -d' ' -f1)" "$u"; done
printf 'box: %s\n' "$(command -v box 2>/dev/null || echo '(absent)')"
} > "$out"
}
# ref_sha <owner/repo> <ref> — the commit the record cites. Tags outrank
# branches (the installer's own precedence, install.sh:117-120). Resolved once
# up front and reused, so the record and the install describe the same instant
# even if the branch moves mid-drill. Empty on failure; the record then says
# 'unresolved' rather than inventing one.
ref_sha() {
local sha
sha="$(git ls-remote "https://github.com/$1" "refs/tags/$2" 2>/dev/null | head -n1 | cut -f1)"
[ -n "$sha" ] || sha="$(git ls-remote "https://github.com/$1" "refs/heads/$2" 2>/dev/null | head -n1 | cut -f1)"
printf '%s' "${sha:0:7}"
}
# emit_record <path> — drills/<version>.md, in the shape drills/README.md
# defines: what ran, on what host, the pinned refs and SHAs, the numbers, and
# what failed. Emitted on EVERY completed run — a failed drill is still a
# valid record; the gate wants evidence, not success. Skipped legs are listed
# by name: a record with no failures listed reads as "nothing broke", so a leg
# that was not run says so instead of being omitted.
emit_record() {
local out="$1" i os cpus ram virt line
os="$(. /etc/os-release 2>/dev/null && printf '%s' "${PRETTY_NAME:-unknown}")"
cpus="$(nproc 2>/dev/null || echo '?')"
ram="$(awk '/MemTotal/{printf "%.0f", $2/1024/1024}' /proc/meminfo 2>/dev/null || echo '?')"
virt="$(systemd-detect-virt 2>/dev/null || echo unknown)"
{
printf '# Release drill — %s — %s\n\n' "$DRILL_VERSION" "$(date -u +%F)"
printf 'Run ID: %s. Host: %s, %s vCPU / %s GB RAM (%s).\n' "$RUN_ID" "${os:-unknown}" "$cpus" "$ram" "$virt"
printf 'Candidate refs: rig@%s (RIG_REF=%s), box@%s (BOX_REF=%s).\n' \
"${RIG_SHA:-unresolved}" "$REF" "${BOX_SHA:-unresolved}" "$BOXREF"
printf 'Template registry: %s@%s (ref %s, %s) — the rig-templates source the converge read (#110/#153).\n' \
"${TPLREPO:-heavy-duty/rig-templates}" "${TPL_SHA:-unresolved}" \
"${TPLREF:-unresolved}" "${TPL_SOURCE:-fetched}"
printf 'Instrument: drill/drill.sh, legs in execution order.\n\n'
printf '| Leg | Result |\n'
printf '| --- | --- |\n'
for i in "${!LEG_NAMES[@]}"; do
printf '| %s | %s |\n' "${LEG_NAMES[$i]}" "${LEG_RESULTS[$i]}"
done
printf '\nChecks: %s passed, %s failed, %s skipped.\n' "$pass" "$fail" "$skipped"
if [ "$fail" -eq 0 ] && [ "$skipped" -eq 0 ]; then
printf '\nFailed: nothing. Every leg ran and every check passed.\n'
else
# printf --: a format opening with '- ' reads as an option to bash's
# printf and emits NOTHING — a record whose Failed section silently
# vanished is exactly the lie this file exists to make impossible.
[ "$fail" -gt 0 ] && printf '\nFailed:\n'
for line in "${findings[@]:-}"; do
case "$line" in FAIL:*) printf -- '- %s\n' "$line" ;; esac
done
[ "$skipped" -gt 0 ] && printf '\nSkipped — these did NOT run, and this record is not evidence for them:\n'
for line in "${findings[@]:-}"; do
case "$line" in SKIP:*) printf -- '- %s\n' "$line" ;; esac
done
fi
printf '\nThe isolation boundary was NOT asserted here: it is box'\''s drill'\''s\n'
printf 'assertion (heavy-duty/box drill/drill.sh), joined to this record by the run ID.\n'
} > "$out"
}
# =============================================================================
# Pre-flight — every refusal this run can see coming fires here, before
# anything is installed or any credential is spent. Args are validated BEFORE
# the root check (repo doctrine, bootstrap.sh:114 — so the refusals are
# testable without root, and a typo costs a re-type, never a re-ssh).
# =============================================================================
# Both refs EXPLICIT, or nothing runs. Defaulting either to main is exactly
# the #103 hazard this harness exists to refuse: "I drilled the release" must
# not quietly mean "I drilled whatever main was that afternoon".
if [ -z "$REF" ] || [ -z "$BOXREF" ]; then
echo "drill: both refs must be pinned explicitly — a drill against an unstated ref is not evidence (#103):" >&2
echo " --rig-ref <ref> (or RIG_REF) the rig candidate, e.g. release/0.4.0 [got: ${REF:-<unset>}]" >&2
echo " --box-ref <ref> (or BOX_REF) the box that will ship with it [got: ${BOXREF:-<unset>}]" >&2
exit 2
fi
case "$ROLE" in
staging-server|dev-server|control-plane-server|workload-server|runner-server) ;;
*) echo "drill: --role $ROLE is not a machine role this drill can converge unattended" >&2; exit 2 ;;
esac
if [ -z "$USERS_FILE" ]; then
echo "drill: --users <path> is required — leg 1 asserts operators converged, and bootstrap requires the file (its --no-users opt-out would leave leg 1 asserting nothing)" >&2
exit 2
fi
[ -r "$USERS_FILE" ] || { echo "drill: cannot read users file: $USERS_FILE" >&2; exit 2; }
[ "$(id -u)" -eq 0 ] || { echo "drill: must run as root (bootstrap, runner, coolify and db all require it) — ssh in as root on the throwaway machine" >&2; exit 1; }
# The tailnet join needs a key unless this machine already joined (a re-drill
# on the same throwaway). Caught here, not 10 apt-minutes into bootstrap.
if [ -z "${TS_AUTHKEY:-}" ]; then
if ! { command -v tailscale >/dev/null 2>&1 && tailscale status >/dev/null 2>&1; }; then
echo "drill: TS_AUTHKEY is unset and this machine has not joined a tailnet — leg 1's bootstrap will refuse. Mint a single-use TAGGED pre-auth key and export TS_AUTHKEY." >&2
exit 2
fi
fi
command -v curl >/dev/null 2>&1 || { echo "drill: curl is required (the pinned installs download over it)" >&2; exit 1; }
if [ "$YES" -ne 1 ]; then
cat <<EOF
This will, ON THIS HOST ($(hostname)):
· wipe any installed rig and reinstall $REPO@$REF from scratch
· run 'rig bootstrap $ROLE --users $USERS_FILE' — sshd hardening, hostname
change, tailnet join, box ($BOXREPO@$BOXREF) + its Incus stack — TWICE
(the second run is the idempotence assertion)
· install Coolify${COOLIFY_VERSION:+ $COOLIFY_VERSION}, a GitHub runner${RUNNER_REPO:+ against $RUNNER_REPO} and a Forgejo runner${FJ_INSTANCE:+ against $FJ_INSTANCE}${FJ_RUNNER_REPO:+ ($FJ_RUNNER_REPO)}
Only do this on a THROWAWAY machine you can format.
EOF
[ -t 0 ] || { echo "drill: no TTY to confirm on — pass --yes if you mean it." >&2; exit 2; }
printf 'Continue? [y/N] '
read -r reply
case "$reply" in y|Y|yes) ;; *) echo "stopped."; exit 1 ;; esac
fi
phase "Pinned candidates"
RIG_SHA="$(ref_sha "$REPO" "$REF")"
BOX_SHA="$(ref_sha "$BOXREPO" "$BOXREF")"
inf "rig: $REPO@$REF (${RIG_SHA:-unresolved})"
inf "box: $BOXREPO@$BOXREF (${BOX_SHA:-unresolved})"
inf "run ID: $RUN_ID — drills sharing this substrate share it (drills/README.md)"
# =============================================================================
phase "Installing rig ($REPO@$REF) from scratch"
# =============================================================================
# The drill proves a tree from SCRATCH every run — a fresh machine, not a
# converged install — so any prior rig goes first (root's install lands at
# \$HOME/.local/share/rig with the /usr/local/bin symlink).
rm -rf "$HOME/.local/share/rig" /usr/local/bin/rig
if ! run_logged /tmp/drill-rig-install.log \
env RIG_REPO="$REPO" RIG_REF="$REF" \
bash -c "bash <(curl -fsSL \"https://raw.githubusercontent.com/$REPO/$REF/install.sh\")"; then
echo "drill: rig's installer failed — tail of /tmp/drill-rig-install.log:" >&2
tail -5 /tmp/drill-rig-install.log >&2
exit 1
fi
command -v rig >/dev/null 2>&1 || { echo "drill: installer reported success but no 'rig' on PATH" >&2; exit 1; }
# ASSERT WHAT LANDED — the up-front ref assertion, fatal on mismatch.
RIG_TREE="$(tree_of "$(command -v rig)")"
assert_installed_from rig "$RIG_TREE" "$REPO@$REF" || exit 1
DRILL_VERSION="$(head -n1 "$RIG_TREE/VERSION" 2>/dev/null || echo unknown)"
ok "installed tree confirms: $REPO@$REF (version $DRILL_VERSION)"
# The rig-templates ref this candidate converges (#110), for the record: the
# env override when the drill was pointed somewhere, else the pin read from
# the INSTALLED tree — what actually landed, never this checkout's copy. A
# 40-hex ref IS its own SHA (the pin's normal shape); anything else resolves
# through ref_sha like the two candidates above.
if [ -z "$TPLREF" ]; then
TPLREF="$(sed -n 's/^RIG_TEMPLATES_PIN=//p' "$RIG_TREE/commands/lib/templates.sh" 2>/dev/null | head -n1)"
if [ -n "$TPLREF" ] &&
[ -n "$(find "$RIG_TREE/templates@$TPLREF" -mindepth 2 -maxdepth 2 -type f -name template.env -print -quit 2>/dev/null)" ]; then
TPL_SOURCE="snapshot"
fi
fi
if [[ "$TPLREF" =~ ^[0-9a-f]{40}$ ]]; then
TPL_SHA="${TPLREF:0:7}"
elif [ -n "$TPLREF" ]; then
TPL_SHA="$(ref_sha "$TPLREPO" "$TPLREF")"
fi
inf "templates: $TPLREPO@${TPLREF:-unresolved} (${TPL_SHA:-unresolved}, $TPL_SOURCE)"
[ -n "$RECORD" ] || RECORD="$ROOT/drills/$DRILL_VERSION.md"
# =============================================================================
phase "Leg 1 — convergence: rig bootstrap $ROLE"
# =============================================================================
# BOX_REPO/BOX_REF ride the environment into bootstrap's host=yes box install,
# so the box that lands is the pinned candidate, not what bootstrap falls back
# to unexported (the BOX_RELEASE pin, since rig#103 landed).
export BOX_REPO="$BOXREPO" BOX_REF="$BOXREF"
t0=$SECONDS
if run_logged /tmp/drill-bootstrap-1.log rig bootstrap "$ROLE" --users "$USERS_FILE"; then
ok "rig bootstrap $ROLE --users … exited 0 ($((SECONDS - t0))s)"
BOOTSTRAP_OK=1
else
no "rig bootstrap $ROLE FAILED — tail: $(tail -3 /tmp/drill-bootstrap-1.log | tr '\n' ' ')"
BOOTSTRAP_OK=0
fi
MARKER_LINE="$(cat "${RIG_ROLE_MARKER:-/etc/rig/role}" 2>/dev/null || true)"
if [ "$BOOTSTRAP_OK" -eq 1 ]; then
# The role, asserted on EFFECTIVE state — the marker, the daemon's resolved
# config, the netmap's granted tags — never on what was requested (the
# sshd-first-wins lesson, lib/sshd.sh:63-70).
case "$MARKER_LINE" in
"role=$ROLE "*) ok "role marker: $MARKER_LINE" ;;
*) no "role marker is '$MARKER_LINE' — expected role=$ROLE …" ;;
esac
sshd -T 2>/dev/null | grep -qx 'passwordauthentication no' \
&& ok "sshd -T resolves passwordauthentication no (the hardening took)" \
|| no "sshd still resolves password auth — the 00-rig.conf drop-in is not winning"
ts_tags="$(tailscale status --json 2>/dev/null | tr -d '\n ' | grep -o '"Tags":\[[^]]*\]' | head -n1)"
if [ -n "$ts_tags" ] && [ "$ts_tags" != '"Tags":[]' ]; then
ok "tailnet joined, tagged: $ts_tags"
else
no "tailnet join did not leave a tagged node (got: ${ts_tags:-nothing}) — bootstrap's verify should have refused this"
fi
grep -q 'Unattended-Upgrade "1"' /etc/apt/apt.conf.d/20auto-upgrades 2>/dev/null \
&& ok "unattended-upgrades enabled" || no "20auto-upgrades missing or wrong"
grep -q "converged_by=$DRILL_VERSION" "${RIG_MANIFEST:-/etc/rig/manifest}" 2>/dev/null \
&& ok "manifest: converged_by=$DRILL_VERSION" || no "manifest does not name $DRILL_VERSION as the converging rig"
users_bad=""
while read -r u state; do
[ "$state" = active ] || continue
id -u "$u" >/dev/null 2>&1 || { users_bad="$users_bad $u(no-account)"; continue; }
uhome="$(getent passwd "$u" | cut -d: -f6)"
[ -s "$uhome/.ssh/authorized_keys" ] || users_bad="$users_bad $u(no-keys)"
done < <(cat "${DRILL_LEDGER:-/etc/rig/users}" 2>/dev/null)
# NOT 'grep -c … || echo 0': grep -c already prints 0 on no match (and then
# exits 1), so the fallback would emit a second line into the substitution.
n_users="$(grep -c ' active$' "${DRILL_LEDGER:-/etc/rig/users}" 2>/dev/null)" || true
n_users="${n_users:-0}"
[ -z "$users_bad" ] && [ "$n_users" -gt 0 ] \
&& ok "operators converged: $n_users active, accounts and keys present" \
|| no "operators NOT converged:${users_bad:- ledger empty}"
leg "convergence — bootstrap $ROLE reaches its role" \
"$([ "$fail" -eq 0 ] && echo "PASS ($((SECONDS - t0))s)" || echo "FAIL — see Failed below")"
# --- idempotence: the claim this drill exists to make ----------------------
# Capture, re-run, capture, diff. Mechanically — never "watched it not
# obviously break". An empty diff IS the definition of converged.
phase "Leg 1 — idempotence: the re-run must change nothing"
pre="$(mktemp)"; post="$(mktemp)"
capture_state "$pre"
t0=$SECONDS
if run_logged /tmp/drill-bootstrap-2.log rig bootstrap "$ROLE" --users "$USERS_FILE"; then
ok "second bootstrap exited 0 ($((SECONDS - t0))s)"
else
no "second bootstrap FAILED — tail: $(tail -3 /tmp/drill-bootstrap-2.log | tr '\n' ' ')"
fi
capture_state "$post"
if statediff="$(diff -u "$pre" "$post")"; then
ok "re-converge is a no-op: the state diff is empty"
leg "re-converge (idempotence)" "clean, no changes"
else
dlines="$(printf '%s\n' "$statediff" | grep -c '^[+-][^+-]')"
no "re-converge CHANGED the box — $dlines state line(s) differ:"
printf '%s\n' "$statediff" | sed 's/^/ /'
leg "re-converge (idempotence)" "DIRTY — $dlines state line(s) changed on the re-run"
fi
rm -f "$pre" "$post"
else
leg "convergence — bootstrap $ROLE reaches its role" "FAIL — bootstrap exited non-zero"
skip "idempotence not asserted — the first converge already failed, a re-run diff would measure noise"
leg "re-converge (idempotence)" "SKIPPED — first converge failed"
fi
# =============================================================================
phase "--host yes — the box that will ship"
# =============================================================================
# The assertions #105 settles this leg at: the installer ran, INSTALLED_FROM
# matches the requested BOX_REF, setup-host exited clean, the stack it claims
# stands. Then it STOPS. Not one isolation probe: two records that both claim
# the trust boundary will eventually disagree with no tiebreaker, and a
# partial isolation check reads — months later, in a record — as though the
# boundary was drilled (box#153's shape through a different door). Resist
# adding "just one" probe here; that is box's drill's whole job.
case "$MARKER_LINE" in
*"host=yes"*)
if command -v box >/dev/null 2>&1; then
ok "box CLI on PATH"
BOX_TREE="$(tree_of "$(command -v box)")"
# Fatal, like rig's own: a wrong box under --host yes poisons the pair.
assert_installed_from box "$BOX_TREE" "$BOXREPO@$BOXREF" || exit 1
ok "installed box confirms: $BOXREPO@$BOXREF"
if box doctor >/dev/null 2>&1; then
ok "box doctor passes — setup-host converged; the host stack stands (box's own effective-state verdict)"
leg "--host yes: pinned box installed, host stack up" "PASS — $BOXREPO@$BOXREF, box doctor clean"
else
no "box is installed but 'box doctor' does not pass — the host stack is unproven (run 'box doctor' for box's verdict)"
leg "--host yes: pinned box installed, host stack up" "FAIL — box doctor does not pass"
fi
else
no "no 'box' on PATH after a host=yes bootstrap — the box install did not take (bootstrap warns rather than dies there; the drill does not)"
leg "--host yes: pinned box installed, host stack up" "FAIL — box CLI never landed"
fi
inf "isolation NOT asserted here — deliberately. The VM trust boundary is box's"
inf "assertion, made by box's own drill (~85 probes); this leg stops at 'the pinned"
inf "box installed and its host stack stands'. The records join on the run ID."
;;
*)
skip "--host yes assertions: role $ROLE left host=no (marker: ${MARKER_LINE:-absent})"
leg "--host yes: pinned box installed, host stack up" "SKIPPED — this role does not host VMs"
;;
esac
# =============================================================================
phase "Leg 4 — coolify install (pinned, AUTOUPDATE=false)"
# =============================================================================
# Runs BEFORE leg 2 on purpose: Coolify's installer is what puts Docker on the
# box, and the db leg needs a daemon — ordering them the other way around
# would manufacture a skip this same run could have avoided.
if [ -z "$COOLIFY_VERSION" ]; then
skip "coolify install: no --coolify-version pin given — the leg did not run (rig's own install refuses to default a version, and so does its drill)"
leg "coolify install" "SKIPPED — no version pin provided"
else
t0=$SECONDS
if run_logged /tmp/drill-coolify.log rig coolify install --version "$COOLIFY_VERSION"; then
ok "rig coolify install --version $COOLIFY_VERSION exited 0 ($((SECONDS - t0))s)"
grep -qx 'AUTOUPDATE=false' /data/coolify/source/.env 2>/dev/null \
&& ok "AUTOUPDATE=false landed in /data/coolify/source/.env — the platform will not move under its operators" \
|| no "AUTOUPDATE=false is NOT in coolify's .env — the pin is not holding"
cstate="$(docker inspect -f '{{.State.Status}}' coolify 2>/dev/null || echo absent)"
[ "$cstate" = running ] && ok "the coolify container is running" \
|| no "coolify container state: $cstate (expected running)"
leg "coolify install ($COOLIFY_VERSION)" \
"$([ "$cstate" = running ] && echo "PASS ($(((SECONDS - t0) / 60)) min)" || echo "FAIL — container $cstate")"
else
no "coolify install FAILED — tail: $(tail -3 /tmp/drill-coolify.log | tr '\n' ' ')"
leg "coolify install ($COOLIFY_VERSION)" "FAIL — installer exited non-zero"
fi
fi
# =============================================================================
phase "Leg 2 — db dump/restore round-trip (test/db-integration.sh)"
# =============================================================================
# Driven from the INSTALLED tree — the drill exercises what shipped, not the
# checkout this script happens to sit in. The leg's skip contract is the
# script's own (loud, reasoned, exit 0) and classify_leg keeps it a SKIP:
# counted, rendered distinctly, named in the record — never a pass.
db_out="$(mktemp)"
bash "$RIG_TREE/test/db-integration.sh" >"$db_out" 2>&1
db_rc=$?
case "$(classify_leg "$db_rc" "$db_out")" in
pass)
db_numbers="$(tail -1 "$db_out")"
ok "db round-trip: $db_numbers"
leg "test/db-integration.sh" "PASS — $db_numbers"
;;
skip)
db_reason="$(grep -m1 '^skip:' "$db_out")"
skip "db round-trip did not run — $db_reason"
leg "test/db-integration.sh" "SKIPPED — ${db_reason#skip: }"
;;
fail)
no "db round-trip FAILED (exit $db_rc) — tail: $(tail -3 "$db_out" | tr '\n' ' ')"
leg "test/db-integration.sh" "FAIL — exit $db_rc"
;;
esac
rm -f "$db_out"
# =============================================================================
phase "Leg 3 — runner lifecycle against a fork"
# =============================================================================
# Register, take a job, deregister. The fork must carry a workflow_dispatch
# workflow (default drill.yml) whose job runs-on the 'drill' label — see
# drill/README.md. Tokens: RUNNER_TOKEN / RUNNER_REMOVE_TOKEN env, or minted
# via an authenticated gh. Without a fork or a token source the leg SKIPS,
# loudly, and the record says it did not run.
if [ -z "$RUNNER_REPO" ]; then
skip "runner lifecycle: no --runner-repo fork given — the leg did not run"
leg "runner lifecycle" "SKIPPED — no fork provided"
else
GH_OK=0
command -v gh >/dev/null 2>&1 && gh auth status >/dev/null 2>&1 && GH_OK=1
reg_token="${RUNNER_TOKEN:-}"
if [ -z "$reg_token" ] && [ "$GH_OK" -eq 1 ]; then
reg_token="$(gh api -X POST "repos/$RUNNER_REPO/actions/runners/registration-token" --jq .token 2>/dev/null)"
fi
if [ -z "$reg_token" ]; then
skip "runner lifecycle: no RUNNER_TOKEN and no authenticated gh to mint one — the leg did not run"
leg "runner lifecycle ($RUNNER_REPO)" "SKIPPED — no registration token source"
else
RUNNER_NAME="drill-$(hostname)-$$"
if RUNNER_TOKEN="$reg_token" run_logged /tmp/drill-runner-install.log \
rig runner install --repo "$RUNNER_REPO" --name "$RUNNER_NAME" --labels drill; then
ok "rig runner install --repo $RUNNER_REPO exited 0 (registered as $RUNNER_NAME)"
else
no "runner install FAILED — tail: $(tail -3 /tmp/drill-runner-install.log | tr '\n' ' ')"
fi
rig runner status 2>/dev/null | grep -q "$RUNNER_REPO" \
&& ok "runner status names the fork: $RUNNER_REPO" \
|| no "runner status does not name $RUNNER_REPO"
took_job=none
if [ "$GH_OK" -eq 1 ]; then
# Dispatch, then poll the newest run of that workflow to completion.
# The newest run's ID is read BEFORE dispatching, so an old completed
# run can never be mistaken for the one just dispatched (the poll's
# verdict must be about OUR run, and workflow_dispatch takes a few
# seconds to materialize a run at all). ~5 min bound: a queued-forever
# run means the runner never picked the job up, which is exactly what
# this check exists to catch.
pre_id="$(gh run list -R "$RUNNER_REPO" --workflow "$RUNNER_WORKFLOW" --limit 1 --json databaseId --jq '.[0].databaseId' 2>/dev/null)"
if gh workflow run "$RUNNER_WORKFLOW" -R "$RUNNER_REPO" >/dev/null 2>&1; then
inf "dispatched $RUNNER_WORKFLOW on $RUNNER_REPO — waiting for the runner to take it (≤5 min)…"
took_job=timeout
for _i in $(seq 1 30); do
sleep 10
run_line="$(gh run list -R "$RUNNER_REPO" --workflow "$RUNNER_WORKFLOW" --limit 1 \
--json databaseId,status,conclusion --jq '.[0] | "\(.databaseId) \(.status) \(.conclusion)"' 2>/dev/null)"
read -r rid rstatus rconc <<< "$run_line"
[ -n "${rid:-}" ] || continue
[ "$rid" != "${pre_id:-}" ] || continue
if [ "${rstatus:-}" = completed ]; then
case "${rconc:-}" in
success) took_job=success ;;
*) took_job=failed ;;
esac
break
fi
done
else
took_job=nodispatch
fi
case "$took_job" in
success) ok "the runner took a job and it succeeded ($RUNNER_WORKFLOW)" ;;
failed) no "the dispatched job completed UNSUCCESSFULLY — the runner ran it, the workflow failed; read the run on $RUNNER_REPO" ;;
timeout) no "the dispatched job never completed within 5 min — the runner did not take it (is the workflow's runs-on label 'drill'?)" ;;
nodispatch) no "could not dispatch $RUNNER_WORKFLOW on $RUNNER_REPO — does the fork carry it, with workflow_dispatch? (see drill/README.md)" ;;
esac
else
skip "took a job: not attempted — no authenticated gh to dispatch $RUNNER_WORKFLOW with"
fi
rem_token="${RUNNER_REMOVE_TOKEN:-}"
if [ -z "$rem_token" ] && [ "$GH_OK" -eq 1 ]; then
rem_token="$(gh api -X POST "repos/$RUNNER_REPO/actions/runners/remove-token" --jq .token 2>/dev/null)"
fi
if [ -n "$rem_token" ]; then
RUNNER_REMOVE_TOKEN="$rem_token" rig runner remove >/dev/null 2>&1 \
&& ok "rig runner remove deregistered cleanly" \
|| no "runner remove FAILED"
else
rig runner remove --local >/dev/null 2>&1 \
&& note "deregistered --local only (no removal token source) — delete the stale runner from $RUNNER_REPO's settings by hand" \
|| no "runner remove --local FAILED"
fi
rig runner status >/dev/null 2>&1 \
&& no "runner status still answers after remove — the deregistration did not take" \
|| ok "runner status confirms: nothing registered"
leg "runner lifecycle ($RUNNER_REPO)" \
"$(case "$took_job" in
success) echo "PASS — registered, took a job, deregistered clean" ;;
none) echo "PARTIAL — registered and deregistered; took a job: not attempted (no gh)" ;;
*) echo "FAIL — see Failed below" ;;
esac)"
fi
fi
# =============================================================================
phase "Leg 3 — forgejo runner lifecycle against an instance"
# =============================================================================
# The same leg as above, for the other forge. Both forges ship a runner family
# (#109 added `rig forgejo-runner` beside `rig runner`), so a release that
# evidences only GitHub evidences half of what it ships (#129).
#
# Three things differ from the GitHub leg, all measured against
# forgejo.heavyduty.builders (8.0.3+gitea-1.22.0) on 2026-07-30, not read:
#
# * Scope is the TOKEN's, never a flag — `rig forgejo-runner install` refuses
# --repo on purpose (commands/forgejo-runner-install.sh:160). The repo here
# is only where the registration token is minted from, and where the
# dispatched workflow lives.
# * The mint path is /repos/<o>/<r>/actions/runners/registration-token. The
# instance's own swagger documents /repos/<o>/<r>/runners/registration-token
# — WITHOUT /actions/ — and that path 404s. Do not "fix" this to match the
# published API reference.
# * There is no deregistration endpoint, so there is no removal token and no
# remote deregistration: `rig forgejo-runner remove` is local-only by
# design (commands/forgejo-runner-remove.sh:7-11) and the runner row
# survives in the UI until a human deletes it. The record says so rather
# than implying a clean remote teardown the way the GitHub leg can.
#
# Tokens: FORGEJO_RUNNER_TOKEN (a registration token) is used directly; else
# FORGEJO_API_TOKEN mints one from FJ_RUNNER_REPO. Without an instance, a repo,
# or a token source the leg SKIPS loudly and the record says it did not run.
if [ -z "$FJ_INSTANCE" ] || [ -z "$FJ_RUNNER_REPO" ]; then
skip "forgejo runner lifecycle: no --forgejo-instance/--forgejo-runner-repo given — the leg did not run"
leg "forgejo runner lifecycle" "SKIPPED — no instance/repo provided"
else
fj_reg="${FORGEJO_RUNNER_TOKEN:-}"
if [ -z "$fj_reg" ] && [ -n "${FORGEJO_API_TOKEN:-}" ]; then
fj_reg="$(curl -fsSL -H "Authorization: token ${FORGEJO_API_TOKEN}" \
"${FJ_INSTANCE%/}/api/v1/repos/${FJ_RUNNER_REPO}/actions/runners/registration-token" 2>/dev/null \
| grep -o '"token"[[:space:]]*:[[:space:]]*"[^"]*"' | head -n1 \
| sed 's/.*:[[:space:]]*"//; s/"$//')"
fi
fj_tok_verdict="$(forgejo_token_verdict "$fj_reg" "${FORGEJO_API_TOKEN:-}")"
if [ "$fj_tok_verdict" = mint-failed ]; then
# Configured, and it did not work. Never a skip: see forgejo_token_verdict.
no "registration-token mint FAILED against ${FJ_INSTANCE} — is it reachable, and does FORGEJO_API_TOKEN own ${FJ_RUNNER_REPO}? (the token is never printed)"
leg "forgejo runner lifecycle ($FJ_RUNNER_REPO)" "FAIL — registration-token mint failed"
elif [ "$fj_tok_verdict" = no-source ]; then
skip "forgejo runner lifecycle: no FORGEJO_RUNNER_TOKEN and no FORGEJO_API_TOKEN to mint one — the leg did not run"
leg "forgejo runner lifecycle ($FJ_RUNNER_REPO)" "SKIPPED — no registration token source"
else
FJ_NAME="drill-$(hostname)-$$"
fj_install_ok=0 fj_status_ok=0 fj_remove_ok=0 fj_absent_ok=0
# The label MUST carry a docker:// image: forgejo-runner runs jobs in
# containers, and a bare label leaves runs-on matched but unrunnable.
if FORGEJO_RUNNER_TOKEN="$fj_reg" run_logged /tmp/drill-forgejo-runner-install.log \
rig forgejo-runner install --instance "$FJ_INSTANCE" --name "$FJ_NAME" \
--labels 'drill:docker://node:22-bookworm'; then
ok "rig forgejo-runner install --instance $FJ_INSTANCE exited 0 (registered as $FJ_NAME)"
fj_install_ok=1
else
no "forgejo-runner install FAILED — tail: $(tail -3 /tmp/drill-forgejo-runner-install.log | tr '\n' ' ')"
fi
if rig forgejo-runner status 2>/dev/null | grep -qF "${FJ_INSTANCE%/}"; then
ok "forgejo-runner status names the instance: $FJ_INSTANCE"; fj_status_ok=1
else
no "forgejo-runner status does not name ${FJ_INSTANCE}"
fi
fj_took=none
# Do NOT dispatch once install or status has failed. The job would be taken
# by whatever else answers runs-on: drill — a stale runner this leg's own
# hand-delete caveat leaves behind — and its success would be evidence about
# someone else's runner (codex/grok/kimi, !130).
if [ "$fj_install_ok" != 1 ] || [ "$fj_status_ok" != 1 ]; then
skip "took a job: not attempted — install or status failed, and a foreign runner answering 'drill' could only manufacture a false pass"
elif [ -n "${FORGEJO_API_TOKEN:-}" ]; then
fj_api="${FJ_INSTANCE%/}/api/v1/repos/${FJ_RUNNER_REPO}"
# Read the newest ASSIGNED task id BEFORE dispatching, same guard as the
# GitHub leg: an already-completed run must never be read as ours.
fj_pre_body="$(curl -fsSL -H "Authorization: token ${FORGEJO_API_TOKEN}" \
"$fj_api/actions/tasks" 2>/dev/null || echo '{}')"
printf '%s' "$fj_pre_body" > /tmp/drill-forgejo-pre.json
fj_pre="$(forgejo_max_task_id /tmp/drill-forgejo-pre.json)"
if curl -fsSL -o /dev/null -X POST -H "Authorization: token ${FORGEJO_API_TOKEN}" \
-H "Content-Type: application/json" -d "{\"ref\":\"${FJ_REF}\"}" \
"$fj_api/actions/workflows/${RUNNER_WORKFLOW}/dispatches" 2>/dev/null; then
inf "dispatched $RUNNER_WORKFLOW on $FJ_RUNNER_REPO — waiting for the runner to take it (≤5 min)…"
fj_took=timeout
for _i in $(seq 1 30); do
sleep 10
curl -fsSL -H "Authorization: token ${FORGEJO_API_TOKEN}" \
"$fj_api/actions/tasks" -o /tmp/drill-forgejo-tasks.json 2>/dev/null || continue
case "$(forgejo_run_verdict "${fj_pre:-}" /tmp/drill-forgejo-tasks.json)" in
success) fj_took=success; break ;;
failed) fj_took=failed; break ;;
*) : ;; # pending — queued, or assigned and still running
esac
done
else
fj_took=nodispatch
fi
case "$fj_took" in
success) ok "the forgejo runner took a job and it succeeded ($RUNNER_WORKFLOW)" ;;
failed) no "the dispatched job completed UNSUCCESSFULLY — the runner ran it, the workflow failed; read the run on $FJ_RUNNER_REPO" ;;
# A queued task is INVISIBLE in this API until a runner claims it, so a
# timeout means "nothing ever took it". Two causes, and the second one
# is not rig's: the runs-on label may not match, or the daemon's poller
# can go quiet — a restarted daemon claims a minutes-old backlog in
# about a second. Check 'systemctl restart forgejo-runner' before
# reading this as a rig defect.
timeout) no "the dispatched job was never taken within 5 min — check the workflow's runs-on is 'drill', then restart forgejo-runner and re-read (a quiet poller looks exactly like this)" ;;
nodispatch) no "could not dispatch $RUNNER_WORKFLOW on $FJ_RUNNER_REPO — does it carry that workflow, with workflow_dispatch, on its default branch?" ;;
esac
else
skip "took a job: not attempted — no FORGEJO_API_TOKEN to dispatch $RUNNER_WORKFLOW with"
fi
# No removal token exists on this forge — remove is local by design.
if rig forgejo-runner remove >/dev/null 2>&1; then
note "forgejo-runner removed locally — Forgejo has no deregistration endpoint, so DELETE the stale '$FJ_NAME' row under $FJ_RUNNER_REPO > Settings > Actions > Runners by hand"
fj_remove_ok=1
else
no "forgejo-runner remove FAILED"
fi
if rig forgejo-runner status >/dev/null 2>&1; then
no "forgejo-runner status still answers after remove — the removal did not take"
else
ok "forgejo-runner status confirms: nothing registered"; fj_absent_ok=1
fi
leg "forgejo runner lifecycle ($FJ_RUNNER_REPO)" \
"$(forgejo_leg_row "$fj_install_ok" "$fj_status_ok" "$fj_took" \
"$fj_remove_ok" "$fj_absent_ok")"
fi
fi
# =============================================================================
phase "Summary"
# =============================================================================
printf ' %s passed, %s failed, %s skipped\n' "$pass" "$fail" "$skipped"
if [ "${#findings[@]}" -gt 0 ]; then
echo
printf ' %s\n' "${findings[@]}"
fi
mkdir -p "$(dirname "$RECORD")"
emit_record "$RECORD"
echo
inf "record written: $RECORD"
inf "commit it on the release branch as drills/$DRILL_VERSION.md — the"
inf "drill-recorded gate reads that file and nothing else (drills/README.md)."
[ "$fail" -eq 0 ]

12
drills/0.3.1.md
View file

@ -1,12 +0,0 @@
# rig 0.3.1 — release drill
**WAIVED. No real-hardware drill was run for this release.**
The maintainer (@danmt) directed “Add a waiver as the drill” on
2026-07-25 at 09:02Z:
https://github.com/heavy-duty/rig/pull/145#issuecomment-5077845754
The real-hardware run requested in discussion #132 was not performed. At the
time of the ruling, the harness and procedure (`drill/drill.sh` and
`drill/README.md`) were available on `main`, unlike for the 0.3.0 release.
The ruling gave no reason for the waiver.

39
drills/README.md
View file

@ -15,18 +15,16 @@ The directory is `drills/`, not `.drills/` — a dot-directory is invisible to
any glob without `dotglob`, which is how #70 here and box#116 / box#118 all any glob without `dotglob`, which is how #70 here and box#116 / box#118 all
happened. happened.
**This directory is the record, not the instrument.** The instrument is **This directory is the record, not the instrument.** rig has **no drill
[`drill/drill.sh`](../drill/README.md) (#105): it runs the legs, asserts the harness script of its own**; its legs are run by following the documented
pinned refs actually landed, decides idempotence by a mechanical state diff, procedure, and the harness lives in heavy-duty/box's `drill/`. rig does not
and emits the record file this directory holds. rig does not reach into reach into it to decide whether rig may ship: a cross-repo lookup that fails
another repo's harness to decide whether rig may ship: a cross-repo lookup silently degrades to "pass", which is the UNREADABLE-vs-NONE shape #90 fixed.
that fails silently degrades to "pass", which is the UNREADABLE-vs-NONE shape The gate reads a file in this repo, and nothing else.
#90 fixed. The gate reads a file in this repo, and nothing else.
## What the gate requires ## What the gate requires
The `drill-recorded` guard (heavy-duty/ceremony's action, pinned in `.github/scripts/drill-recorded.sh` runs on every PR. On a `-dev` tree it
`ci.yml`) runs on every PR. On a `-dev` tree it
asserts nothing — a development tree has no release to evidence. On a bare asserts nothing — a development tree has no release to evidence. On a bare
`VERSION` — a release ceremony tree — it requires `drills/<version>.md` to `VERSION` — a release ceremony tree — it requires `drills/<version>.md` to
exist and to hold at least one non-whitespace character. An empty file, or one exist and to hold at least one non-whitespace character. An empty file, or one
@ -41,16 +39,13 @@ not success.
## The drill ## The drill
rig's legs (#105; `drill/drill.sh` runs them): rig's legs:
- `rig bootstrap <role>` converges the machine to its role — then runs - tenant guests minted and converged **via box**
**again**, and the captured state must diff **empty** (idempotence,
decided mechanically). On a host=yes role this is also what installs the
pinned box and asserts its host stack stands.
- `bash test/db-integration.sh` against a real Postgres on the machine - `bash test/db-integration.sh` against a real Postgres on the machine
- the GitHub runner lifecycle — register, take a job, deregister — against a - the GitHub runner lifecycle — register, take a job, deregister — against a
fork fork
- a coolify install, pinned, `AUTOUPDATE=false` - a coolify install
box and rig are **mutually recursive**: `rig bootstrap --host yes` installs box box and rig are **mutually recursive**: `rig bootstrap --host yes` installs box
and runs box's `setup-host`, while box's guests converge back through rig's and runs box's `setup-host`, while box's guests converge back through rig's
@ -63,11 +58,7 @@ schedule, in separate sittings. What makes that safe is that every drill
**pins the same fixed set of candidate refs**: rig's drill runs `--host yes` **pins the same fixed set of candidate refs**: rig's drill runs `--host yes`
with `BOX_REF=release/<box-version>`, so it exercises the box that will with `BOX_REF=release/<box-version>`, so it exercises the box that will
actually ship; box's drill mints with `RIG_REF=release/<rig-version>`, so it actually ship; box's drill mints with `RIG_REF=release/<rig-version>`, so it
exercises the rig that will actually ship. Both measure the same pair. The exercises the rig that will actually ship. Both measure the same pair.
record also cites the **rig-templates SHA** the converge read (#110) — the
candidate tree's `RIG_TEMPLATES_PIN` unless the drill was pointed elsewhere
via `RIG_TEMPLATES_REF` — so the mechanism+registry pair a release freezes
is the pair the drill proved.
That — not sequencing — is what dissolves the box↔rig recursion. The refs are That — not sequencing — is what dissolves the box↔rig recursion. The refs are
static identifiers that exist as soon as the release branches do, long before static identifiers that exist as soon as the release branches do, long before
@ -108,13 +99,11 @@ Candidate refs: box@1a2b3c4 (BOX_REF=release/0.4.0), rig@5d6e7f8, cast@9a0b1c2.
| Leg | Result | | Leg | Result |
| --- | --- | | --- | --- |
| convergence — bootstrap staging-server reaches its role | PASS (312s) | | tenant guests minted + converged via box | 3/3 |
| re-converge (idempotence) | clean, no changes | | re-converge (idempotence) | clean, no changes |
| --host yes: pinned box installed, host stack up | PASS — box doctor clean | | `test/db-integration.sh` | 14/14 |
| `test/db-integration.sh` | PASS — 14 passed, 0 failed |
| runner lifecycle against a fork | PASS — registered, took a job, deregistered clean | | runner lifecycle against a fork | PASS — registered, took a job, deregistered clean |
| forgejo runner lifecycle (you/drill-probe) | PASS — registered, took a job, removed (stale row needs deleting by hand) | | coolify install | PASS, ~6 min |
| coolify install (4.1.2) | PASS (6 min) |
Failed: `rig users apply` left one revoked key in `authorized_keys` Failed: `rig users apply` left one revoked key in `authorized_keys`
(filed #NNN). Everything else clean. (filed #NNN). Everything else clean.

164
install.sh
View file

@ -29,19 +29,12 @@ set -euo pipefail
# protect, and the operator flipping versions on purpose is the common case. # protect, and the operator flipping versions on purpose is the common case.
# A pre-versioning flat tree is migrated in place, so upgrading is seamless. # A pre-versioning flat tree is migrated in place, so upgrading is seamless.
# #
# RIG_INSTALL_SOURCE=<dir-or-tarball> is the LOCAL channel, a supported input # RIG_INSTALL_SOURCE=<dir-or-tarball> installs from a local tree instead of
# like RIG_REF (#106): installs from that tree instead of downloading — CI's # downloading — for CI and the test suite, so what lands is the code under
# install-lifecycle job and the test suites use it, so what lands is the code # review.
# under review. A path that is neither refuses by name, never falls back to
# a download.
REPO="${RIG_REPO:-heavy-duty/rig}" REPO="${RIG_REPO:-heavy-duty/rig}"
REF="${RIG_REF:-}" # empty = the latest release, resolved below REF="${RIG_REF:-}" # empty = the latest release, resolved below
# The forge this REPO lives on is RIG_HOST (#111), default https://github.com.
# Parallel to RIG_TEMPLATES_HOST — not the same variable, because the registry
# and rig itself may live on different forges. Default stays GitHub so every
# existing curl|bash one-liner is byte-unchanged; set
# RIG_HOST=https://forgejo.heavyduty.builders to install from this instance.
# cloud-init's runcmd runs with NO $HOME in the environment, and under set -u # cloud-init's runcmd runs with NO $HOME in the environment, and under set -u
# the expansions just below turned that into a death instead of an install — # the expansions just below turned that into a death instead of an install —
@ -106,18 +99,11 @@ warn_bootstrapped() { # $1 = what is about to happen
# (curl's %{redirect_url} is that header, parsed): no API, no token, no # (curl's %{redirect_url} is that header, parsed): no API, no token, no
# rate-limit pain. A repo with no releases redirects to /releases — not to # rate-limit pain. A repo with no releases redirects to /releases — not to
# /releases/tag/<tag> — so this returns 1 there instead of inventing a ref, # /releases/tag/<tag> — so this returns 1 there instead of inventing a ref,
# and the CALLER owns the loud story. Host comes from RIG_HOST (default # and the CALLER owns the loud story. test/release.sh extracts this function
# GitHub); both GitHub and Forgejo serve the same /releases/latest → # (awk, the valid_version idiom) and drives it against a stubbed curl.
# /releases/tag/<tag> redirect grammar, measured 2026-07-29 (#111).
# test/release.sh extracts this function (awk, the valid_version idiom) and
# drives it against a stubbed curl.
resolve_latest_tag() { resolve_latest_tag() {
# Default is inlined (not $RIG_HOST_DEFAULT) so test/release.sh's awk local loc
# extract of this function stays self-contained — same discipline as loc="$(curl -fsSI -o /dev/null -w '%{redirect_url}' "https://github.com/$1/releases/latest")" || return 1
# valid_version.
local host="${RIG_HOST:-https://github.com}" loc
host="${host%/}"
loc="$(curl -fsSI -o /dev/null -w '%{redirect_url}' "$host/$1/releases/latest")" || return 1
case "$loc" in case "$loc" in
*/releases/tag/?*) printf '%s\n' "${loc##*/releases/tag/}" ;; */releases/tag/?*) printf '%s\n' "${loc##*/releases/tag/}" ;;
*) return 1 ;; *) return 1 ;;
@ -125,69 +111,12 @@ resolve_latest_tag() {
} }
# ref_candidate_urls <owner/repo> <ref> — the download candidates for an # ref_candidate_urls <owner/repo> <ref> — the download candidates for an
# explicit RIG_REF, in order. Host comes from RIG_HOST. Both GitHub and # explicit RIG_REF, in order: refs/tags first, so a tag always outranks a
# Forgejo (measured 2026-07-29 on forgejo.heavyduty.builders 8.0.3) serve # branch that happens to share its name (the pin must win), refs/heads as
# the same two paths and the same disambiguation: refs/tags first so a pin # the fallback that keeps RIG_REF=main the dev channel.
# always outranks a same-named branch, then refs/heads for RIG_REF=main.
# Host is the only forge-specific input — no second grammar (#111).
#
# The RELEASE channel (RIG_REF unset) must NOT use this list: a missing tag
# archive must fail loudly, never fall through to a same-named branch and
# still report the resolved tag in INSTALLED_FROM. Use release_tag_url.
ref_candidate_urls() { ref_candidate_urls() {
local host="${RIG_HOST:-https://github.com}" printf 'https://github.com/%s/archive/refs/tags/%s.tar.gz\n' "$1" "$2"
host="${host%/}" printf 'https://github.com/%s/archive/refs/heads/%s.tar.gz\n' "$1" "$2"
printf '%s/%s/archive/refs/tags/%s.tar.gz\n' "$host" "$1" "$2"
printf '%s/%s/archive/refs/heads/%s.tar.gz\n' "$host" "$1" "$2"
}
# release_tag_url <owner/repo> <tag> — the RELEASE channel is tag-only on
# every forge (#111 / #32). One URL, refs/tags only: if that archive is
# gone the install dies, it never quietly takes refs/heads/<tag>.
release_tag_url() {
local host="${RIG_HOST:-https://github.com}"
host="${host%/}"
printf '%s/%s/archive/refs/tags/%s.tar.gz\n' "$host" "$1" "$2"
}
# install_script_url — the curl|bash entrypoint URL for this REPO on RIG_HOST.
# GitHub serves raw files at raw.githubusercontent.com; Forgejo at
# /raw/branch/<ref>/<path>. The refusal hint and bin/rig usage() both print
# this, so a Forgejo install never tells the operator to hit a 404 (#111).
# REPO is the installer's global (RIG_REPO); tests that extract this function
# must set it.
install_script_url() {
local host="${RIG_HOST:-https://github.com}"
host="${host%/}"
case "$host" in
https://github.com|http://github.com|*//github.com)
printf 'https://raw.githubusercontent.com/%s/main/install.sh\n' "${REPO:-heavy-duty/rig}" ;;
*)
printf '%s/%s/raw/branch/main/install.sh\n' "$host" "${REPO:-heavy-duty/rig}" ;;
esac
}
# The registry's candidate URLs, forge-aware — a byte-identical copy of
# commands/lib/templates.sh's, diffed by test/cli.sh so the two cannot drift
# (#109; the valid_version / warn_bootstrapped precedent). One decision about
# where a registry lives, made in one grammar: a snapshot fetched from a forge
# converge would never fetch from is worse than no snapshot at all.
#
# Copied rather than sourced on purpose. snapshot_templates runs against an
# extracted tree, so sourcing WOULD work here — but it would make the
# installer's behaviour depend on executing code from the tarball it just
# downloaded, ahead of any of it being installed. The installer reads that
# tree (sed for the pin); it does not run it.
templates_archive_urls() {
local host="${1%/}" repo="$2" ref="$3"
case "$host" in
https://github.com|http://github.com|*//github.com)
printf '%s/%s/archive/refs/tags/%s.tar.gz\n' "$host" "$repo" "$ref"
printf '%s/%s/archive/refs/heads/%s.tar.gz\n' "$host" "$repo" "$ref"
printf '%s/%s/archive/%s.tar.gz\n' "$host" "$repo" "$ref" ;;
*)
printf '%s/%s/archive/%s.tar.gz\n' "$host" "$repo" "$ref" ;;
esac
} }
# --- prerequisites ----------------------------------------------------------- # --- prerequisites -----------------------------------------------------------
@ -269,15 +198,12 @@ else
if [ -z "$REF" ]; then if [ -z "$REF" ]; then
log "resolving the latest release of $REPO" log "resolving the latest release of $REPO"
if ! REF="$(resolve_latest_tag "$REPO")"; then if ! REF="$(resolve_latest_tag "$REPO")"; then
warn "could not resolve the latest release of $REPO — either no release exists yet, or ${RIG_HOST:-https://github.com} was unreachable." warn "could not resolve the latest release of $REPO — either no release exists yet, or GitHub was unreachable."
warn "(install the development tree explicitly with RIG_REF=main when no release exists yet.)" warn "(rig has no release until 0.1.0 is cut — rig#32. Until then, install the development tree explicitly.)"
die "set RIG_REF: e.g. curl -fsSL $(install_script_url) | RIG_REF=main bash" die "set RIG_REF: e.g. curl -fsSL https://raw.githubusercontent.com/$REPO/main/install.sh | RIG_REF=main bash"
fi fi
log "latest release: $REF" log "latest release: $REF"
# Tag-only: the channel resolved a RELEASE tag, so the download is that urls=("https://github.com/$REPO/archive/refs/tags/$REF.tar.gz")
# tag's archive and nothing else. Falling through to refs/heads would
# install a branch while INSTALLED_FROM still names the tag (#111 review).
mapfile -t urls < <(release_tag_url "$REPO" "$REF")
else else
mapfile -t urls < <(ref_candidate_urls "$REPO" "$REF") mapfile -t urls < <(ref_candidate_urls "$REPO" "$REF")
fi fi
@ -293,7 +219,7 @@ else
fi fi
done done
[ -n "$got" ] \ [ -n "$got" ] \
|| die "failed to download $REPO@$REF — no candidate URL worked (host ${RIG_HOST:-https://github.com}; tried ${urls[*]})" || die "failed to download $REPO@$REF — not a tag and not a branch (tried refs/tags then refs/heads)"
log "extracting archive" log "extracting archive"
tar -xzf "$TMPDIR/rig.tar.gz" -C "$TMPDIR" \ tar -xzf "$TMPDIR/rig.tar.gz" -C "$TMPDIR" \
@ -321,60 +247,6 @@ set_exec() { # $1 = a rig tree: the executable bits install.sh owns
fi fi
} }
# snapshot_templates <rig-tree> — best-effort install-time cache of the exact
# registry pin carried by that tree. The pin remains the sole source of truth;
# the directory name makes a stale snapshot invisible after an upgrade.
# Failure is deliberately a warning: rig itself is still a complete install,
# and templates_resolve preserves the live-fetch fallback.
snapshot_templates() {
local tree="$1" pin repo host url got="" unpack top snapshot
pin="$(sed -n 's/^RIG_TEMPLATES_PIN=//p' "$tree/commands/lib/templates.sh" 2>/dev/null | head -n1 || true)"
if [ -z "$pin" ]; then
warn "installed tree carries no RIG_TEMPLATES_PIN; template registry snapshot skipped."
return 0
fi
repo="${RIG_TEMPLATES_REPO:-heavy-duty/rig-templates}"
# The same forge knob templates_resolve reads, from the same variable (#109).
# Snapshot and live fetch MUST agree about where the registry lives: an
# install that cached from GitHub while converge fetches from Forgejo would
# serve a snapshot the pin never named, and the pin-in-the-directory-name
# staleness guard cannot catch a WRONG-ORIGIN snapshot, only an old one.
host="${RIG_TEMPLATES_HOST:-https://github.com}"
snapshot="$tree/templates@$pin"
if ! command -v curl >/dev/null 2>&1; then
warn "curl is unavailable; template registry snapshot $repo@$pin was not installed (converge will retry the live fetch)."
return 0
fi
unpack="$TMPDIR/templates-unpack"
rm -rf "$unpack"
mkdir -p "$unpack"
log "downloading template registry snapshot ${host%/}/$repo@$pin"
while IFS= read -r url; do
if curl -fsSL "$url" -o "$TMPDIR/templates.tar.gz" 2>/dev/null; then got="$url"; break; fi
done < <(templates_archive_urls "$host" "$repo" "$pin")
if [ -z "$got" ]; then
warn "could not fetch template registry snapshot $repo@$pin; rig installed without it (converge will retry the live fetch)."
return 0
fi
if ! tar -xzf "$TMPDIR/templates.tar.gz" -C "$unpack"; then
warn "could not extract template registry snapshot from $got; rig installed without it (converge will retry the live fetch)."
return 0
fi
set -- "$unpack"/*/
if ! { [ $# -eq 1 ] && [ -d "$1" ]; }; then
warn "template registry snapshot from $got has an unexpected archive shape; rig installed without it (converge will retry the live fetch)."
return 0
fi
top="${1%/}"
if [ -z "$(find "$top" -mindepth 2 -maxdepth 2 -type f -name template.env -print -quit 2>/dev/null)" ]; then
warn "template registry snapshot from $got has no definitions; rig installed without it (converge will retry the live fetch)."
return 0
fi
rm -rf "$snapshot"
mv "$top" "$snapshot"
log "template registry snapshot installed: $repo@$pin"
}
# --- install into $DEST/versions/<version> ----------------------------------- # --- install into $DEST/versions/<version> -----------------------------------
VDIR="$DEST/versions/$new_ver" VDIR="$DEST/versions/$new_ver"
newly_installed=0 newly_installed=0
@ -385,7 +257,6 @@ if [ -d "$VDIR" ]; then
log "RIG_REINSTALL=1 — replacing the installed $new_ver tree" log "RIG_REINSTALL=1 — replacing the installed $new_ver tree"
stage="$VDIR.new.$$"; old="$VDIR.old.$$" stage="$VDIR.new.$$"; old="$VDIR.old.$$"
rm -rf "$stage" "$old" rm -rf "$stage" "$old"
snapshot_templates "$EXTRACTED"
set_exec "$EXTRACTED" set_exec "$EXTRACTED"
mv "$EXTRACTED" "$stage" mv "$EXTRACTED" "$stage"
# Swap by renames, delete LAST: rm-then-move leaves a hole the whole # Swap by renames, delete LAST: rm-then-move leaves a hole the whole
@ -403,7 +274,6 @@ if [ -d "$VDIR" ]; then
else else
log "installing $new_ver into $VDIR" log "installing $new_ver into $VDIR"
mkdir -p "$DEST/versions" mkdir -p "$DEST/versions"
snapshot_templates "$EXTRACTED"
set_exec "$EXTRACTED" set_exec "$EXTRACTED"
mv "$EXTRACTED" "$VDIR" mv "$EXTRACTED" "$VDIR"
newly_installed=1 newly_installed=1

1637
test/cli.sh
View file

File diff suppressed because it is too large Load diff

402
test/drill.sh
View file

@ -1,402 +0,0 @@
#!/usr/bin/env bash
# test/drill.sh — the drill harness's HONESTY, proven without hardware.
#
# drill/drill.sh is the instrument (#105), so what this suite tests is the
# instrument itself: the refusals, the classifications, the capture-and-diff
# that decides idempotence, and the record emitter — the parts whose lies
# would be believed, months later, by a reader of drills/<version>.md. The
# four-leg live run on a real Debian machine is #107's exercise, not this
# file's: nothing here needs root, Docker, a tailnet or the network.
#
# Extraction pattern is test/release.sh's: the functions under test are
# awk-extracted from drill/drill.sh and driven against fixtures, so the tests
# exercise the shipped bytes, and the extraction check itself guards the awk
# against a drifted function boundary.
# Deliberately no `set -e` — the harness asserts on failing commands.
set -u
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT" || exit 1
PASS=0 FAIL=0
# check <desc> <want_exit> <want_substr> <cmd...>
check() {
local desc="$1" want="$2" substr="$3"; shift 3
local out rc
out="$("$@" 2>&1)"; rc=$?
if [ "$rc" -ne "$want" ]; then
echo "FAIL: $desc — exit $rc, wanted $want"
printf '%s\n' "$out" | sed 's/^/ /'
FAIL=$((FAIL + 1)); return
fi
if [ -n "$substr" ] && ! printf '%s' "$out" | grep -qF -e "$substr"; then
echo "FAIL: $desc — output missing '$substr'"
printf '%s\n' "$out" | sed 's/^/ /'
FAIL=$((FAIL + 1)); return
fi
echo "ok: $desc"; PASS=$((PASS + 1))
}
# refute <desc> <substr> <file> — the file must NOT contain the substring.
refute() {
if grep -qF -e "$2" "$3"; then
echo "FAIL: $1 — found forbidden '$2'"
FAIL=$((FAIL + 1)); return
fi
echo "ok: $1"; PASS=$((PASS + 1))
}
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
# --- the functions under test, extracted -------------------------------------
FNS="$WORK/drill-fns.sh"
for fn in tree_of assert_installed_from classify_leg capture_state emit_record forgejo_run_verdict forgejo_token_verdict forgejo_max_task_id forgejo_leg_row; do
awk "/^${fn}\(\) \{/,/^\}/" "$ROOT/drill/drill.sh" >> "$FNS"
done
for fn in tree_of assert_installed_from classify_leg capture_state emit_record forgejo_run_verdict forgejo_token_verdict forgejo_max_task_id forgejo_leg_row; do
check "extraction guards the awk: ${fn}() landed" 0 "${fn}() {" grep -F "${fn}() {" "$FNS"
done
# shellcheck source=/dev/null
. "$FNS"
# =============================================================================
# tree_of — the versioned tree behind a CLI's symlink chain
# =============================================================================
IR="$WORK/install"; mkdir -p "$IR/versions/1.2.3/bin"
: > "$IR/versions/1.2.3/bin/rig"
ln -s "versions/1.2.3" "$IR/current"
mkdir -p "$WORK/bin"
ln -s "$IR/current/bin/rig" "$WORK/bin/rig"
check "tree_of resolves a current-symlink chain to versions/<v>" 0 "$IR/versions/1.2.3" \
tree_of "$WORK/bin/rig"
ln -s "$IR/gone/bin/rig" "$WORK/bin/dangling"
check "tree_of refuses a dangling chain — a tree that is not there is not a tree" 1 "" \
tree_of "$WORK/bin/dangling"
# =============================================================================
# assert_installed_from — the up-front ref refusal, naming both refs
# =============================================================================
TREE="$WORK/tree-main"; mkdir -p "$TREE"
printf 'heavy-duty/rig@main\n' > "$TREE/INSTALLED_FROM"
check "matching INSTALLED_FROM passes silently" 0 "" \
assert_installed_from rig "$TREE" "heavy-duty/rig@main"
check "a mismatch refuses (the #103 hazard: asked release, got main)" 1 "FATAL" \
assert_installed_from rig "$TREE" "heavy-duty/rig@release/9.9.9"
check "…the refusal names the ref that was ASKED for" 1 "heavy-duty/rig@release/9.9.9" \
assert_installed_from rig "$TREE" "heavy-duty/rig@release/9.9.9"
check "…and the ref that actually LANDED" 1 "heavy-duty/rig@main" \
assert_installed_from rig "$TREE" "heavy-duty/rig@release/9.9.9"
check "an unreadable INSTALLED_FROM refuses too — absence is not a match" 1 "<unreadable>" \
assert_installed_from rig "$WORK/no-such-tree" "heavy-duty/rig@main"
# =============================================================================
# classify_leg — a loud skip is a SKIP, never a pass (box#153's defect class)
# =============================================================================
printf 'skip: docker not installed — nothing to exercise\n' > "$WORK/out-skip"
printf 'ok: seeded\nok: restored\n---\n14 passed, 0 failed\n' > "$WORK/out-pass"
printf 'FAIL: restore blew up\n' > "$WORK/out-fail"
check "exit 0 + 'skip:' line classifies as skip" 0 "skip" classify_leg 0 "$WORK/out-skip"
check "exit 0, no skip line, classifies as pass" 0 "pass" classify_leg 0 "$WORK/out-pass"
check "non-zero exit classifies as fail" 0 "fail" classify_leg 1 "$WORK/out-fail"
check "a skip line cannot rescue a non-zero exit (fail wins)" 0 "fail" \
classify_leg 1 "$WORK/out-skip"
# =============================================================================
# capture_state + diff — the idempotence verdict's machinery. The claim in
# #105's acceptance criteria: the assertion is a REAL diff of captured state,
# and it FAILS when convergence is broken — demonstrated here, mechanically,
# on every CI run, by breaking the state between two captures.
# =============================================================================
FIX="$WORK/fix"; mkdir -p "$FIX/sudoers.d"
printf 'role=staging-server root-door=open host=yes join=authkey\n' > "$FIX/role"
printf 'schema=1\nbootstrapped_by=9.9.9\nbootstrapped_at=T\nconverged_by=9.9.9\nconverged_at=T\n' > "$FIX/manifest"
printf 'dan active\nghost revoked\n' > "$FIX/ledger"
printf 'APT::Periodic::Update-Package-Lists "1";\n' > "$FIX/autoup"
printf '127.0.0.1 localhost\n127.0.1.1\tstaging-server\n' > "$FIX/hosts"
printf 'nosuchdrilluser ALL=(ALL) NOPASSWD:ALL\n' > "$FIX/sudoers.d/00-rig-nosuch"
# A stubbed sshd, so the effective-config section is exercised rather than
# skipped on a box with no daemon (repo precedent: test/release.sh's curl).
STUB="$WORK/stub"; mkdir -p "$STUB"
# The single-quoted $SSHD_FIXTURE is the STUB's expansion, not this shell's.
# shellcheck disable=SC2016
printf '#!/usr/bin/env bash\ncat "$SSHD_FIXTURE"\n' > "$STUB/sshd"; chmod +x "$STUB/sshd"
printf 'passwordauthentication no\npermitrootlogin prohibit-password\n' > "$FIX/sshd-T"
cap() { # cap <outfile> — capture_state against the fixture set
RIG_ROLE_MARKER="$FIX/role" RIG_MANIFEST="$FIX/manifest" \
DRILL_LEDGER="$FIX/ledger" DRILL_AUTOUPGRADES="$FIX/autoup" \
DRILL_ETC_HOSTS="$FIX/hosts" DRILL_SUDOERS_DIR="$FIX/sudoers.d" \
SSHD_FIXTURE="$FIX/sshd-T" PATH="$STUB:$PATH" \
bash -c '. "$1"; capture_state "$2"' _ "$FNS" "$2" 2>/dev/null
:
}
# cap runs capture_state in a child bash so the PATH stub cannot leak into
# this harness; $2 arrives as the capture's outfile.
cap out "$WORK/cap1"
cap out "$WORK/cap2"
check "two captures over untouched state diff EMPTY (the converged verdict)" 0 "" \
diff -u "$WORK/cap1" "$WORK/cap2"
check "the capture reads the fixtures, not the machine (marker line present)" 0 "role=staging-server" \
grep -o 'role=staging-server[^"]*' "$WORK/cap1"
check "…the sshd section captured the effective config" 0 "passwordauthentication no" \
cat "$WORK/cap1"
check "…a ledger user with no account reads as one, deterministically" 0 "(no account)" \
cat "$WORK/cap1"
# Break convergence: the re-run "changed" the role marker and root's door.
printf 'role=staging-server root-door=closed host=yes join=authkey\n' > "$FIX/role"
printf 'passwordauthentication yes\npermitrootlogin prohibit-password\n' > "$FIX/sshd-T"
cap out "$WORK/cap3"
check "a broken convergence makes the diff NON-empty — the assertion can fail" 1 "root-door=closed" \
diff -u "$WORK/cap1" "$WORK/cap3"
check "…and the diff names the drifted sshd keyword, not just 'differs'" 1 "passwordauthentication yes" \
diff -u "$WORK/cap1" "$WORK/cap3"
# =============================================================================
# emit_record — the record is drills/README.md's shape, and it cannot lie:
# a failed run still emits, a skipped leg is named, no clean-sweep reading.
# =============================================================================
emit() { # emit <outfile> — emit_record with the harness globals staged
DRILL_VERSION="9.9.9" RUN_ID="drill-2026-01-01-a" \
REF="release/9.9.9" BOXREF="release/0.4.0" RIG_SHA="5d6e7f8" BOX_SHA="1a2b3c4" \
TPLREPO="heavy-duty/rig-templates" TPLREF="9f8e7d6c5b4a39281706f5e4d3c2b1a098765432" TPL_SHA="9f8e7d6" TPL_SOURCE="snapshot" \
bash -c '
. "$1"
pass=12 fail=1 skipped=1
findings=("FAIL: coolify container state: absent" "SKIP: runner lifecycle: no --runner-repo fork given — the leg did not run" "NOTE: something worth a line")
LEG_NAMES=("convergence — bootstrap staging-server reaches its role" "re-converge (idempotence)" "coolify install (4.1.2)" "runner lifecycle")
LEG_RESULTS=("PASS (312s)" "clean, no changes" "FAIL — container absent" "SKIPPED — no fork provided")
emit_record "$2"
' _ "$FNS" "$2"
}
emit out "$WORK/record.md"
check "record: the version-and-date heading" 0 "# Release drill — 9.9.9 — " head -1 "$WORK/record.md"
check "record: the run ID that joins the family's records" 0 "Run ID: drill-2026-01-01-a" cat "$WORK/record.md"
check "record: both pinned refs with their SHAs" 0 "rig@5d6e7f8 (RIG_REF=release/9.9.9)" cat "$WORK/record.md"
check "record: …box's too" 0 "box@1a2b3c4 (BOX_REF=release/0.4.0)" cat "$WORK/record.md"
check "record: the template registry SHA and actual source ride alongside the pair (#110/#153)" 0 "rig-templates@9f8e7d6 (ref 9f8e7d6c5b4a39281706f5e4d3c2b1a098765432, snapshot)" cat "$WORK/record.md"
check "record: one table row per leg, result verbatim" 0 "| re-converge (idempotence) | clean, no changes |" cat "$WORK/record.md"
check "record: the numbers, skips counted apart from passes" 0 "12 passed, 1 failed, 1 skipped" cat "$WORK/record.md"
check "record: a FAILED run still names what failed (evidence, not success)" 0 "FAIL: coolify container state: absent" cat "$WORK/record.md"
check "record: a skipped leg is stated as NOT run, by name" 0 "SKIP: runner lifecycle" cat "$WORK/record.md"
check "record: the skip section says the record is not evidence for it" 0 "not evidence" cat "$WORK/record.md"
check "record: the isolation boundary is named as box's, in words" 0 "NOT asserted here" cat "$WORK/record.md"
refute "record with a skip cannot read as a clean sweep" "Failed: nothing" "$WORK/record.md"
refute "notes are findings for the log, not failures for the record" "NOTE: something" "$WORK/record.md"
# The all-green shape: says so plainly, and only then.
DRILL_VERSION="9.9.9" RUN_ID="drill-2026-01-01-a" \
REF="release/9.9.9" BOXREF="release/0.4.0" RIG_SHA="5d6e7f8" BOX_SHA="1a2b3c4" \
bash -c '
. "$1"
pass=20 fail=0 skipped=0
findings=()
LEG_NAMES=("convergence" "re-converge (idempotence)")
LEG_RESULTS=("PASS" "clean, no changes")
emit_record "$2"
' _ "$FNS" "$WORK/record-green.md"
check "an all-green record says every leg ran and passed" 0 "Every leg ran and every check passed" \
cat "$WORK/record-green.md"
# =============================================================================
# the shipped script itself
# =============================================================================
# =============================================================================
# forgejo_run_verdict — did OUR dispatched run land, and how (#129)
# =============================================================================
# The Forgejo half of the runner leg cannot reuse the GitHub reader. Measured
# against forgejo.heavyduty.builders (8.0.3+gitea-1.22.0) on 2026-07-30, a
# completed run in GET /repos/{o}/{r}/actions/tasks carries NO `conclusion`
# field at all — `status` holds the terminal outcome directly, where GitHub
# splits status:completed + conclusion:success. And `id` is a global task id
# (25) while the run's own URL ends in run_number (1), so the pre-dispatch
# guard has to compare `id`.
#
# The payload is also an ASSIGNED-task view: it reads total_count 0 for as
# long as a run sits queued (measured: 200s), so "no new id" is the ONLY
# signal that the runner never took the job. That is the verdict this leg
# exists to produce, which is why it gets its own function and its own tests.
FJ="$WORK/fj"; mkdir -p "$FJ"
printf '%s' '{"workflow_runs":[],"total_count":0}' > "$FJ/empty.json"
printf '%s' '{"workflow_runs":[{"id":25,"status":"success","run_number":1,"url":"https://f/o/r/actions/runs/1"}],"total_count":1}' > "$FJ/new-success.json"
printf '%s' '{"workflow_runs":[{"id":25,"status":"failure","run_number":1,"url":"https://f/o/r/actions/runs/1"}],"total_count":1}' > "$FJ/new-failure.json"
printf '%s' '{"workflow_runs":[{"id":25,"status":"cancelled","run_number":1,"url":"https://f/o/r/actions/runs/1"}],"total_count":1}' > "$FJ/new-cancelled.json"
printf '%s' '{"workflow_runs":[{"id":24,"status":"success","run_number":1,"url":"https://f/o/r/actions/runs/1"}],"total_count":1}' > "$FJ/stale-only.json"
check "verdict: an empty task list is PENDING, never a pass" 0 "pending" \
forgejo_run_verdict "" "$FJ/empty.json"
check "verdict: a queued run the runner never took stays PENDING" 0 "pending" \
forgejo_run_verdict "24" "$FJ/stale-only.json"
check "verdict: OUR new run, status success, is SUCCESS" 0 "success" \
forgejo_run_verdict "24" "$FJ/new-success.json"
check "verdict: status carries the outcome — failure is FAILED, not success" 0 "failed" \
forgejo_run_verdict "24" "$FJ/new-failure.json"
check "verdict: a cancelled run is FAILED, not silently passed" 0 "failed" \
forgejo_run_verdict "24" "$FJ/new-cancelled.json"
check "verdict: the first run ever (no pre-id) still resolves" 0 "success" \
forgejo_run_verdict "" "$FJ/new-success.json"
# A task appears in this payload the moment it is ASSIGNED, which can be before
# it finishes — so a non-terminal status must read as pending, not as a failure.
# Calling a still-running job "failed" would make the leg flaky in exactly the
# window the leg is watching.
printf '%s' '{"workflow_runs":[{"id":25,"status":"running","run_number":1}],"total_count":1}' > "$FJ/new-running.json"
check "verdict: an assigned-but-running task is PENDING, not FAILED" 0 "pending" \
forgejo_run_verdict "24" "$FJ/new-running.json"
# grok/kimi on !130: the reader must not stop at the FIRST run. actions/tasks
# accumulates — the moment a repo is drilled twice, our run shares the payload
# with older ones, and nothing documents the sort order. Reading entry[0] makes
# a green job read as a timeout, which is a FALSE FAILURE on the very gate this
# leg exists to provide.
printf '%s' '{"workflow_runs":[{"id":24,"status":"success"},{"id":25,"status":"success"}],"total_count":2}' > "$FJ/oldest-first.json"
printf '%s' '{"workflow_runs":[{"id":25,"status":"success"},{"id":24,"status":"success"}],"total_count":2}' > "$FJ/newest-first.json"
printf '%s' '{"workflow_runs":[{"id":25,"status":"running"},{"id":26,"status":"success"}],"total_count":2}' > "$FJ/ours-not-first.json"
printf '%s' '{"workflow_runs":[{"id":23,"status":"success"},{"id":24,"status":"failure"}],"total_count":2}' > "$FJ/all-stale.json"
# Pretty-printed: the instance may or may not compact its JSON, and a parser
# that silently depends on one-line objects is a latent failure (kimi, !130).
# Written HERE, like every other fixture: a suite that copies from a scratch
# path passes only on the box that built it (grok/kimi, !130 round 2).
printf '%s\n' '{
"workflow_runs": [
{"id": 24, "status": "success"},
{"id": 25, "name": "drill", "status": "success"}
],
"total_count": 2
}' > "$FJ/pretty.json"
check "verdict: ours is LAST in the payload — order must not decide" 0 "success" \
forgejo_run_verdict "24" "$FJ/oldest-first.json"
check "verdict: ours is FIRST in the payload — same answer" 0 "success" \
forgejo_run_verdict "24" "$FJ/newest-first.json"
check "verdict: a stale RUNNING entry ahead of ours does not mask it" 0 "success" \
forgejo_run_verdict "24" "$FJ/ours-not-first.json"
check "verdict: every entry at or below pre is stale — PENDING" 0 "pending" \
forgejo_run_verdict "24" "$FJ/all-stale.json"
check "verdict: a pretty-printed payload parses too" 0 "success" \
forgejo_run_verdict "24" "$FJ/pretty.json"
# The PRE-DISPATCH snapshot has the same multi-entry hazard as the verdict, and
# getting it wrong is worse: a `head -n1` pre-id on an oldest-first payload
# names an OLD run as the baseline, so a later poll that finds the same body
# reports the PREVIOUS drill's run as ours — a false PASS on the take-a-job
# assertion, where the entry[0] bug only produced a false failure (grok, !130).
# Both sides must fold max over every id, which is why they share one function.
check "max id: oldest-first payload yields the NEWEST id, not the first" 0 "25" \
forgejo_max_task_id "$FJ/oldest-first.json"
check "max id: newest-first payload yields the same answer" 0 "25" \
forgejo_max_task_id "$FJ/newest-first.json"
check "max id: an empty payload has no id at all" 0 "" \
forgejo_max_task_id "$FJ/empty.json"
check "max id: a pretty-printed payload folds too" 0 "25" \
forgejo_max_task_id "$FJ/pretty.json"
# The false PASS, pinned end to end: snapshot the oldest-first body, dispatch,
# the runner never takes it so the body is unchanged — the verdict must stay
# pending. With head -n1 this returned success.
# The two halves composed exactly as the leg composes them.
verdict_after_no_new_run() { forgejo_run_verdict "$(forgejo_max_task_id "$1")" "$1"; }
check "no new run after dispatch: max-id baseline keeps it PENDING (false-PASS guard)" 0 "pending" \
verdict_after_no_new_run "$FJ/oldest-first.json"
check "…and the same composition on a pretty payload" 0 "pending" \
verdict_after_no_new_run "$FJ/pretty.json"
# =============================================================================
# forgejo_leg_row — the row is the WHOLE lifecycle, not just the job
# =============================================================================
# codex/grok/kimi on !130: keying the record row on the take-a-job outcome alone
# lets it read "PASS — registered, took a job, removed" when install failed, so
# long as SOMETHING answered runs-on: drill. That is not contrived — this leg
# removes locally and tells the operator to delete the stale runner by hand, so
# a leftover drill-labeled runner from the previous drill is the DESIGNED-FOR
# aftermath, and it answers the fixture exactly.
#
# drills/<v>.md is the release's durable evidence. A row claiming a lifecycle
# that did not happen is precisely what the gate exists to refuse, so PASS
# requires every assertion, not just the interesting one.
check "leg row: everything succeeded is the only PASS" 0 "PASS" \
forgejo_leg_row 1 1 success 1 1
check "leg row: install failed cannot PASS, even when a foreign runner took the job" 0 "FAIL" \
forgejo_leg_row 0 1 success 1 1
check "leg row: status failed cannot PASS either" 0 "FAIL" \
forgejo_leg_row 1 0 success 1 1
check "leg row: remove failed cannot PASS" 0 "FAIL" \
forgejo_leg_row 1 1 success 0 1
check "leg row: a runner still registered after remove cannot PASS" 0 "FAIL" \
forgejo_leg_row 1 1 success 1 0
check "leg row: no dispatch attempted, everything else clean, is PARTIAL" 0 "PARTIAL" \
forgejo_leg_row 1 1 none 1 1
check "leg row: a job that was never taken is a FAIL" 0 "FAIL" \
forgejo_leg_row 1 1 timeout 1 1
check "leg row: PARTIAL requires a clean lifecycle too" 0 "FAIL" \
forgejo_leg_row 0 1 none 1 1
# The end-to-end shape codex/grok/kimi asked for, composed the way the leg
# composes it: a tasks payload carrying a NEWER successful run (as a foreign
# drill-labeled runner would produce) must still not yield a PASS row when the
# drill's own install failed. This is the exact false-evidence case.
row_after_failed_install() {
forgejo_leg_row 0 1 "$(forgejo_run_verdict "$(forgejo_max_task_id "$1")" "$1")" 1 1
}
printf '%s' '{"workflow_runs":[{"id":24,"status":"success"},{"id":99,"status":"success"}],"total_count":2}' \
> "$FJ/foreign-runner-took-it.json"
check "install failed + a newer successful run in the payload is still FAIL, never PASS" 0 "FAIL" \
row_after_failed_install "$FJ/foreign-runner-took-it.json"
# …and the same payload with a clean lifecycle is the PASS, so the check above
# is discriminating rather than always-FAIL.
row_after_clean_install() {
forgejo_leg_row 1 1 "$(forgejo_run_verdict "" "$1")" 1 1
}
check "…while the same payload with a clean lifecycle does PASS" 0 "PASS" \
row_after_clean_install "$FJ/foreign-runner-took-it.json"
# =============================================================================
# forgejo_token_verdict — a configured leg that cannot mint must FAIL, not SKIP
# =============================================================================
# #129's own acceptance: "Token source present but the instance is unreachable
# -> the leg FAILS; it must not skip and must not pass". A mint that returns
# nothing because the instance is unreachable, the token is under-scoped or the
# repo name is wrong is a CONFIGURED leg failing — reporting "no token source"
# sends the operator to check an env var they already set, and writes SKIPPED
# where the record owes a FAIL. That is the UNREADABLE-vs-NONE shape
# drills/README.md names.
check "token: a resolved registration token is ok" 0 "ok" \
forgejo_token_verdict "reg-tok" ""
check "token: an explicit token wins even with no API token" 0 "ok" \
forgejo_token_verdict "reg-tok" ""
check "token: no token at all and no API token is a genuine SKIP" 0 "no-source" \
forgejo_token_verdict "" ""
check "token: API token offered but mint produced nothing is a FAILURE" 0 "mint-failed" \
forgejo_token_verdict "" "api-tok"
# The anti-false-positive guard, stated as its own case: an OLD completed run
# with the SAME id as pre_id must never be read as this dispatch's result.
check "verdict: a pre-existing success with the pre-id is NOT our run" 0 "pending" \
forgejo_run_verdict "25" "$FJ/new-success.json"
# Arg refusals fire before the root check (repo doctrine, bootstrap.sh:114),
# which is what makes them provable here without a throwaway machine.
check "drill.sh refuses to run without BOTH refs pinned (#103)" 2 "--box-ref" \
env -u RIG_REF -u BOX_REF bash "$ROOT/drill/drill.sh" --rig-ref release/9.9.9 --yes
check "…and the refusal shows which ref is missing" 2 "<unset>" \
env -u RIG_REF -u BOX_REF bash "$ROOT/drill/drill.sh" --rig-ref release/9.9.9 --yes
check "a tenant role is refused — the drill converges machines, not guests" 2 "not a machine role" \
bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --role claude-box --yes
check "no --users is a refusal, naming why the drill will not default it" 2 "--users <path> is required" \
bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --yes
check "an unreadable users file dies before anything is spent" 2 "cannot read users file" \
bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --users "$WORK/no-such-users" --yes
check "an unknown flag dies loudly, exit 2" 2 "unknown option" \
bash "$ROOT/drill/drill.sh" --frobnicate
check "--help prints the header and exits 0" 0 "THROWAWAY" \
bash "$ROOT/drill/drill.sh" --help
check "--forgejo-instance is a known flag (the leg's opt-in)" 2 "--users <path> is required" \
bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --forgejo-instance https://f.example.com --yes
check "--forgejo-runner-repo is a known flag" 2 "--users <path> is required" \
bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --forgejo-runner-repo o/r --yes
check "--help names the forgejo runner leg's flags" 0 "--forgejo-instance" \
bash "$ROOT/drill/drill.sh" --help
check "--forgejo-ref is a known flag (Forgejo's dispatch needs a ref)" 2 "--users <path> is required" \
bash "$ROOT/drill/drill.sh" --rig-ref r --box-ref b --forgejo-ref dev --yes
echo "---"
echo "$PASS passed, $FAIL failed"
[ "$FAIL" -eq 0 ]

189
test/install-lifecycle.sh
View file

@ -1,189 +0,0 @@
#!/usr/bin/env bash
# The install LIFECYCLE, driven end to end against a tree install.sh itself
# produced (#106) — the four beats box and cast already run in CI, which rig,
# the repo whose headline claim is convergence, ran nowhere:
#
# 1. install from THIS checkout (RIG_INSTALL_SOURCE — the local channel)
# 2. assert what landed (layout, current, the PATH chain)
# 3. a converging re-run (an EMPTY DIFF, never an exit code)
# 4. uninstall --all (ending in the absence assert)
#
# test/cli.sh drives the same verbs against throwaway roots; this suite runs
# them in the environment cli.sh deliberately fakes — the real default paths
# under the runner's own $HOME. Run: bash test/install-lifecycle.sh (CI's
# `install:` job). RIG_HOME/RIG_BIN redirect the roots for a local run; the
# refusal below explains when you need them.
#
# Deliberately no `set -e` — a failing beat is data, and the summary is the
# verdict (the test/release.sh harness shape).
set -u
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT" || exit 1
PASS=0 FAIL=0
# check <desc> <want_exit> <want_substr> <cmd...>
# Runs cmd, asserts exit code and (if non-empty) that combined output
# contains want_substr.
check() {
local desc="$1" want="$2" substr="$3"; shift 3
local out rc
out="$("$@" 2>&1)"; rc=$?
if [ "$rc" -ne "$want" ]; then
echo "FAIL: $desc — exit $rc, wanted $want"
printf '%s\n' "$out" | sed 's/^/ /'
FAIL=$((FAIL + 1)); return
fi
if [ -n "$substr" ] && ! printf '%s' "$out" | grep -qF -e "$substr"; then
echo "FAIL: $desc — output missing '$substr'"
printf '%s\n' "$out" | sed 's/^/ /'
FAIL=$((FAIL + 1)); return
fi
echo "ok: $desc"; PASS=$((PASS + 1))
}
# The roots install.sh will use, computed by ITS rules (install.sh:55-60), so
# every assert below points at what the installer actually touched.
DEST="${RIG_HOME:-$HOME/.local/share/rig}"
if [ "$(id -u)" -eq 0 ]; then
BINDIR="${RIG_BIN:-/usr/local/bin}"
else
BINDIR="${RIG_BIN:-$HOME/.local/bin}"
fi
# Beat 4 REMOVES the install at those roots, so a rig that already lives there
# is a refusal, not a fixture — this suite must never eat an operator's
# install. CI runners are clean; a workstation run points the roots at
# something disposable.
if [ -e "$DEST" ] || [ -L "$DEST" ] || [ -e "$BINDIR/rig" ] || [ -L "$BINDIR/rig" ]; then
echo "install-lifecycle: a rig install already exists ($DEST or $BINDIR/rig)" >&2
echo "install-lifecycle: refusing to drive the lifecycle over it — re-run against scratch roots:" >&2
echo " W=\$(mktemp -d); RIG_HOME=\$W/rig RIG_BIN=\$W/bin bash test/install-lifecycle.sh" >&2
exit 2
fi
VER="$(cat "$ROOT/VERSION")"
WORK="$(mktemp -d)"
trap 'rm -rf "$WORK"' EXIT
# install.sh snapshots the pinned registry even for the local source channel.
# Serve a deterministic archive so this lifecycle remains fully offline.
SNAPBIN="$WORK/snapshot-bin"
mkdir -p "$SNAPBIN" "$WORK/snapshot-stage/rig-templates-pin/test-box"
printf 'USER="test"\n' > "$WORK/snapshot-stage/rig-templates-pin/test-box/template.env"
tar -czf "$WORK/snapshot.tar.gz" -C "$WORK/snapshot-stage" rig-templates-pin
cat > "$SNAPBIN/curl" <<'CURLEOF'
#!/usr/bin/env bash
cp "${SNAPSHOT_TARBALL:?}" "$4"
CURLEOF
chmod +x "$SNAPBIN/curl"
# tree_state <root> — what "changed nothing" must mean: every file's bytes,
# every path's type and mode, every symlink's target. Beat 3 captures this
# before and after the re-run and diffs the two.
tree_state() {
(cd "$1" || return 1
find . -type f -exec sha256sum {} + | LC_ALL=C sort
find . -type l -printf '%p -> %l\n' | LC_ALL=C sort
find . -printf '%y %m %p\n' | LC_ALL=C sort)
}
diff_state() { # diff_state <capture> <root> — how the tree drifted, by name
tree_state "$2" | diff "$1" -
}
no_residue() { # no_residue <path>... — 0 iff every path is GONE: file, dir OR
local p bad=0 # symlink. `! -e` alone follows the link and cannot see it
for p in "$@"; do # dangling — the residue a broken uninstall actually leaves.
if [ -e "$p" ] || [ -L "$p" ]; then echo "still present: $p"; bad=1; fi
done
return "$bad"
}
# --- instrument honesty ------------------------------------------------------
# The diff and the absence assert must be able to FAIL, or beats 3 and 4 prove
# nothing — so break each one against a scratch tree first, on every run
# (the test/drill.sh doctrine: mechanical, not a one-off claim in a PR).
SCR="$WORK/scr"; mkdir -p "$SCR/tree/bin"
echo content > "$SCR/tree/bin/rig"
ln -s bin/rig "$SCR/tree/link"
tree_state "$SCR/tree" > "$SCR/cap"
check "honesty: an untouched tree reads as zero drift" 0 "" \
diff_state "$SCR/cap" "$SCR/tree"
echo drift >> "$SCR/tree/bin/rig"
check "honesty: a mutated file is drift, named" 1 "bin/rig" \
diff_state "$SCR/cap" "$SCR/tree"
tree_state "$SCR/tree" > "$SCR/cap"
ln -sfn ../elsewhere "$SCR/tree/link"
check "honesty: a retargeted symlink is drift" 1 "elsewhere" \
diff_state "$SCR/cap" "$SCR/tree"
tree_state "$SCR/tree" > "$SCR/cap"
touch "$SCR/tree/leftover"
check "honesty: an ADDED file is drift (what a non-convergent installer leaves)" 1 "leftover" \
diff_state "$SCR/cap" "$SCR/tree"
# The beat-4 distinction, demonstrated: `test ! -e` PASSES on a dangling
# symlink (it follows the link), so on its own it would certify a broken
# uninstall clean — only `! -L` sees the corpse.
ln -s "$SCR/nowhere" "$SCR/dangling-rig"
check "honesty: test ! -e cannot see a dangling symlink (the lie)" 0 "" \
test ! -e "$SCR/dangling-rig"
check "honesty: the absence assert can (! -L is the catch)" 1 "still present" \
no_residue "$SCR/dangling-rig"
rm "$SCR/dangling-rig"
check "honesty: a really-gone path passes the absence assert" 0 "" \
no_residue "$SCR/dangling-rig"
# --- beat 1: install from THIS checkout --------------------------------------
# RIG_INSTALL_SOURCE is the supported local channel (its contract — dir,
# tarball, loud refusal, no silent download fallback — is test/release.sh's);
# in CI $ROOT is $GITHUB_WORKSPACE, so what lands is the code under review.
b1() {
PATH="$SNAPBIN:$PATH" SNAPSHOT_TARBALL="$WORK/snapshot.tar.gz" \
RIG_INSTALL_SOURCE="$ROOT" bash "$ROOT/install.sh"
}
check "beat 1: install.sh installs this checkout" 0 "done" b1
# --- beat 2: assert what landed ----------------------------------------------
check "beat 2: the tree landed in versions/$VER" 0 "" \
test -x "$DEST/versions/$VER/bin/rig"
check "beat 2: current points at versions/$VER" 0 "versions/$VER" \
readlink "$DEST/current"
check "beat 2: the PATH symlink rides the chain" 0 "$DEST/current/bin/rig" \
readlink "$BINDIR/rig"
check "beat 2: ...and resolves into versions/ (cast's assert)" 0 "/versions/$VER/bin/rig" \
readlink -f "$BINDIR/rig"
check "beat 2: rig --version answers through the whole chain" 0 "rig $VER" \
"$BINDIR/rig" --version
check "beat 2: INSTALLED_FROM names the local source" 0 "local:$ROOT" \
cat "$DEST/versions/$VER/INSTALLED_FROM"
TPL_PIN="$(sed -n 's/^RIG_TEMPLATES_PIN=//p' "$ROOT/commands/lib/templates.sh")"
check "beat 2: pinned registry snapshot landed in the version tree" 0 "" \
test -f "$DEST/versions/$VER/templates@$TPL_PIN/test-box/template.env"
# --- beat 3: the converging re-run -------------------------------------------
# "Ran twice without crashing" is the self-deception this beat exists to
# refuse (#106): the assert is an empty diff of captured state, plus current
# still pointing where it did.
tree_state "$DEST" > "$WORK/before"
CUR_BEFORE="$(readlink "$DEST/current")"
check "beat 3: the re-run is a no-op that says so" 0 "already installed" b1
check "beat 3: ...and changed NOTHING — the diff is the verdict" 0 "" \
diff_state "$WORK/before" "$DEST"
check "beat 3: current did not move" 0 "" \
test "$(readlink "$DEST/current")" = "$CUR_BEFORE"
# --- beat 4: uninstall --all, ending in the absence assert -------------------
check "beat 4: uninstall --all removes the whole install" 0 "uninstalled" \
"$BINDIR/rig" uninstall --all --force
check "beat 4: zero residue at the install root" 0 "" no_residue "$DEST"
check "beat 4: zero residue on PATH — not even a dangling symlink" 0 "" \
no_residue "$BINDIR/rig"
# The doctrine spelled out as its two distinct asserts (#106): -e for
# presence, -L for the dangling link -e cannot see.
check "beat 4: test ! -e on the PATH entry" 0 "" test ! -e "$BINDIR/rig"
check "beat 4: test ! -L on the PATH entry" 0 "" test ! -L "$BINDIR/rig"
check "beat 4: test ! -e on the install root" 0 "" test ! -e "$DEST"
check "beat 4: test ! -L on the install root" 0 "" test ! -L "$DEST"
echo "---"
echo "$PASS passed, $FAIL failed"
[ "$FAIL" -eq 0 ]

416
test/labels-reconcile.sh Normal file
View file

@ -0,0 +1,416 @@
#!/usr/bin/env bash
set -euo pipefail
# Fixture tests for the labels-reconcile state machine: a comment is a
# non-verdict whatever its body says (the AUTHOR escalates by requesting the
# human), a stale approval does not promote unreviewed code, and an explicit
# human request outranks everything.
# Dependency-free beyond jq; no network, no daemon — pure decide_state.
cd "$(dirname "$0")/.."
# shellcheck source=.github/scripts/labels-reconcile.sh
. .github/scripts/labels-reconcile.sh
# The DRAFT/HEAD_SHA/REQUESTED/REVIEWS_JSON assignments below are the state
# machine's inputs, consumed inside the sourced decide_state — not unused.
# shellcheck disable=SC2034
BOT1="${BOTS[0]}" BOT2="${BOTS[1]}" BOT3="${BOTS[2]}"
pass=0 fail=0
expect() { # $1 = description, $2 = want, $3 = got
if [ "$2" = "$3" ]; then
pass=$((pass + 1))
else
fail=$((fail + 1))
printf 'FAIL: %s — want %s, got %s\n' "$1" "$2" "$3"
fi
}
rev() { # $1=login $2=state $3=commit $4=body $5=submitted_at → one review object
jq -n --arg u "$1" --arg s "$2" --arg c "$3" --arg b "$4" --arg t "$5" \
'{user: {login: $u}, state: $s, commit_id: $c, body: $b, submitted_at: $t}'
}
reviews() { jq -s '.' <<<"$*"; } # collect review objects into an array
# -- drafts are building, whoever is requested --------------------------------
DRAFT=true HEAD_SHA=head1 REQUESTED="" REVIEWS_JSON='[]'
expect "draft PR is building" state:building "$(decide_state)"
# -- fresh ready PR with bots requested ---------------------------------------
DRAFT=false REQUESTED="$BOT1
$BOT2
$BOT3" REVIEWS_JSON='[]'
expect "requested bots mean bots-reviewing" state:bots-reviewing "$(decide_state)"
# -- a bot that never reviewed keeps the round open ---------------------------
# With a live request that is the bots' ball; with NO request outstanding it
# is the agent's, because nothing is coming until somebody asks.
REQUESTED="$BOT3" REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" APPROVED head1 "" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)")"
expect "a missing bot WITH a live request is bots-reviewing" state:bots-reviewing "$(decide_state)"
REQUESTED=""
expect "...but with nobody asked it is the agent's ball" state:addressing "$(decide_state)"
expect "...and the blocker names the stall" blocker:unrequested "$(blockers)"
# -- a comment is a non-verdict, agreement body or not: the author escalates --
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" COMMENTED head1 "✅ **Reviewed — I agree with everything.**" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
expect "comment-only agreement still parks on the author" state:addressing "$(decide_state)"
# ...and the author's escalation — requesting the human — flips it
REQUESTED="$HUMAN"
expect "author escalation flips to needs-human" state:needs-human "$(decide_state)"
REQUESTED=""
# -- three formal approvals need no author judgment ---------------------------
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" APPROVED head1 "" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
expect "three formal approvals reach needs-human" state:needs-human "$(decide_state)"
# -- a comment WITHOUT a verdict parks the PR on the agent --------------------
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" COMMENTED head1 "🔧 Reviewed — I agree with most; feedback below." t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
expect "comment without verdict is addressing" state:addressing "$(decide_state)"
# -- changes requested blocks, at any head ------------------------------------
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" CHANGES_REQUESTED old1 "blockers below" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
expect "changes-requested blocks even from an old head" state:addressing "$(decide_state)"
# -- a stale approval must not promote unreviewed code ------------------------
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" APPROVED old1 "" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
expect "stale approval is addressing (agent owes re-request)" state:addressing "$(decide_state)"
# -- a re-requested bot reopens the round even with an old approval on file ---
REQUESTED="$BOT1"
expect "re-requested bot means bots-reviewing" state:bots-reviewing "$(decide_state)"
REQUESTED=""
# -- only the LATEST review per bot counts ------------------------------------
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" CHANGES_REQUESTED head1 "blockers" t1)" \
"$(rev "$BOT1" APPROVED head1 "" t2)" \
"$(rev "$BOT2" APPROVED head1 "" t3)" \
"$(rev "$BOT3" APPROVED head1 "" t4)")"
expect "later approval supersedes earlier block" state:needs-human "$(decide_state)"
# -- an explicit human request outranks the bot rounds ------------------------
REQUESTED="$HUMAN" REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" COMMENTED head1 "feedback, no verdict" t1)")"
expect "human requested outranks bots" state:needs-human "$(decide_state)"
REQUESTED=""
# -- human CHANGES_REQUESTED puts the ball back on the agent ------------------
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" APPROVED head1 "" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)" \
"$(rev "$HUMAN" CHANGES_REQUESTED head1 "not yet" t4)")"
expect "human block with bots approving is addressing" state:addressing "$(decide_state)"
# ...and re-requesting the human hands it back to them
REQUESTED="$HUMAN"
expect "re-requested human is needs-human again" state:needs-human "$(decide_state)"
REQUESTED=""
# -- an old human comment must not wedge the handoff (codex, #85 round 3) -----
REVIEWS_JSON="$(reviews \
"$(rev "$HUMAN" COMMENTED old1 "early thoughts" t0)" \
"$(rev "$BOT1" APPROVED head1 "" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
expect "old human comment + three approvals is needs-human" state:needs-human "$(decide_state)"
expect "old human comment still needs a fresh request" needed "$(human_request_needed && echo needed || echo not-needed)"
# ...a stale human APPROVAL likewise needs a re-request for the new head
REVIEWS_JSON="$(reviews \
"$(rev "$HUMAN" APPROVED old1 "" t0)" \
"$(rev "$BOT1" APPROVED head1 "" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
expect "stale human approval needs a fresh request" needed "$(human_request_needed && echo needed || echo not-needed)"
# ...a HEAD-CURRENT human approval needs nothing more
REVIEWS_JSON="$(reviews \
"$(rev "$HUMAN" APPROVED head1 "" t0)" \
"$(rev "$BOT1" APPROVED head1 "" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
expect "head-current human approval needs no request" not-needed "$(human_request_needed && echo needed || echo not-needed)"
# ...and a live request suppresses re-requesting
REQUESTED="$HUMAN"
expect "live human request suppresses re-request" not-needed "$(human_request_needed && echo needed || echo not-needed)"
REQUESTED=""
# ---------------------------------------------------------------------------
# #136: state:needs-human must mean "a human could merge this RIGHT NOW".
# Both cases below were observed live in this repo on 2026-07-20, and both
# showed state:needs-human while being unmergeable in different ways.
# ---------------------------------------------------------------------------
ALL_APPROVE="$(reviews \
"$(rev "$BOT1" APPROVED head1 "" t1)" \
"$(rev "$BOT2" APPROVED head1 "" t2)" \
"$(rev "$BOT3" APPROVED head1 "" t3)")"
# -- flavour 1: not mergeable. The merge button is disabled, yet the board
# said "your turn" on #119/#120/#127 for hours. The branch fact now rides
# the blocker axis; the state says whose ball it is, which is the agent's.
DRAFT=false HEAD_SHA=head1 REQUESTED="" REVIEWS_JSON="$ALL_APPROVE" MERGEABLE=CONFLICTING CHECKS=SUCCESS
expect "a CONFLICTING PR is the agent's, not the human's" state:addressing "$(decide_state)"
expect "...and says WHY on the blocker axis" blocker:conflict "$(blockers)"
REQUESTED="$HUMAN"
expect "...even with the human explicitly requested" state:addressing "$(decide_state)"
# -- red CI is the same claim, but NOT the same work: a rebase does not fix a
# failing test. Collapsing both into one needs-rebase label told the agent
# to do the wrong thing, which is why the axis split exists.
REQUESTED="" MERGEABLE=MERGEABLE CHECKS=FAILURE
expect "a red PR is the agent's" state:addressing "$(decide_state)"
expect "...and is distinguishable from a conflict" blocker:ci-red "$(blockers)"
REQUESTED="$HUMAN"
expect "...and a human request does not override red CI" state:addressing "$(decide_state)"
# -- both at once. The single-axis design could not say this at all: one label
# had to win, and the loser silently vanished off the board.
REQUESTED="" MERGEABLE=CONFLICTING CHECKS=FAILURE
expect "a conflicted AND red PR reports both blockers" "blocker:conflict
blocker:ci-red" "$(blockers)"
expect "...and is still just the agent's ball" state:addressing "$(decide_state)"
# -- UNKNOWN is NOT unmergeable. GitHub reports it for ~a minute after every
# merge while it recomputes; treating it as broken would flap every open PR
# on each merge — worse than the bug being fixed.
REQUESTED="" MERGEABLE=UNKNOWN CHECKS=PENDING
expect "UNKNOWN mergeability blocks nothing" state:needs-human "$(decide_state)"
expect "...and raises no blocker" "" "$(blockers)"
# -- blocker:unrequested — the stalled round. Nobody owes an answer because
# nobody was ever asked, yet the board read "waiting on the bots" until
# `stale` noticed 48h later.
MERGEABLE=MERGEABLE CHECKS=SUCCESS REQUESTED="" REVIEWS_JSON='[]'
expect "ready, nobody asked, nothing reviewed raises unrequested" blocker:unrequested "$(blockers)"
# ...the partial case is equally stalled: one verdict in, nobody asked for the rest
REVIEWS_JSON="$(reviews "$(rev "$BOT1" APPROVED head1 "" t1)")"
expect "one bot in, none requested is still unrequested" blocker:unrequested "$(blockers)"
# ...a STALE round with nobody asked is the same debt, and arguably worse: the
# page carries approvals that no longer describe the tree. Guarding on
# MISSING alone let this one through with no blocker at all.
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" APPROVED oldhead "" t1)" \
"$(rev "$BOT2" APPROVED oldhead "" t2)" \
"$(rev "$BOT3" APPROVED oldhead "" t3)")"
expect "a stale round with nobody asked is unrequested too" blocker:unrequested "$(blockers)"
expect "...and is still the agent's ball" state:addressing "$(decide_state)"
# ...but a live request means an answer IS coming
REVIEWS_JSON="$(reviews "$(rev "$BOT1" APPROVED head1 "" t1)")"
REQUESTED="$BOT2"
expect "a live bot request is not a stalled round" "" "$(blockers)"
# ...and a draft is exempt: the bots ignore drafts by design
DRAFT=true REQUESTED="" REVIEWS_JSON='[]'
expect "a draft with nobody asked is not stalled" "" "$(blockers)"
# ...as is an explicit human request — claiming a PR early is deliberate
DRAFT=false REQUESTED="$HUMAN"
expect "an early human claim is not a stalled round" "" "$(blockers)"
REQUESTED="" REVIEWS_JSON="$ALL_APPROVE" MERGEABLE=MERGEABLE CHECKS=SUCCESS
# -- flavour 2 (the dangerous one): mergeable, green, human requested, and
# NOBODY has reviewed this head. Observed on #119 after a rebase: every
# signal read "merge me" and nothing on the page contradicted it.
MERGEABLE=MERGEABLE CHECKS=SUCCESS REQUESTED="$HUMAN"
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" APPROVED oldhead "" t1)" \
"$(rev "$BOT2" APPROVED oldhead "" t2)" \
"$(rev "$BOT3" APPROVED oldhead "" t3)")"
expect "stale approvals outrank the human request (nobody reviewed this tree)" state:addressing "$(decide_state)"
# -- ...and a round that is BOTH unfinished and staled is still the agent's.
# Deciding inside the bot loop made this depend on BOTS order: the MISSING
# returned before any later bot's STALE was read, so the mixed round came
# out needs-human with nothing bound to the head. Pinned at both ends of
# the array, because the whole failure was one of ordering.
MERGEABLE=MERGEABLE CHECKS=SUCCESS REQUESTED="$HUMAN"
REVIEWS_JSON="$(reviews \
"$(rev "$BOT1" APPROVED oldhead "" t1)" \
"$(rev "$BOT2" APPROVED oldhead "" t2)")"
expect "stale approvals + a bot yet to review is addressing, not needs-human" \
state:addressing "$(decide_state)"
REVIEWS_JSON="$(reviews "$(rev "$BOT3" APPROVED oldhead "" t3)")"
expect "...and the same when the stale verdict is the LAST bot in BOTS" \
state:addressing "$(decide_state)"
# -- but an UNFINISHED round still yields to an explicit human request: a
# maintainer pulling a PR to themselves early is deliberate, and was the
# original precedence. MISSING differs from STALE — nobody has reviewed
# YET, versus everyone reviewed something else.
REVIEWS_JSON="$(reviews "$(rev "$BOT1" APPROVED head1 "" t1)")"
expect "an unfinished round still yields to an explicit human request" state:needs-human "$(decide_state)"
REQUESTED=""
expect "...and without that request the agent owes the ask" state:addressing "$(decide_state)"
# ---------------------------------------------------------------------------
# checks_state: the rollup classifier. It lived inline in main() for the first
# round of this PR, which is why nothing here caught it calling ERROR,
# CANCELLED and STALE green. Extracted so the enum can be pinned down.
# ---------------------------------------------------------------------------
rollup() { jq -n --argjson c "$1" '{statusCheckRollup: $c}'; }
run_() { jq -n --arg n "$1" --arg o "$2" --arg t "${3:-2026-07-20T15:00:00Z}" \
'{__typename:"CheckRun", workflowName:"ci", name:$n, conclusion:$o, completedAt:$t}'; }
ctx_() { jq -n --arg n "$1" --arg s "$2" --arg t "${3:-2026-07-20T15:00:00Z}" \
'{__typename:"StatusContext", context:$n, state:$s, createdAt:$t}'; }
expect "no checks at all is NONE" NONE "$(rollup '[]' | checks_state)"
# A failed fetch leaves no rollup KEY; a PR with no checks leaves an empty
# ARRAY. Collapsing the two let an API hiccup read as "nothing is failing" —
# the same unknown-certified-as-green shape as #136, in the one place that
# fix did not look. The caller skips an UNREADABLE PR rather than relabelling.
expect "a failed read is UNREADABLE, not NONE" UNREADABLE "$(echo '{}' | checks_state)"
expect "...and a real empty rollup is still NONE" NONE \
"$(echo '{"mergeable":"MERGEABLE","statusCheckRollup":[]}' | checks_state)"
expect "all green is SUCCESS" SUCCESS \
"$(rollup "[$(run_ a SUCCESS),$(run_ b SUCCESS)]" | checks_state)"
expect "a queued run is PENDING" PENDING \
"$(rollup "[$(run_ a SUCCESS),$(run_ b QUEUED)]" | checks_state)"
expect "a plain failure is FAILURE" FAILURE \
"$(rollup "[$(run_ a SUCCESS),$(run_ b FAILURE)]" | checks_state)"
# -- the round-1 gap: outcomes that are neither success nor pending, and that
# leave a required check unsatisfied. All three reached the old `else`.
expect "a commit status ERROR blocks" FAILURE \
"$(rollup "[$(run_ a SUCCESS),$(ctx_ lint ERROR)]" | checks_state)"
expect "a CANCELLED run blocks" FAILURE \
"$(rollup "[$(run_ a SUCCESS),$(run_ b CANCELLED)]" | checks_state)"
expect "a STALE run blocks" FAILURE \
"$(rollup "[$(run_ a SUCCESS),$(run_ b STALE)]" | checks_state)"
expect "an outcome the enum does not know blocks, it does not pass" FAILURE \
"$(rollup "[$(run_ a SUCCESS),$(run_ b SOME_FUTURE_STATE)]" | checks_state)"
# -- NEUTRAL and SKIPPED satisfy branch protection; path-filtered jobs skip
# constantly, and calling that red would park every PR on the agent.
expect "NEUTRAL and SKIPPED are not failures" SUCCESS \
"$(rollup "[$(run_ a SUCCESS),$(run_ b NEUTRAL),$(run_ c SKIPPED)]" | checks_state)"
# -- latest-wins. The rollup keeps superseded runs, so this PR's own tip
# carried a CANCELLED `scope` beside the SUCCESS `scope` that replaced it.
# Without collapsing, making CANCELLED block would strand it forever.
expect "a re-run supersedes the cancelled original" SUCCESS \
"$(rollup "[$(run_ scope CANCELLED 2026-07-20T15:19:39Z),\
$(run_ scope SUCCESS 2026-07-20T15:19:45Z)]" | checks_state)"
expect "...and the reverse order is not a re-run passing, it is one failing" FAILURE \
"$(rollup "[$(run_ scope SUCCESS 2026-07-20T15:19:39Z),\
$(run_ scope CANCELLED 2026-07-20T15:19:45Z)]" | checks_state)"
# same job name in a different workflow is a different context, not a re-run
expect "same name in another workflow does not supersede" FAILURE \
"$(rollup "[$(jq -n '{__typename:"CheckRun",workflowName:"labels",name:"scope",conclusion:"FAILURE",completedAt:"2026-07-20T15:00:00Z"}'),\
$(run_ scope SUCCESS 2026-07-20T15:19:45Z)]" | checks_state)"
# -- a run still IN FLIGHT. `run_()` cannot express this: it always carries a
# real completedAt, which is exactly why the supersede rule shipped dating
# runs by completion and nothing caught it. Both spellings of "no
# completion" are pinned, because `gh` emits the zero sentinel (a string,
# which `//` does not fall through) while the API emits null.
inflight_() { jq -n --arg n "$1" --arg t "$2" --arg c "${3:-0001-01-01T00:00:00Z}" \
'{__typename:"CheckRun", workflowName:"ci", name:$n, status:"IN_PROGRESS",
conclusion:"", startedAt:$t, completedAt:(if $c == "null" then null else $c end)}'; }
expect "a re-run in flight beats the success it superseded (zero sentinel)" PENDING \
"$(rollup "[$(run_ build SUCCESS 2026-07-20T15:00:00Z),\
$(inflight_ build 2026-07-20T15:10:00Z)]" | checks_state)"
expect "...and the same when the absent completion is null" PENDING \
"$(rollup "[$(run_ build SUCCESS 2026-07-20T15:00:00Z),\
$(inflight_ build 2026-07-20T15:10:00Z null)]" | checks_state)"
expect "a replacement in flight for a CANCELLED run is pending, not failed" PENDING \
"$(rollup "[$(run_ build CANCELLED 2026-07-20T15:00:00Z),\
$(inflight_ build 2026-07-20T15:10:00Z)]" | checks_state)"
# an entry carrying no usable timestamp is treated as newest, not oldest —
# ambiguity resolves toward "not settled" rather than toward a stale success.
# Guarded by the sort tiebreak rather than the dating expression: reverting
# only `at:` leaves this passing, so the two changes are separately pinned.
expect "an undateable in-flight run is not discarded for a stale success" PENDING \
"$(rollup "[$(run_ build SUCCESS 2026-07-20T15:00:00Z),\
$(jq -n '{__typename:"CheckRun",workflowName:"ci",name:"build",conclusion:"",startedAt:null,completedAt:null}')]" \
| checks_state)"
# ...and the reverse direction, which stops "in flight sorts last" being
# widened into "in flight always wins": a run that FINISHED after an earlier
# in-flight entry is the newer word, and the context is settled.
expect "a finished re-run supersedes an earlier in-flight run" SUCCESS \
"$(rollup "[$(inflight_ build 2026-07-20T15:19:00Z),\
$(run_ build SUCCESS 2026-07-20T15:19:45Z)]" | checks_state)"
# -- the wind-down window. A predecessor cancelled by the concurrency group
# does not stop the instant its replacement starts, so its completion
# routinely lands AFTER the successor's start — on box's aa5a6ba the
# replacement started 15:19:38 and the run it cancelled finished 15:19:51.
# Dating by "newest stamp of any kind" compares the dead run's completion
# against the live run's start, which is not an ordering on runs, and the
# predecessor wins. Every fixture above spaces completion before start, so
# none of them can see it. run_() cannot express the overlap either — it
# carries no startedAt — hence the explicit payloads.
overlap_() { jq -n --arg n "$1" --arg o "$2" --arg s "$3" --arg c "$4" \
'{__typename:"CheckRun", workflowName:"ci", name:$n, conclusion:$o,
startedAt:$s, completedAt:$c}'; }
expect "a predecessor finishing after its replacement started is still older (CANCELLED)" PENDING \
"$(rollup "[$(overlap_ scope CANCELLED 2026-07-20T15:19:00Z 2026-07-20T15:19:51Z),\
$(inflight_ scope 2026-07-20T15:19:38Z)]" | checks_state)"
expect "...and the same when it finished green — mid-flight is not mergeable" PENDING \
"$(rollup "[$(overlap_ build SUCCESS 2026-07-20T15:19:00Z 2026-07-20T15:19:51Z),\
$(inflight_ build 2026-07-20T15:19:38Z)]" | checks_state)"
# -- the classifier feeds the state machine: a cancelled required check must
# take the PR off the human's plate, which is the whole point of #136.
DRAFT=false HEAD_SHA=head1 REQUESTED="$HUMAN" REVIEWS_JSON="$ALL_APPROVE" MERGEABLE=MERGEABLE
CHECKS="$(rollup "[$(run_ a SUCCESS),$(run_ b CANCELLED)]" | checks_state)"
expect "a cancelled check reaches decide_state as the agent's ball" state:addressing "$(decide_state)"
expect "...via blocker:ci-red, not a conflict" blocker:ci-red "$(blockers)"
# -- the happy path survives all of the above.
REVIEWS_JSON="$ALL_APPROVE" MERGEABLE=MERGEABLE CHECKS=SUCCESS REQUESTED=""
expect "mergeable + green + three head-current approvals is needs-human" state:needs-human "$(decide_state)"
# -- and a draft outranks everything, including a conflict.
DRAFT=true MERGEABLE=CONFLICTING
expect "a draft is building even when conflicted" state:building "$(decide_state)"
DRAFT=false MERGEABLE=MERGEABLE CHECKS=SUCCESS REQUESTED="" REVIEWS_JSON='[]'
# ---------------------------------------------------------------------------
# reconcile_pr's cold-start path. Everything above tests pure functions, which
# is exactly why a per-PR `return` in the label pre-flight got through review:
# the fixtures could not reach it. A missing state:* label must skip the label
# EDIT only — merge-next clearing and the stale sweep are independent of the
# taxonomy, and stranding them reintroduced the false-invitation bug (a
# `merge-next` claim surviving on a PR the board had moved to the agent).
# ---------------------------------------------------------------------------
reconcile_probe() { # $1 = REPO_LABELS content → the log lines reconcile_pr emits
(
REPO_LABELS="$1" REPO=owner/repo NOW="$(date +%s)"
LABELS="merge-next" # the PR carries a queue claim
DRAFT=false HEAD_SHA=head1 REQUESTED="" REVIEWS_JSON='[]'
MERGEABLE=MERGEABLE CHECKS=SUCCESS
PR_JSON='{"created_at":"2020-01-01T00:00:00Z"}'
run() { :; } # swallow mutations
gh() { :; } # no network
reconcile_pr 777 2>&1
)
}
cold="$(reconcile_probe "merge-next")" # state:* labels absent entirely
expect "a cold-start repo still clears merge-next" \
yes "$(grep -q 'cleared merge-next' <<<"$cold" && echo yes || echo no)"
expect "...and still runs the stale sweep" \
yes "$(grep -q 'stale (' <<<"$cold" && echo yes || echo no)"
expect "...while warning that the state label is missing" \
yes "$(grep -q "state label 'state:addressing' does not exist" <<<"$cold" && echo yes || echo no)"
warm="$(reconcile_probe "$(printf 'state:addressing\nmerge-next\nstale\nblocker:unrequested')")"
expect "a bootstrapped repo converges the state as well" \
yes "$(grep -q 'state -> state:addressing' <<<"$warm" && echo yes || echo no)"
printf 'labels-reconcile tests: %d passed, %d failed\n' "$pass" "$fail"
[ "$fail" -eq 0 ]

893
test/release.sh
View file

@ -1,16 +1,17 @@
#!/usr/bin/env bash #!/usr/bin/env bash
# Rig's own half of the release surface (#32; trimmed in ceremony#13's # The release flow's testable half (#32): changelog extraction, latest-tag
# conversion): latest-tag resolution and the installer's three channels. # resolution, and the installer's three channels. Dependency-free and
# The machinery halves — changelog extraction, the arming rule,
# monotonicity, the drill gate, the workflow-shape pins — moved to
# heavy-duty/ceremony, which tests them in its own test/; what stays is
# everything that drives rig's install.sh and bin/. Dependency-free and
# NETWORK-FREE — wherever the code under test would call curl, the curl on # NETWORK-FREE — wherever the code under test would call curl, the curl on
# PATH is a stub this harness wrote. Run: bash test/release.sh # PATH is a stub this harness wrote. Run: bash test/release.sh
# Deliberately no `set -e` — the harness asserts on failing commands. # Deliberately no `set -e` — the harness asserts on failing commands.
set -u set -u
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
cd "$ROOT" || exit 1 cd "$ROOT" || exit 1
# The extraction the workflow runs is the extraction under test — one
# function, sourced by release.yml and by this harness (repo precedent:
# test/labels-reconcile.sh sourcing the reconciler's decide_state).
# shellcheck source=.github/scripts/release-lib.sh
. "$ROOT/.github/scripts/release-lib.sh"
PASS=0 FAIL=0 PASS=0 FAIL=0
# check <desc> <want_exit> <want_substr> <cmd...> # check <desc> <want_exit> <want_substr> <cmd...>
@ -36,20 +37,753 @@ check() {
WORK="$(mktemp -d)" WORK="$(mktemp -d)"
FAKEHOME="$WORK/home"; mkdir -p "$FAKEHOME" FAKEHOME="$WORK/home"; mkdir -p "$FAKEHOME"
# --- changelog_section: the release body, extracted --------------------------
# A fixture changelog with the three heading shapes the flow produces: the
# bare '## Unreleased', stamped '## X.Y.Z — date' releases, and a last
# section that runs to EOF.
FIXCH="$WORK/CHANGELOG.fixture.md"
cat > "$FIXCH" <<'MD'
# Changelog
History before 0.1.0 lives in git.
## Unreleased
- an unreleased entry
## 0.2.0 — 2026-07-18
### Added
- **the newer entry** (#42) — prose.
### Fixed
- a fix in 0.2.0
## 0.1.0 — 2026-07-01
- the first entry
MD
sect_has() { changelog_section "$1" "$2" | grep -qF -e "$3"; }
check "changelog: extracts the asked-for section" 0 "the newer entry" \
changelog_section "$FIXCH" 0.2.0
check "changelog: the whole section, subheadings included" 0 "a fix in 0.2.0" \
changelog_section "$FIXCH" 0.2.0
check "changelog: stops at the next release heading" 1 "" \
sect_has "$FIXCH" 0.2.0 "the first entry"
check "changelog: never leaks the preceding section" 1 "" \
sect_has "$FIXCH" 0.2.0 "an unreleased entry"
check "changelog: the heading itself is not the body" 1 "" \
sect_has "$FIXCH" 0.2.0 "## 0.2.0"
first_line() { changelog_section "$1" "$2" | head -n1; }
check "changelog: leading blank lines are dropped" 0 "### Added" \
first_line "$FIXCH" 0.2.0
check "changelog: the bare Unreleased heading matches too" 0 "an unreleased entry" \
changelog_section "$FIXCH" Unreleased
check "changelog: the last section runs to EOF" 0 "the first entry" \
changelog_section "$FIXCH" 0.1.0
absent() { [ -z "$(changelog_section "$1" "$2")" ]; }
check "changelog: an unknown version yields NOTHING (the refusal signal)" 0 "" \
absent "$FIXCH" 3.3.3
check "changelog: a date-stamped heading never matches by date" 0 "" \
absent "$FIXCH" 2026-07-18
# ...and the SHIPPED changelog fits the extractor. The real file has two
# legitimate states, and the old check knew only one (#44, found the day the
# first release PR turned CI red): BETWEEN releases there is an `## Unreleased`
# section feature PRs append to; on a `release: X.Y.Z` tree — and on main
# right after it, until the next feature PR — that section IS the stamped
# `## X.Y.Z — date`. Demanding the literal heading (or, worse, an issue
# number inside it) made the release PR of the ceremony unshippable by
# construction. What the guard is FOR is format drift: whatever the top
# section is called, the exact function release.yml runs must extract it
# non-empty.
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "CHANGELOG.md: has a top section (Unreleased or a stamped release)" 0 "" \
bash -c '[ -n "$(grep -m1 "^## " "$1")" ]' _ "$ROOT/CHANGELOG.md"
# --- the arming rule: is main's changelog ready for a late merge? ------------
# #66: stamping the Unreleased heading DISARMS the file. A PR authored before
# a release and merged after it wrote its entry under `## Unreleased`; once
# that heading has become `## X.Y.Z — date`, git lands the entry under the
# release that already shipped — cleanly, no conflict, nothing for the author
# to notice. It happened here: #60's #58 entry landed inside `## 0.1.0` at
# 67386b4, repaired two minutes later by 0ff520c.
#
# The check above cannot see this, and #44 is why: demanding a literal
# `## Unreleased` is FALSE BY CONSTRUCTION on the tree the ceremony's own PR
# produces, which made the release PR unshippable. That relaxation must not
# be undone.
#
# What distinguishes the two states the old guard collapsed is VERSION.
# A stamped top section is legal exactly when VERSION is bare — the ceremony
# PR, and main until the -dev bump lands. The moment VERSION carries -dev,
# main is a place feature PRs merge into, and the top section MUST be
# `## Unreleased` or the next late merge is misfiled.
#
# Note the asymmetry, which is deliberate: on a BARE version the top heading
# is not constrained at all. The ceremony re-arms in the same PR
# (CONTRIBUTING step 1), so its tree legitimately carries an EMPTY
# `## Unreleased` above the section it just stamped — and an empty top
# section is exactly what the old non-empty assert would have rejected.
# What must extract non-empty on a bare VERSION is the section that SHIPS,
# which is the same assert release.yml makes before it publishes.
#
# changelog_armed <version> <changelog-file> — 0 armed, 1 disarmed.
changelog_armed() {
local ver="$1" file="$2" top
top="$(grep -m1 '^## ' "$file")"
[ -n "$top" ] || return 1
case "$ver" in
*-dev) [ "$top" = "## Unreleased" ] ;;
*) [ -n "$(changelog_section "$file" "$ver")" ] ;;
esac
}
# The guard itself, against the real tree.
check "CHANGELOG.md: armed for the VERSION it carries (#66)" 0 "" \
changelog_armed "$(cat "$ROOT/VERSION")" "$ROOT/CHANGELOG.md"
# ...and the rule proven against trees built for the purpose, because a guard
# that is only ever run against a passing tree has not been shown to fail.
# Each is a real VERSION + CHANGELOG.md pair the flow actually produces.
armtree() { # armtree <name> <version> <changelog-body...> -> prints the dir
local d="$WORK/arm-$1"; mkdir -p "$d"; printf '%s\n' "$2" > "$d/VERSION"
shift 2; printf '%s\n' "$@" > "$d/CHANGELOG.md"; printf '%s' "$d"
}
armed() { changelog_armed "$(cat "$1/VERSION")" "$1/CHANGELOG.md"; }
# The ceremony PR's own tree, re-armed per CONTRIBUTING step 1: VERSION bare,
# an empty Unreleased sitting above the section it just stamped. GREEN — this
# is the case #44 was about, and the empty section must not break it.
T="$(armtree ceremony 0.2.0 '# Changelog' '' '## Unreleased' '' '## 0.2.0 — 2026-07-19' '' '- **A shipped thing** (#1) — prose.')"
check "arming: the re-armed ceremony tree passes (#44 stays fixed)" 0 "" armed "$T"
# The same ceremony WITHOUT the re-arm — old-style, stamped straight over the
# heading. Also GREEN: VERSION is bare, so a stamped top is legal. The guard
# refuses to make the ceremony unshippable, which is the whole #44 lesson.
T="$(armtree ceremony-old 0.2.0 '# Changelog' '' '## 0.2.0 — 2026-07-19' '' '- **A shipped thing** (#1) — prose.')"
check "arming: an un-re-armed ceremony tree still passes (bare VERSION)" 0 "" armed "$T"
# main AFTER release.yml's -dev bump, with the changelog left disarmed. This
# is #66 exactly, and the state cast sat in at the time of writing. RED.
T="$(armtree disarmed 0.2.1-dev '# Changelog' '' '## 0.2.0 — 2026-07-19' '' '- **A shipped thing** (#1) — prose.')"
check "arming: a -dev main with a stamped top section FAILS (#66)" 1 "" armed "$T"
# The same main, re-armed. The Unreleased section is EMPTY — no feature PR has
# merged since the release — and that is a correct, expected state. GREEN.
T="$(armtree rearmed 0.2.1-dev '# Changelog' '' '## Unreleased' '' '## 0.2.0 — 2026-07-19' '' '- **A shipped thing** (#1) — prose.')"
check "arming: a -dev main with an EMPTY Unreleased passes (no entries yet)" 0 "" armed "$T"
# Steady state between releases: entries accumulating under Unreleased.
T="$(armtree steady 0.2.1-dev '# Changelog' '' '## Unreleased' '' '### Fixed' '' '- **A pending thing** (#2) — prose.' '' '## 0.2.0 — 2026-07-19' '' '- **A shipped thing** (#1) — prose.')"
check "arming: the normal between-releases tree passes" 0 "" armed "$T"
# A release PR that bumped VERSION but forgot to stamp: the version it claims
# to ship has no section, so release.yml would publish empty notes. RED here,
# one round earlier than the workflow's own refusal.
T="$(armtree unstamped 0.3.0 '# Changelog' '' '## Unreleased' '' '- **A pending thing** (#2) — prose.')"
check "arming: a bare VERSION whose section was never stamped FAILS" 1 "" armed "$T"
# And a file with no '## ' heading at all is disarmed, not silently fine.
T="$(armtree headless 0.2.1-dev '# Changelog' '' 'no sections here')"
check "arming: a changelog with no sections FAILS" 1 "" armed "$T"
# --- the monotonicity rule: was a SHIPPED heading deleted? -------------------
# #98. Arming asks about ONE heading — does the top section agree with
# VERSION? — so it is silent about the rest of the file. The failure it cannot
# see is an entry written under '## Unreleased' that REPLACES the heading
# below it instead of inserting above it: git merges the one-line edit
# cleanly, arming stays green (the top section is still right), and the
# shipped release loses its section entirely. "A heading disappeared" is not a
# property of a tree, it is a property of a DIFF — so unlike every check
# above, these cases need real git repos, which is why the guard is its own
# script rather than a function sourced here.
MONO="$ROOT/.github/scripts/changelog-monotonic.sh"
check "changelog-monotonic.sh: exists and is the guard under test" 0 "" test -f "$MONO"
# The stock changelog every case below starts from: an Unreleased section and
# two shipped releases, committed on branch 'base' — which plays origin/main.
# The caller then rewrites CHANGELOG.md on 'work' and commits.
MONO_BASE=('# Changelog' '' '## Unreleased' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.' '' '## 0.1.0 — 2026-07-01' '' \
'- **The first thing** (#0) — prose.')
monorepo() { # monorepo <name> -> prints the dir, left checked out on 'work'
local d="$WORK/mono-$1"; mkdir -p "$d"
git -C "$d" init -q -b base
git -C "$d" config user.email harness@example.invalid
git -C "$d" config user.name harness
printf '%s\n' "${MONO_BASE[@]}" > "$d/CHANGELOG.md"
git -C "$d" add CHANGELOG.md
git -C "$d" commit -qm 'base: two shipped releases'
git -C "$d" checkout -q -b work
printf '%s' "$d"
}
monowrite() { # monowrite <dir> <line...> — rewrite CHANGELOG.md and commit
local d="$1"; shift
printf '%s\n' "$@" > "$d/CHANGELOG.md"
git -C "$d" commit -qam 'work: edit the changelog'
}
mono() { # mono <dir> [VAR=val ...] — run the guard there, base ref 'base'
local d="$1"; shift
( cd "$d" && env "$@" bash "$MONO" base 2>&1 )
}
# An untouched branch with NO commit of its own: 'work' still points at the
# base commit, so the merge base IS HEAD and containment compared the file
# against itself. That is the vacuous path (#98), not a containment result —
# the green message therefore names uniqueness, the half that actually ran.
# A guard that prints nothing is indistinguishable from one that did nothing,
# but a guard that prints the WRONG half is worse: it is a false receipt.
T="$(monorepo clean)"
check "monotonic: an untouched branch passes" 0 "uniqueness on HEAD checked 2" mono "$T"
check "monotonic: ...saying containment was VACUOUS, not that it verified 2" 0 \
"containment vacuous" mono "$T"
# A negative, because the point is that the two wordings do NOT collapse: with
# the pull_request gate gone (#98) this is the shape of EVERY push to main, and
# "are still present" there would be a containment claim on the one event where
# deletion is undetectable by construction.
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "monotonic: ...and never claims the headings are still present" 1 "" \
bash -c 'cd "$1" && bash "$2" base | grep -q "are still present"' _ "$T" "$MONO"
# The same shape against a REAL base — an unrelated commit on 'work', the
# changelog untouched — which is what an untouched-changelog PR branch
# actually looks like. Here containment genuinely ran and held, so this is
# the case that pins the containment wording and its count. The two forms
# must not collapse into one another.
T="$(monorepo clean-realbase)"
printf '%s\n' '# rig' > "$T/README.md"
git -C "$T" add README.md
git -C "$T" commit -qm 'work: an unrelated commit, changelog untouched'
check "monotonic: an untouched changelog on a REAL base reports containment" 0 \
"all 2 release heading(s)" mono "$T"
check "monotonic: ...and says they are still present, the containment claim" 0 \
"are still present" mono "$T"
# The legitimate edit this guard must never object to: a new entry INSERTED
# above the shipped heading, which is left alone.
T="$(monorepo insert)"
monowrite "$T" '# Changelog' '' '## Unreleased' '' '### Fixed' '' \
'- **A pending thing** (#2) — prose.' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.' '' '## 0.1.0 — 2026-07-01' '' \
'- **The first thing** (#0) — prose.'
check "monotonic: an entry inserted ABOVE the shipped heading passes" 0 "" mono "$T"
# ...and the bug itself: the same entry typed OVER '## 0.2.0'. 0.2.0's body is
# now under '## Unreleased' and 0.2.0 has no section. RED, naming the version.
T="$(monorepo deleted)"
monowrite "$T" '# Changelog' '' '## Unreleased' '' '### Fixed' '' \
'- **A pending thing** (#2) — prose.' '' \
'- **A shipped thing** (#1) — prose.' '' '## 0.1.0 — 2026-07-01' '' \
'- **The first thing** (#0) — prose.'
check "monotonic: a DELETED shipped heading FAILS (#98)" 1 "DELETES release heading" mono "$T"
check "monotonic: ...and the failure names the version that vanished" 1 "## 0.2.0" mono "$T"
# Deleting the OLDEST release is the same defect, not a lesser one — the set
# is a set, position in the file buys no leniency.
T="$(monorepo deleted-old)"
monowrite "$T" '# Changelog' '' '## Unreleased' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.'
check "monotonic: deleting an OLDER release heading fails too" 1 "## 0.1.0" mono "$T"
# The duplicate half. Containment cannot catch this: the second copy is
# head-side SURPLUS and `comm -23` (base minus head) is blind to extras on the
# head side, so uniqueness-on-HEAD is a separate assert. rig's symptom is not
# box's — changelog_section() has `if (found) exit`, so it stops at the second
# copy and TRUNCATES rather than absorbing.
T="$(monorepo dupe)"
monowrite "$T" '# Changelog' '' '## Unreleased' '' '## 0.2.0 — 2026-07-19' '' \
'- **A pending thing** (#2) — prose.' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.' '' '## 0.1.0 — 2026-07-01' '' \
'- **The first thing** (#0) — prose.'
check "monotonic: a DUPLICATED version heading FAILS" 1 "DUPLICATE release heading" mono "$T"
check "monotonic: ...and the failure names the repeated version" 1 "## 0.2.0" mono "$T"
# ...and that the duplicate really does truncate, so the assert above is
# guarding a live defect rather than a stylistic preference: extraction stops
# at the second copy, dropping the body that sits under it.
check "monotonic: the duplicate TRUNCATES extraction (rig's symptom, not box's)" 0 \
"A pending thing" changelog_section "$T/CHANGELOG.md" 0.2.0
check "monotonic: ...the real body under the second copy is dropped" 1 "" \
sect_has "$T/CHANGELOG.md" 0.2.0 "A shipped thing"
# '## Unreleased' is deliberately OUTSIDE the guarded set: it fails the
# version shape, so the ceremony stamping it away — the one edit that legally
# removes a top heading — is invisible here. This is the case that would make
# every release PR unshippable if the set were "all '## ' headings".
T="$(monorepo stamp)"
monowrite "$T" '# Changelog' '' '## 0.3.0 — 2026-07-20' '' \
'- **A pending thing** (#2) — prose.' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.' '' '## 0.1.0 — 2026-07-01' '' \
'- **The first thing** (#0) — prose.'
check "monotonic: stamping '## Unreleased' into a release passes (not guarded)" 0 "" mono "$T"
# ...and the ceremony's re-arm — a fresh empty Unreleased above the stamp —
# is equally fine, which is CONTRIBUTING step 1's tree.
T="$(monorepo stamp-rearmed)"
monowrite "$T" '# Changelog' '' '## Unreleased' '' '## 0.3.0 — 2026-07-20' '' \
'- **A pending thing** (#2) — prose.' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.' '' '## 0.1.0 — 2026-07-01' '' \
'- **The first thing** (#0) — prose.'
check "monotonic: the re-armed ceremony tree passes too" 0 "" mono "$T"
# The skip path, both halves. A base ref that does not resolve is a sensible
# local degradation — and a silent one, which is the failure shape this family
# of checks exists to refuse. So STRICT flips exactly that case red.
mono_noref() { local d="$1"; shift; ( cd "$d" && env "$@" bash "$MONO" no/such/ref 2>&1 ); }
T="$(monorepo noref)"
check "monotonic: an unresolvable base ref SKIPS containment locally" 0 "containment SKIPPED" mono_noref "$T"
check "monotonic: ...and the skip says uniqueness already ran, not that nothing did" 0 \
"already ran and passed" mono_noref "$T"
check "monotonic: ...but is a FAILURE under STRICT=1 (what CI sets)" 1 "STRICT=1" \
mono_noref "$T" CHANGELOG_MONOTONIC_STRICT=1
check "monotonic: ...and the STRICT failure blames the checkout, not the script" 1 \
"fetch-depth: 0" mono_noref "$T" CHANGELOG_MONOTONIC_STRICT=1
# A missing changelog is an error on any setting — it is not a degradation,
# it is a wrong invocation.
T="$(monorepo nofile)"
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "monotonic: a missing changelog file is an error, never a skip" 1 "no such file" \
bash -c 'cd "$1" && bash "$2" base nope.md 2>&1' _ "$T" "$MONO"
# --- #98: uniqueness is a property of HEAD, so nothing base-side may gate it --
# Containment needs the merge base. Uniqueness needs only the file in front of
# it. As first written (and as inherited from heavy-duty/box, fixed there in
# box#144 for box#143) the duplicate check sat DOWNSTREAM of the base-ref,
# merge-base and base-blob conditions, so each of the degradation paths below
# exited 0 on a tree carrying a duplicate in plain sight — the base-blob one
# not even through skip(), but a bare `exit 0` that STRICT could not reach.
#
# These cases pin the ORDER, which is the actual invariant. Every monorepo
# fixture above commits MONO_BASE on 'base', so no case up there ever reaches
# the base-absent branch at all; and asserting the exit code alone is what let
# the original ship, since the clean base-absent case is green either way.
mononocl() { # mononocl <name> -> a repo whose 'base' has NO changelog, on 'work'
local d="$WORK/mono-$1"; mkdir -p "$d"
git -C "$d" init -q -b base
git -C "$d" config user.email harness@example.invalid
git -C "$d" config user.name harness
printf '%s\n' '# rig' > "$d/README.md"
git -C "$d" add README.md
git -C "$d" commit -qm 'base: no changelog yet'
git -C "$d" checkout -q -b work
printf '%s' "$d"
}
monoadd() { # monoadd <dir> <line...> — the branch INTRODUCES CHANGELOG.md
local d="$1"; shift
printf '%s\n' "$@" > "$d/CHANGELOG.md"
git -C "$d" add CHANGELOG.md
git -C "$d" commit -qm 'work: introduce the changelog'
}
# The changelog is absent at the merge base AND the branch introduces a
# duplicate. Before the fix this exited 0 on "nothing could have been deleted".
T="$(mononocl 98-newdup)"
monoadd "$T" '# Changelog' '' '## Unreleased' '' '## 0.2.0 — 2026-07-19' '' \
'- **A pending thing** (#2) — prose.' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.'
check "monotonic: a duplicate introduced where the base had NO changelog is CAUGHT (#98)" 1 \
"DUPLICATE release heading" mono "$T"
check "monotonic: ...and STRICT does not change that (it was never a skip)" 1 \
"DUPLICATE release heading" mono "$T" CHANGELOG_MONOTONIC_STRICT=1
# ...and the clean counterpart still passes, now SAYING uniqueness ran. Without
# this the case above could be satisfied by failing the base-absent path
# outright, which would redden every changelog-introducing branch.
T="$(mononocl 98-newok)"
monoadd "$T" '# Changelog' '' '## Unreleased' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.'
check "monotonic: ...while a CLEAN introduced changelog still passes" 0 \
"nothing could have been deleted" mono "$T"
check "monotonic: ...saying uniqueness was checked, not that nothing was" 0 \
"uniqueness on HEAD already passed" mono "$T"
# No git at all (a tarball, an unpacked release): uniqueness still has
# everything it needs, so a duplicate is caught rather than skipped past.
mkdir -p "$WORK/mono-98-nogit"
printf '%s\n' '# Changelog' '' '## 0.2.0 — 2026-07-19' '' \
'## 0.2.0 — 2026-07-19' > "$WORK/mono-98-nogit/CHANGELOG.md"
check "monotonic: a duplicate OUTSIDE a git work tree is caught (#98)" 1 \
"DUPLICATE release heading" mono "$WORK/mono-98-nogit"
# An unresolvable base ref: same — the skip belongs to containment, not to the
# script, so uniqueness has already run by the time skip() is reachable.
T="$(monorepo 98-nobase)"
monowrite "$T" '# Changelog' '' '## Unreleased' '' '## 0.2.0 — 2026-07-19' '' \
'- **A pending thing** (#2) — prose.' '' '## 0.2.0 — 2026-07-19' '' \
'- **A shipped thing** (#1) — prose.' '' '## 0.1.0 — 2026-07-01' '' \
'- **The first thing** (#0) — prose.'
check "monotonic: a duplicate is caught even when the base ref will not resolve (#98)" 1 \
"DUPLICATE release heading" mono_noref "$T"
# --- ci.yml: the monotonic step is actually wired (#98) ----------------------
# The guard runs from ci.yml, not from this suite, so pin the wiring the same
# way release.yml's is pinned — a script nothing invokes is not a check.
CIY="$ROOT/.github/workflows/ci.yml"
check "ci.yml: runs the monotonic guard" 0 "" \
grep -q "changelog-monotonic.sh" "$CIY"
check "ci.yml: ...with STRICT=1, so a skip is red rather than quietly green" 0 "" \
grep -qF "CHANGELOG_MONOTONIC_STRICT: '1'" "$CIY"
# shellcheck disable=SC2016 # the $-string is a literal in the target file
check "ci.yml: ...against the PR's base branch" 0 "" \
grep -qF 'origin/${{ github.base_ref' "$CIY"
# The step must NOT be pull-request-only. Deletion is vacuous on a push to main
# (the merge base IS HEAD), but DUPLICATION is vacuous on no tree at all, so
# gating the whole script left a duplicate reaching main by any other route
# unasserted. Dropping the gate is only safe with the ref_name fallback:
# `github.base_ref` is EMPTY on a push, a bare `origin/` does not resolve, and
# STRICT=1 promotes that to a hard failure on every push to main.
#
# Scoped to the step's OWN block, deliberately. As a file-wide grep this
# negative forbade any FUTURE step in ci.yml from being pull_request-gated and
# would have failed citing #98 when one legitimately was — #98 constrains this
# step, not the file. The companion check below is what keeps the awk honest:
# an extractor that matched nothing would turn the negative into a tautology
# that passes forever, including after someone renames the step and re-adds
# the gate.
# Terminates on a new STEP or a new JOB. The job boundary is not optional: the
# monotonic step is the LAST step of its job, so stopping only at the next
# `- name:` runs the block into the job below and swallows that job's
# level `if:` — the same bug this scoping fixed, moved from "any step in the
# file" to "this step plus the head of the next job" (found on box#144).
mono_step_block() {
awk '/^ - name: no shipped changelog heading/ {f=1; print; next}
f && (/^ - / || /^ [^ ]/) {exit}
f {print}' "$CIY"
}
# Anchored: an `if:` inside a `run:` line is not a step condition.
mono_step_gated() { mono_step_block | grep -q '^ if:'; }
check "ci.yml: the monotonic step itself is NOT pull_request-gated (#98)" 1 "" \
mono_step_gated
check "ci.yml: ...and the block was actually found (guards the awk above)" 0 \
"changelog-monotonic" mono_step_block
# shellcheck disable=SC2016 # the $-string is a literal in the target file
check "ci.yml: ...and falls back to ref_name, so a push has a base to resolve" 0 "" \
grep -qF 'github.base_ref || github.ref_name' "$CIY"
# Without full history the base ref does not resolve, and STRICT turns that
# into a red run — so the fetch depth is load-bearing, not incidental.
check "ci.yml: the checkout has full history (the base ref must resolve)" 0 "" \
grep -qF "fetch-depth: 0" "$CIY"
# --- the drill rule: does the version being shipped have a record? ----------
# CONTRIBUTING ("Releasing") has always required a real-hardware drill and
# nothing enforced it, so no release in this family has ever carried one: every
# other ceremony step is checked by a script, and the one that costs an
# afternoon was checked by a reviewer remembering. A bot finally blocked on it.
#
# Records are ONE FILE PER VERSION, at drills/<version>.md. The first cut of
# this guard kept them as sections in a single drill/RUNS.md and needed a
# heading grammar, an optional-date tail, a whole-version comparison and a
# non-blank-body rule to read them back — all of it there only because the
# records shared a file, and both sibling repos shipped a defect out of it in
# review. Splitting the files deletes most of these tests along with the code
# they covered: `0.3.0.md` and `0.3.0-rc1.md` cannot be confused by any
# grammar, because there is no grammar.
#
# Fixtures carry their OWN version file and their OWN drills dir, inside the
# fixture dir. This is not tidiness — it is heavy-duty/box#146, verbatim:
# fixtures that read the REPO's VERSION exercised the `-dev` branch on every
# ordinary tree, so the whole bare-version half of the guard was untested and
# went red for the first time while somebody was cutting a release. A fixture
# must state the tree it is about.
DRILL="$ROOT/.github/scripts/drill-recorded.sh"
check "drill-recorded.sh: exists and is the guard under test" 0 "" test -f "$DRILL"
check "drill-recorded.sh: is executable" 0 "" test -x "$DRILL"
drilltree() { # drilltree <name> <version> -> prints the dir (no drills/ yet)
local d="$WORK/drill-$1"; mkdir -p "$d"; printf '%s\n' "$2" > "$d/VERSION"
printf '%s' "$d"
}
drillrec() { # drillrec <dir> <version> <line...> — write drills/<version>.md
mkdir -p "$1/drills"; local f="$1/drills/$2.md"; shift 2
printf '%s\n' "$@" > "$f"
}
drill() { bash "$DRILL" "$1/drills" "$1/VERSION" 2>&1; }
# A development tree. Vacuous by construction — every ordinary PR looks like
# this, and none of them can be asked to have drilled a release that does not
# exist. It passes with no drills/ directory present AT ALL, which is the
# state this repo ships in today.
T="$(drilltree dev 0.2.1-dev)"
check "drill: a -dev tree passes with NO drills dir at all" 0 "" drill "$T"
check "drill: ...saying so out loud, not exiting 0 in silence" 0 \
"nothing to assert" drill "$T"
# The release ceremony tree, drilled and recorded. GREEN.
T="$(drilltree recorded 0.3.0)"
drillrec "$T" 0.3.0 '# Release drill — 0.3.0 — 2026-07-21' '' \
'Host: bare Debian 13. Candidate refs pinned: box@1a2b3c4, cast@9a0b1c2.' '' \
'- convergence, then re-converge: clean' \
'- db-integration: 14/14' '- runner lifecycle: PASS'
drillrec "$T" 0.2.0 '# Release drill — 0.2.0 — 2026-07-01' '' 'an older run'
check "drill: a bare VERSION with a non-empty record for it passes" 0 \
"records a drill for 0.3.0" drill "$T"
# The gate itself: a release tree with no drills/ directory at all. RED,
# naming the version, because "which release is unevidenced" is the only fact
# the author needs. This is the state a repo is in the first time it cuts a
# release under the gate — it must read as a to-do, not a broken invocation.
T="$(drilltree norecord 0.3.0)"
check "drill: a bare VERSION with NO drills dir FAILS" 1 \
"no drill record" drill "$T"
check "drill: ...and the failure names the version" 1 "VERSION is 0.3.0" drill "$T"
check "drill: ...and names the file it wanted" 1 "drills/0.3.0.md" drill "$T"
# A drills/ that exists but holds nothing for THIS version. Same failure —
# other releases having been drilled says nothing about this one.
T="$(drilltree otherversion 0.4.0)"
drillrec "$T" 0.3.0 '# Release drill — 0.3.0' '' 'the previous release'
check "drill: a drills dir with no file for THIS version FAILS" 1 \
"no drill record" drill "$T"
check "drill: ...naming the version that is unevidenced" 1 "VERSION is 0.4.0" drill "$T"
# ...and the failure has to say how to get out of it. Both moves are a commit
# on the PR, and the second one is the point of asking for a RECORD rather
# than a RESULT: a waiver is allowed, it just cannot be silent.
check "drill: ...and the failure names the unblock — run the drill" 1 \
"RUN THE DRILL" drill "$T"
check "drill: ...and the waiver, recorded, as the other way out" 1 \
"MAINTAINER WAIVER" drill "$T"
check "drill: ...and points at the README for what a record contains" 1 \
"README.md" drill "$T"
# An EMPTY file at the right path. This is the failure a laxer guard invites —
# the ceremony PR touches the file to get green and fills it in never.
T="$(drilltree emptyfile 0.3.0)"
mkdir -p "$T/drills"; : > "$T/drills/0.3.0.md"
check "drill: a PRESENT but EMPTY record FAILS" 1 "no drill record" drill "$T"
# ...and WHITESPACE is not a record either. This is the ONE piece of the old
# section-parsing rule set that splitting the files did not make
# unrepresentable, so it is the one that still needs a test. It is here because
# the siblings got it wrong: box#149 and cast#138 both extracted with
# `sed '/./,$!d'`, where `.` matches a space, so one tab satisfied the gate and
# shipped an evidence-free release. All three reviewers caught it there.
# Nothing caught it here, because there was nothing to catch — which is exactly
# the state in which a later simplification quietly reintroduces it.
T="$(drilltree blank 0.3.0)"
drillrec "$T" 0.3.0 ' ' ' ' ''
check "drill: a record of only spaces, tabs and newlines FAILS (box#149, cast#138)" \
1 "no drill record" drill "$T"
# The version is matched WHOLE, both directions — and now the filesystem does
# it, since the version IS the filename. A drill run against a release
# candidate is not evidence for the final release, and the reverse is equally
# false: in both cases the string that matched is not the artefact that ships.
T="$(drilltree whole-rc 0.3.0)"
drillrec "$T" 0.3.0-rc1 '# Release drill — 0.3.0-rc1' '' 'the rc drill'
check "drill: an -rc1 record does NOT satisfy the bare version" 1 \
"no drill record" drill "$T"
T="$(drilltree whole-final 0.3.0-rc1)"
drillrec "$T" 0.3.0 '# Release drill — 0.3.0' '' 'the final drill'
check "drill: ...and a bare-version record does NOT satisfy the -rc1" 1 \
"no drill record" drill "$T"
# A missing VERSION file is a wrong invocation, not a degradation — there is
# no version to be lenient about, so it must never read as a pass.
T="$(drilltree noversion 0.3.0)"
rm -f "$T/VERSION"
check "drill: a missing VERSION file is an error, never a pass" 1 "no such file" drill "$T"
# The real files, last. The shipped README must be readable, and the guard the
# repo actually runs must pass on the VERSION the repo actually carries.
check "drills/README.md: exists" 0 "" test -f "$ROOT/drills/README.md"
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "drills/README.md: documents the one-file-per-version naming rule" 0 "" \
bash -c 'grep -qF "drills/<version>.md" "$1"' _ "$ROOT/drills/README.md"
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "drills/README.md: says a FAILED drill is still a valid record" 0 "" \
bash -c 'grep -qi "failed drill is still a valid record" "$1"' _ "$ROOT/drills/README.md"
# The old single-file record must be gone, not merely unreferenced: a stale
# drill/RUNS.md would be a second place to write a record that nothing reads.
check "drill/RUNS.md: is gone — records live one per version now" 1 "" \
test -e "$ROOT/drill/RUNS.md"
# The property is that the guard's VERDICT IS CORRECT FOR THIS TREE — not that
# it always passes. Those come apart on a ceremony tree: a -dev tree is vacuous
# and must pass, but a ceremony tree passes only once a human has run the drill
# and written the record, which is the entire point of the gate. Asserting
# exit 0 unconditionally made test/release.sh UN-GREENABLE on every release
# branch before its drill, and surfaced as a `release-flow tests` failure rather
# than as the gate doing its job — the same misattribution shape as box#146,
# where a fixture read the repo's real VERSION and only misbehaved on the
# ceremony tree. Caught when box#148 went red for the wrong-looking reason.
THIS_VER="$(tr -d '[:space:]' < "$ROOT/VERSION")"
case "$THIS_VER" in
*-dev)
check "drill-recorded.sh: THIS tree is -dev, and the guard is vacuous on it" 0 "" \
bash "$DRILL" "$ROOT/drills" "$ROOT/VERSION" ;;
*)
if [ -s "$ROOT/drills/$THIS_VER.md" ]; then
check "drill-recorded.sh: THIS ceremony tree HAS its record, and the guard accepts it" 0 "" \
bash "$DRILL" "$ROOT/drills" "$ROOT/VERSION"
else
check "drill-recorded.sh: THIS ceremony tree has NO record yet, and the guard refuses it" 1 "no drill record at" \
bash "$DRILL" "$ROOT/drills" "$ROOT/VERSION"
fi ;;
esac
# ...and with no arguments at all, since that is how ci.yml invokes it. The
# defaults must be the paths this repo actually uses.
# ...and with its DEFAULT arguments, as CI runs it. What this pins is that the
# defaults ARE the paths this repo uses — so it asserts the defaults reach the
# same verdict as the explicit call above, not a fixed exit code. Hard-coding 0
# here would fail on a ceremony tree for the same wrong reason the check above
# used to.
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "drill-recorded.sh: ...and its DEFAULT arguments agree, as CI runs it" 0 "" \
bash -c '
cd "$1" || exit 9
bash .github/scripts/drill-recorded.sh >/dev/null 2>&1; d=$?
bash .github/scripts/drill-recorded.sh drills VERSION >/dev/null 2>&1; e=$?
[ "$d" -eq "$e" ]' _ "$ROOT"
# ci.yml: the guard runs from there, so pin the wiring — a script nothing
# invokes is not a check (same reasoning as the monotonic pins above).
check "ci.yml: runs the drill guard" 0 "" grep -q "drill-recorded.sh" "$CIY"
# ...and is NOT trigger-gated. It is vacuous on every -dev tree already, so an
# `if:` could only ever exempt the one tree it exists for.
drill_step_block() {
awk '/^ - name: a release version has a recorded drill/ {f=1; print; next}
f && (/^ - / || /^ [^ ]/) {exit}
f {print}' "$CIY"
}
drill_step_gated() { drill_step_block | grep -q '^ if:'; }
check "ci.yml: the drill step itself is NOT trigger-gated" 1 "" drill_step_gated
check "ci.yml: ...and the block was actually found (guards the awk above)" 0 \
"drill-recorded" drill_step_block
# CONTRIBUTING must state the gate, and must state what the drill actually is.
# The three repos' drills are INDEPENDENT — run in any order, on any schedule —
# and what makes that safe is that each one pins the same fixed set of
# CANDIDATE refs, so box and rig measure the same pair. That, not sequencing,
# is what dissolves the mutual recursion (rig builds the host box runs on, and
# box mints the seeds rig converges). An earlier draft of this doc claimed a
# fixed box → rig → cast release order; it is wrong, and this pins the
# correction.
CONTRIB="$ROOT/CONTRIBUTING.md"
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "CONTRIBUTING: the release flow names the drill gate" 0 "" \
bash -c 'grep -qF "drills/<version>.md" "$1"' _ "$CONTRIB"
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "CONTRIBUTING: ...and says the three repos' drills are INDEPENDENT" 0 "" \
bash -c 'grep -qi "drills are independent" "$1"' _ "$CONTRIB"
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "CONTRIBUTING: ...pinned to one fixed set of candidate refs" 0 "" \
bash -c 'grep -qi "same fixed set of candidate refs" "$1"' _ "$CONTRIB"
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "CONTRIBUTING: ...which is what dissolves the recursion, not sequencing" 0 "" \
bash -c 'grep -qF "RIG_REF" "$1"' _ "$CONTRIB"
# The negative that keeps the correction from being re-lost: no fixed release
# order may be claimed. Nothing requires box to ship before rig.
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "CONTRIBUTING: ...and never claims a fixed box-then-rig release order" 1 "" \
bash -c 'grep -qi "box first, then rig" "$1"' _ "$CONTRIB"
# --- release.yml: the pins ---------------------------------------------------
# The workflow itself runs only on a tag push upstream, so pin its
# load-bearing pieces the way the harness pins root-only paths (repo
# precedent: the tag-refusal greps in test/cli.sh).
RY="$ROOT/.github/workflows/release.yml"
check "release.yml: exists" 0 "" test -f "$RY"
check "release.yml: triggers on tag pushes" 0 "" grep -q "tags:" "$RY"
check "release.yml: sources the shared lib (one extractor, not a copy)" 0 "" \
grep -q "release-lib.sh" "$RY"
check "release.yml: the body comes from changelog_section" 0 "" \
grep -q "changelog_section CHANGELOG.md" "$RY"
check "release.yml: a tag/VERSION mismatch refuses to create" 0 "" \
grep -q "refusing to create a release" "$RY"
check "release.yml: an empty changelog section refuses too" 0 "" \
grep -q "has no '## " "$RY"
check "release.yml: gh release create verifies the tag" 0 "" \
grep -q -- "--verify-tag" "$RY"
# Ordering: the mismatch assert must precede the create (line compare, the
# repo's marker-then-box idiom; defaults fail closed).
assert_at="$(grep -n "refusing to create a release" "$RY" | head -n1 | cut -d: -f1)"
create_at="$(grep -n "gh release create" "$RY" | head -n1 | cut -d: -f1)"
check "release.yml: the assert precedes the create" \
0 "" test "${assert_at:-999999}" -lt "${create_at:-0}"
# --- release.yml, the merge path: the pins (#47; box#96's design) ------------
# Merging the release-labeled ceremony PR IS the release. Same grep-pin
# treatment for the merge path's load-bearing pieces: the gate, the four
# fail-loud asserts, the same-job tag+publish, and the surviving tag-push
# fallback.
# The merge door rides pushes to MAIN, not pull_request events: a fork PR's
# pull_request run gets a read-only GITHUB_TOKEN (permissions: cannot raise
# it), and every ceremony PR this org merges is cross-repo from the bot
# fork — the tag create would 403 after green asserts (#48 round 1). The
# label — the operator's intent — is read via the API off the merge commit.
check "release.yml: the merge door rides pushes to main (fork-token-proof)" 0 "" \
grep -qF "branches: [main]" "$RY"
# YAML maps are last-key-wins: a second sibling push: key silently replaces
# the first and kills a door (grok's round-2 catch — the tag fallback had
# stopped triggering). Exactly ONE push key may exist.
check "release.yml: exactly one on.push key (duplicate keys drop a door)" 0 "1" \
grep -cE '^ push:' "$RY"
check "release.yml: ...and the doors split on the ref (tag door takes tags)" 0 "" \
grep -qF "startsWith(github.ref, 'refs/tags/')" "$RY"
# shellcheck disable=SC2016 # the $-string is a literal in the target file
check "release.yml: the release label is read via the API off the merge commit" 0 "" \
grep -qF 'commits/$MERGE_SHA/pulls' "$RY"
check "release.yml: a transition without a labeled PR refuses" 0 "" \
grep -qF "no merged, release-labeled PR is behind this commit" "$RY"
# The decide step tells the label's two meanings apart (LABELS.md gives
# `release` to release-flow WORK as well as to the ceremony PR): work under
# the label is a green NOTICE no-op — in the -dev steady state and in the
# post-release window (bare, unchanged, already released) — while every
# half-ceremony refuses. Pin each verdict's message and the gating output.
check "release.yml: decide — dev-tree work no-ops green (not a red run per infra PR)" 0 "" \
grep -qF "release-flow work under the release label, not a ceremony" "$RY"
check "release.yml: decide — a -dev endstate is always work (the bump PR no-ops green)" 0 "" \
grep -qF "a dev tree is by definition not a release" "$RY"
check "release.yml: decide — post-release-window work no-ops green" 0 "" \
grep -qF "release-flow work merged in the post-release window" "$RY"
check "release.yml: decide — bare, unchanged, never released refuses to guess" 0 "" \
grep -qF "Refusing to guess" "$RY"
# shellcheck disable=SC2016 # the $-refs are the inner bash -c's, deliberately
check "release.yml: decide gates every later step on ceremony=yes" 0 "" \
bash -c '[ "$(grep -cF "if: steps.decide.outputs.ceremony == '\''yes'\''" "$1")" -ge 3 ]' _ "$RY"
check "release.yml: assert 3 — an empty section refuses to publish" 0 "" \
grep -qF "refusing to publish an empty release" "$RY"
check "release.yml: assert 4 — an existing tag or release refuses (idempotent)" 0 "" \
grep -qF "refusing to re-release" "$RY"
# Same-job matters: a GITHUB_TOKEN-created tag fires no tag-push workflow,
# so the publish must live NEXT TO the tag creation. The workflow keeps
# release-on-merge as its last job (pinned by comment there) so the awk
# range runs to EOF; both acts must land inside it.
MJOB="$(awk '/^ release-on-merge:/,0' "$RY")"
mjob_has() { printf '%s' "$MJOB" | grep -qF -e "$1"; }
check "release.yml: the merge job API-creates the tag itself" 0 "" \
mjob_has "git/refs"
# shellcheck disable=SC2016 # the $-string is a literal in the target file
check "release.yml: ...at the pushed main head (github.sha = the merge commit)" 0 "" mjob_has 'sha="$MERGE_SHA"'
# The release re-arms main itself: the post-release -dev bump is arithmetic,
# not judgment, so it rides the same job — direct push, PR fallback.
check "release.yml: the release bumps main to the next -dev itself" 0 "" \
grep -qF "bump main to the next -dev" "$RY"
check "release.yml: ...with a PR fallback when the direct push is refused" 0 "" \
grep -qF "opening the bump PR instead" "$RY"
check "release.yml: ...and publishes in the SAME job" 0 "" \
mjob_has "gh release create"
# Ordering, the marker-then-box idiom again: the last assert's refusal must
# precede the tag creation (asserts first, acts last; defaults fail closed).
massert_at="$(grep -n "refusing to re-release" "$RY" | head -n1 | cut -d: -f1)"
mtag_at="$(grep -n "git/refs" "$RY" | head -n1 | cut -d: -f1)"
check "release.yml: the merge-path asserts precede the tag" \
0 "" test "${massert_at:-999999}" -lt "${mtag_at:-0}"
# ...and the manual path SURVIVES: tag-push trigger plus a push-gated job,
# the documented fallback and backfill.
check "release.yml: the tag-push trigger survives (manual fallback intact)" 0 "" \
grep -qF "tags: ['**']" "$RY"
check "release.yml: the fallback job is gated to push events" 0 "" \
grep -qF "github.event_name == 'push'" "$RY"
# --- the installer's ref logic, extracted ------------------------------------ # --- the installer's ref logic, extracted ------------------------------------
# install.sh must stay a single curl|bash file, so its channel functions live # install.sh must stay a single curl|bash file, so its channel functions live
# inline; extract them here and drive them for real (the valid_version awk # inline; extract them here and drive them for real (the valid_version awk
# idiom from test/cli.sh), against a stub curl — never the network. # idiom from test/cli.sh), against a stub curl — never the network.
RL="$WORK/installer-fns.sh" RL="$WORK/installer-fns.sh"
# Grouped redirect — shellcheck SC2129 flags four individual >> to the same awk '/^resolve_latest_tag\(\) \{/,/^\}/' "$ROOT/install.sh" > "$RL"
# file (crossed the threshold when release_tag_url joined the extract set). awk '/^ref_candidate_urls\(\) \{/,/^\}/' "$ROOT/install.sh" >> "$RL"
{
awk '/^resolve_latest_tag\(\) \{/,/^\}/' "$ROOT/install.sh"
awk '/^ref_candidate_urls\(\) \{/,/^\}/' "$ROOT/install.sh"
awk '/^release_tag_url\(\) \{/,/^\}/' "$ROOT/install.sh"
awk '/^install_script_url\(\) \{/,/^\}/' "$ROOT/install.sh"
} > "$RL"
check "installer fns extracted (guards the awk)" 0 "redirect_url" cat "$RL" check "installer fns extracted (guards the awk)" 0 "redirect_url" cat "$RL"
STUB="$WORK/stub"; mkdir -p "$STUB" STUB="$WORK/stub"; mkdir -p "$STUB"
@ -99,57 +833,14 @@ check "resolve: a tagless releases/tag/ redirect fails" 1 "" \
check "resolve: a failing curl fails (network down is not a channel)" 1 "" \ check "resolve: a failing curl fails (network down is not a channel)" 1 "" \
rlt CURL_STUB_FAIL=1 rlt CURL_STUB_FAIL=1
rcu_line() { # rcu_line <n> [VAR=val ...] — the nth candidate URL for an explicit ref rcu_line() { # rcu_line <n> — the nth candidate URL for an explicit ref
local n="$1"; shift bash -c 'set -euo pipefail
# shellcheck disable=SC2016 . "$1"; ref_candidate_urls acme/widgets 1.2.3 | sed -n "${2}p"' _ "$RL" "$1"
env "$@" bash -c 'set -euo pipefail
. "$1"; ref_candidate_urls acme/widgets 1.2.3 | sed -n "${2}p"' _ "$RL" "$n"
} }
check "candidates: refs/tags first — the pin outranks a same-named branch" 0 \ check "candidates: refs/tags first — the pin outranks a same-named branch" 0 \
"https://github.com/acme/widgets/archive/refs/tags/1.2.3.tar.gz" rcu_line 1 "https://github.com/acme/widgets/archive/refs/tags/1.2.3.tar.gz" rcu_line 1
check "candidates: refs/heads is the fallback" 0 \ check "candidates: refs/heads is the fallback" 0 \
"https://github.com/acme/widgets/archive/refs/heads/1.2.3.tar.gz" rcu_line 2 "https://github.com/acme/widgets/archive/refs/heads/1.2.3.tar.gz" rcu_line 2
# RIG_HOST is host-only (#111): Forgejo serves the same refs/{tags,heads}/
# pair (measured 2026-07-29). Host substituted; grammar unchanged.
check "candidates: Forgejo host uses the same refs/tags form" 0 \
"https://forgejo.example/acme/widgets/archive/refs/tags/1.2.3.tar.gz" \
rcu_line 1 RIG_HOST=https://forgejo.example
check "candidates: Forgejo host keeps refs/heads as fallback" 0 \
"https://forgejo.example/acme/widgets/archive/refs/heads/1.2.3.tar.gz" \
rcu_line 2 RIG_HOST=https://forgejo.example
# shellcheck disable=SC2016
check "candidates: Forgejo host emits exactly two candidates" 0 "2" \
env RIG_HOST=https://forgejo.example bash -c 'set -euo pipefail
. "$1"; ref_candidate_urls acme/widgets 1.2.3 | grep -c .' _ "$RL"
check "candidates: trailing slash on RIG_HOST is stripped" 0 \
"https://forgejo.example/acme/widgets/archive/refs/tags/1.2.3.tar.gz" \
rcu_line 1 RIG_HOST=https://forgejo.example/
# install_script_url — the curl|bash hint must match the forge (#111).
isu() {
# shellcheck disable=SC2016
env "$@" bash -c 'set -euo pipefail
REPO=heavy-duty/rig; . "$1"; install_script_url' _ "$RL"
}
check "install_script_url: GitHub default uses raw.githubusercontent.com" 0 \
"https://raw.githubusercontent.com/heavy-duty/rig/main/install.sh" isu
check "install_script_url: Forgejo uses /raw/branch/main/" 0 \
"https://forgejo.example/heavy-duty/rig/raw/branch/main/install.sh" \
isu RIG_HOST=https://forgejo.example
# resolve_latest_tag follows RIG_HOST too — the probe URL must name the forge.
rlt_log="$WORK/rlt-log"
: > "$rlt_log"
check "resolve: RIG_HOST is the releases/latest origin" 0 "0.2.0" \
rlt CURL_STUB_REDIRECT=https://forgejo.example/heavy-duty/rig/releases/tag/0.2.0 \
RIG_HOST=https://forgejo.example CURL_STUB_LOG="$rlt_log"
check "resolve: the probe hit the Forgejo host" 0 \
"https://forgejo.example/heavy-duty/rig/releases/latest" \
cat "$rlt_log"
# Forgejo's no-release path is a 404 (not GitHub's /releases redirect) —
# curl -f fails and || return 1 fires. Drive that branch under RIG_HOST.
check "resolve: Forgejo no-release is a failing curl (404), not a /releases redirect" 1 "" \
rlt RIG_HOST=https://forgejo.example CURL_STUB_FAIL=1
# --- the three channels, driven through the REAL installer ------------------- # --- the three channels, driven through the REAL installer -------------------
# Full install.sh runs against throwaway roots with the stub curl on PATH: the # Full install.sh runs against throwaway roots with the stub curl on PATH: the
@ -193,27 +884,6 @@ check "channel latest: the refusal says what is missing" 1 "no release" \
rinst "$H2" "$B2" CURL_STUB_REDIRECT=https://github.com/heavy-duty/rig/releases rinst "$H2" "$B2" CURL_STUB_REDIRECT=https://github.com/heavy-duty/rig/releases
check "channel latest: the refusal installed NOTHING" 1 "" test -e "$H2" check "channel latest: the refusal installed NOTHING" 1 "" test -e "$H2"
# Channel 1, regression — a resolved tag whose archive is gone must FAIL,
# never fall through to refs/heads/<tag> and still claim the release
# (claude REQUEST_CHANGES on !114: INSTALLED_FROM would name the tag for a
# branch tree). CURL_STUB_OK only matches heads — if the installer tries it,
# the install would succeed and this check would fail.
H2b="$WORK/h2b"; B2b="$WORK/b2b"; LOG2b="$WORK/log2b"
check "channel latest: missing tag archive does NOT fall through to heads" \
1 "no candidate URL worked" rinst "$H2b" "$B2b" \
CURL_STUB_REDIRECT=https://github.com/heavy-duty/rig/releases/tag/3.3.3 \
CURL_STUB_OK=refs/heads/3.3.3 CURL_STUB_LOG="$LOG2b"
check "channel latest: ...and installed NOTHING (branch was never taken)" 1 "" \
test -e "$H2b"
# The log also holds the releases/latest probe; the download tries are the
# archive URLs. Exactly one archive try, and it is refs/tags — never heads.
check "channel latest: ...exactly one archive URL was tried" 0 "1" \
grep -c '/archive/' "$LOG2b"
check "channel latest: ...that try was refs/tags" 0 "refs/tags/3.3.3" \
cat "$LOG2b"
check "channel latest: ...refs/heads was never consulted" 1 "" \
grep -q 'refs/heads/' "$LOG2b"
# Channel 2 — RIG_REF=<tag>: refs/tags wins, and the latest-release probe is # Channel 2 — RIG_REF=<tag>: refs/tags wins, and the latest-release probe is
# never consulted (a pin resolves nothing). # never consulted (a pin resolves nothing).
H3="$WORK/h3"; B3="$WORK/b3"; LOG3="$WORK/log3" H3="$WORK/h3"; B3="$WORK/b3"; LOG3="$WORK/log3"
@ -237,65 +907,8 @@ check "channel dev: ...then the branch URL" 0 "refs/heads/feature-x" \
# Neither a tag nor a branch: both candidates miss, and the die says so. # Neither a tag nor a branch: both candidates miss, and the die says so.
H5="$WORK/h5"; B5="$WORK/b5" H5="$WORK/h5"; B5="$WORK/b5"
check "channel: a ref that is neither tag nor branch dies naming the tries" \ check "channel: a ref that is neither tag nor branch dies naming both tries" \
1 "no candidate URL worked" rinst "$H5" "$B5" RIG_REF=no-such-ref 1 "not a tag and not a branch" rinst "$H5" "$B5" RIG_REF=no-such-ref
# Channel 4 — RIG_HOST=Forgejo: same refs/tags→refs/heads candidate order and
# the same /releases/latest redirect grammar (#111). The stub succeeds only
# when the refs/tags form is requested — a regression that still emitted the
# bare /archive/<ref> form would fail here.
H9="$WORK/h9"; B9="$WORK/b9"; LOG9="$WORK/log9"
check "channel forgejo latest: resolves and installs via refs/tags archive URL" 0 "done" \
rinst "$H9" "$B9" RIG_HOST=https://forgejo.example \
CURL_STUB_REDIRECT=https://forgejo.example/heavy-duty/rig/releases/tag/7.7.7-relflow \
CURL_STUB_OK='/archive/refs/tags/7.7.7-relflow.tar.gz' CURL_STUB_LOG="$LOG9"
check "channel forgejo latest: download URL is the refs/tags form" 0 \
"https://forgejo.example/heavy-duty/rig/archive/refs/tags/7.7.7-relflow.tar.gz" \
cat "$LOG9"
check "channel forgejo latest: the tree landed" 0 "" \
test -x "$H9/versions/7.7.7-relflow/bin/rig"
H10="$WORK/h10"; B10="$WORK/b10"
check "channel forgejo pinned: RIG_REF=main falls through to refs/heads" 0 "done" \
rinst "$H10" "$B10" RIG_HOST=https://forgejo.example RIG_REF=main \
CURL_STUB_OK='/archive/refs/heads/main.tar.gz'
check "channel forgejo pinned: the tree landed" 0 "" \
test -x "$H10/versions/7.7.7-relflow/bin/rig"
# Refusal hint on a non-GitHub host must not send the operator to
# raw.githubusercontent.com (that 404s from a Forgejo-only tree).
H11="$WORK/h11"; B11="$WORK/b11"
check "channel forgejo latest: no-release hint uses the Forgejo raw URL" \
1 "https://forgejo.example/heavy-duty/rig/raw/branch/main/install.sh" \
rinst "$H11" "$B11" RIG_HOST=https://forgejo.example \
CURL_STUB_REDIRECT=https://forgejo.example/heavy-duty/rig/releases
# --- the local channel: RIG_INSTALL_SOURCE (#106) ----------------------------
# A supported input, not test scaffolding — CI's `install:` job and test/cli.sh
# both install THIS checkout through it. What release.sh owes is the channel's
# contract: a directory installs, a tarball installs, neither touches the
# network, and a bad path refuses BY NAME — never a silent fallback to
# downloading a release, which would leave a green CI job testing the wrong
# tree. The stub curl's log is the network witness: any download, even an
# attempted one, would land a URL in it.
H6="$WORK/h6"; B6="$WORK/b6"; LOG6="$WORK/log6"
check "channel local: a directory installs" 0 "done" \
rinst "$H6" "$B6" RIG_INSTALL_SOURCE="$TBDIR/rig-7.7.7-relflow" CURL_STUB_LOG="$LOG6"
check "channel local: the tree landed under its VERSION" 0 "" \
test -x "$H6/versions/7.7.7-relflow/bin/rig"
check "channel local: INSTALLED_FROM records local:<path>" 0 \
"local:$TBDIR/rig-7.7.7-relflow" cat "$H6/versions/7.7.7-relflow/INSTALLED_FROM"
check "channel local: curl was never consulted" 1 "" test -s "$LOG6"
H7="$WORK/h7"; B7="$WORK/b7"; LOG7="$WORK/log7"
check "channel local: a tarball installs too" 0 "done" \
rinst "$H7" "$B7" RIG_INSTALL_SOURCE="$WORK/release.tgz" CURL_STUB_LOG="$LOG7"
check "channel local: the tarball's tree landed" 0 "" \
test -x "$H7/versions/7.7.7-relflow/bin/rig"
check "channel local: ...also without a download" 1 "" test -s "$LOG7"
H8="$WORK/h8"; B8="$WORK/b8"; LOG8="$WORK/log8"
check "channel local: a missing path refuses BY NAME" 1 "$WORK/no-such-source" \
rinst "$H8" "$B8" RIG_INSTALL_SOURCE="$WORK/no-such-source" CURL_STUB_LOG="$LOG8"
check "channel local: the refusal installed NOTHING" 1 "" test -e "$H8"
check "channel local: ...and downloaded nothing (no silent fallback)" 1 "" \
test -s "$LOG8"
rm -rf "$WORK" rm -rf "$WORK"