feat: CI refuses a release PR with no drill record #102
Labels
No labels
attention
blocked
blocker:ci-red
blocker:conflict
blocker:drill-pending
blocker:unrequested
bug
claimed
documentation
enhancement
epic
merge-next
needs-ruling
needs-triage
offsite
post-merge
ready
release
scope:bootstrap
scope:coolify
scope:db
scope:docs
scope:drill
scope:installer
scope:labels
scope:platform
scope:runner
scope:users
stale
state:addressing
state:bots-reviewing
state:building
state:needs-human
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference: heavy-duty/rig#102
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/drill-gate"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
CI now refuses a
releasePR that has no drill record. The rule stops being something a reviewer has to remember.Why
heavy-duty/box#148's round found it: CONTRIBUTING says the release PR carries the full real-hardware drill recorded in
drill/RUNS.md, and no release has ever done it — box#95, box#114 and box#148 all shipped as version-file +CHANGELOG.mdonly. One of three bots caught it; two did not. That is the predictable failure of a rule that lives only in reviewer prompts: three independent reviewers each have to remember it, every time.This repo family already encodes release invariants as fail-loud scripts —
changelog-armed.sh,changelog-monotonic.sh,release-notes.sh. The drill gate becomes one too.The guard
.github/scripts/drill-recorded.sh, keyed on the version, same shape as the changelog guards:-devtree → passes trivially. Nothing ships from a development tree.## Release drill — X.Y.Z — DATEsection indrill/RUNS.mdthat extracts non-empty.Version is compared whole, so
0.9.0-rc1cannot satisfy0.9.0or vice-versa —release-notes.sh's existing trap, solved the same way (awk field equality, no regex, no dot-escaping).Not gated on the
releaselabel. Gating there would put the assert behind a hand-applied label, making it absent from exactly the PR that mislabels itself. It keys off the version file instead.Not gated to
pull_requesteither — the merge that publishes a release is a push to main carrying the same bare version, so a PR-only check leaves the tree that actually ships unasserted (box#143's reasoning).A waiver is possible, and still recorded
The guard requires a record, not a passing result. A release that must ship without a drill writes the section and says plainly that it was WAIVED and why. That turns a skip into a visible line in the diff instead of the silent gap that let three releases through.
The drill is one orchestrated run — box and rig are mutually recursive
Worth stating because the obvious model is wrong. There is no linear order between box and rig:
rig bootstrap … --host yesinstalls box and runs box'ssetup-hostbox newmints a seed whose cloud-init curls rig's installer at@RIG_REPO@/@RIG_REF@and runsrig bootstrap <tenant>-boxrig sits below box as host-builder and above it as guest-converger. So "release A before you can drill B" is unsatisfiable in principle, not merely inconvenient.
The resolution is to drill candidate refs, not released artifacts.
RIG_REPO/RIG_REFare mint-time environment variables, so a run pins the exact commits under test — and drilling the candidate is drilling the release, because a release PR's diff is version-file +CHANGELOG.mdand nothing executable differs between the drilled tree and the shipped one.One run, in stack order (
rig --host yes→box new→ seed converges → cast), emits one shared run ID. Each repo records its own legs citing that ID and the other two SHAs.Per-repo records, deliberately
Each repo's guard reads only its own
drill/RUNS.md. A guard that reaches into another repository to decide whether this one may ship degrades to "pass" the moment the lookup fails — the same class as the unreadable-rollup bug (heavy-duty/rig#90) where a failed read presented as "nothing is failing". A shared run ID gives correlation without that coupling.Expected consequence
The open release PR in this repo will go red when this merges — correctly, because no drill record exists yet. It is already
blockedpending the drill and goes green when the record lands.Labels
blocker:drill-pendingis documented in LABELS.md but not created — the bot account gets a 403 creating labels, so a maintainer needs to create it in all three repos.blockedstands in meanwhile. It sits outside the reconciler'sBLOCKERSset, so it is hand-applied and not stripped on the next sweep.Verdict: I agree with everything and have no additional feedback.
✅ Approved — I agree with all of this, no concerns.
Verified the one thing that blocked this PR's siblings: box#149 and cast#138 extract the section with
sed '/./,$!d', which lets a spaces-only body count as a record — rig'sdrill_section()does not have that defect, becausefound && !body && /^[[:space:]]*$/ { next }skips whitespace-only lines until genuine content appears, so a blank-padded heading extracts empty and fails as documented. The whole-version comparison holds in both directions and against longer versions (0.3.0.1prefix case is tested, which neither sibling covers), the empty-section-can't-scavenge-the-next-section case is pinned, and treating a missingdrill/RUNS.mdas "no record, here's the unblock" rather than a crash is the right call for the first release under the gate. Non-blocking: when the family aligns on the box#149/cast#138 fix, a spaces-only-body fixture here would pin the behavior this script already gets right.— automated review by
claude-bot-andresmgsl· heavy-duty-review-botVerdict: Approve — I agree with this as-is.
First review of tip
77a9a1a. Same family as box#149 / cast#138, and this port gets the hard part right.What holds
drill_sectionrejects whitespace-only bodies.found && !body && /^[[:space:]]*$/ { next }skips blank padding until real content; a spaces-only section extracts empty and fails. Verified against the sed-based siblings, which do not.""or" —…") — rc / longer-version traps closed without field-index fragility on the date.-devvacuous, missingRUNS.mdsame path as missing section (good first-release UX), per-repo only.Nits (non-blocking)
No blockers from me.
@danmt — round 1 closed at
77a9a1a. Three head-current approvals, zero changes requested.What it does
CI refuses a
releasePR that has no drill record. The rule stops depending on a reviewer remembering it.Why this one passed while its siblings did not
Worth your attention, because it is the useful signal in this batch. box#149 and cast#138 each drew three CHANGES_REQUESTED for the same defect: their extractor piped through
sed '/./,$!d', and.matches a space — so a heading followed by a single tab satisfied the gate. An evidence-free release for the price of an invisible character.rig's implementation does not have that bug. It uses a different extraction, and I verified that against the same fixtures rather than assuming the approvals meant it was safe:
## Release drill — 0.3.0 stray words0.3.0-rc1record for0.3.0Both siblings are now fixed and re-requested; the family agrees on the grammar by measurement, not by hope.
Verification on
77a9a1atest/release.shtest/cli.shtest/labels-reconcile.sh.shchangelog-monotonic.sh origin/maindrill-recorded.shon this tree0.2.1-devis a development treeDesign points, so the merge is an informed one
VERSION, not thereleaselabel. A label-gated check is absent from exactly the PR that mislabels itself.pull_request. The merge that publishes a release is a push to main carrying the same bare VERSION; a PR-only check leaves the shipping tree unasserted.Expected consequence
#101 (
release: 0.3.0) goes red when this merges — correctly, because no drill record exists yet. It already carriesblocker:drill-pendingand goes green when the record lands.One judgement call in it: two of the four
BREAKING:markers and the ordered-rollout text describe box↔rig as mutually recursive rather than linearly ordered. That was a correction mid-flight — the first draft claimed a fixed box-then-rig release order, which is unsatisfiable in principle sincerig --host yesinstalls box and box's guests install rig. A test now pins the corrected claim, including a negative that fails if the wrong ordering returns.Reopening my own handoff at
b234f48, deliberately. @danmt — nothing is wrong with this PR, but I am invalidating your three approvals on purpose and want to say why rather than let a surprise push land.The question that prompted it
"Is the finding fixed in rig#102?" — the whitespace bypass that all three of you blocked box#149 and cast#138 for.
It was never present here. This guard skips leading blanks with
/^[[:space:]]*$/, which matches a spaces-or-tabs line the same as a bare one, and only setsbodyon real content. The siblings usedsed '/./,$!d', where.matches a space. That difference is why this PR drew three approvals in the same round its twins drew three blocks — a different implementation, not luck.But nothing pinned it
The
emptyfixture uses genuinely empty lines. There was no whitespace-only case, so the correct behaviour here was incidental — nobody had tested it because nobody had broken it. That is precisely the state in which a later "simplify this to match its siblings" reintroduces the bypass with no test to catch it. And the siblings are the ones that had it, so that refactor is a plausible one, not a hypothetical.What this commit adds
One fixture: a record body of one spaces-only line and one tab-only line, expected to fail.
Verified to have teeth by mutation, not just by passing. I copied the guard, reverted the skip rule to
/^$/(the siblings' semantics), and ran the fixture against both:So the test fails when the property breaks, rather than passing for free.
test/release.sh: 132 → 133 passed, 0 failed.The cost, stated plainly
This burns a review round on a PR that was ready to merge, and it re-invalidates approvals I had already handed to you. I judged that worth it: the entire argument for this PR is that a gate must be pinned by tests rather than trusted, and shipping the one guard in the family whose correctness rests on nobody having touched it would undercut that. The three bots are re-requested.
Nothing else changed — the guard, the CI wiring and every other test are byte-identical to what you already approved.
✅ Approved — I agree with all of this, no concerns.
Re-approving at
b234f48. The only delta since my77a9a1aapproval is the spaces-only-body fixture I flagged as a non-blocking nit — now added at test/release.sh:548, so that prior point is resolved. I verified it independently rather than taking the handoff's word: the unmodified guard refuses the whitespace-only record (exit 1, "NO drill record"), a mutant with the siblings'/^$/skip semantics accepts the same fixture (exit 0), and a genuine record still passes — so the test distinguishes exactly the regression a future "align with box/cast" refactor could reintroduce. Burning the round to pin it was the right call; a gate whose correctness is incidental is the failure mode this whole PR argues against. CI is green on the new head.— automated review by
claude-bot-andresmgsl· heavy-duty-review-botVerdict: Approve — I agree with this as-is.
Re-review after intentional re-request at
b234f48(was77a9a1a). Script unchanged; only addition is the spaces/tabs-only body fixture.What holds
drill_sectionskips/^[[:space:]]*$/until real content; whole-version match; record-not-pass;-devvacuous; missing RUNS.md = no-record.test/release.sh:drill: a record body of only spaces and tabs FAILS— verified the tip script still refuses spaces/tabs-only body (exit 1) and accepts a one-line waiver. Prevents a later “simplify to match siblings” from reintroducing the box#149 / cast#138sed '/./,$!d'hole.CI green. No blockers.
Verdict: I agree with everything and have no additional feedback.
@danmt — round closed at
b234f48. Three head-current approvals, zero changes requested. Re-handing this off after I reopened it myself.Why this was reopened
I handed this to you at
77a9a1awith three approvals, then invalidated them on purpose. The question that prompted it: is the whitespace bypass that blocked box#149 and cast#138 also fixed here?It was never present here. This guard skips leading blanks with
/^[[:space:]]*$/, which matches a spaces-or-tabs line the same as a bare one; the siblings usedsed '/./,$!d', where.matches a space. Different implementation, not luck — that is why this PR drew three approvals in the round its twins drew three blocks.But nothing pinned it. The
emptyfixture used genuinely empty lines, so the correct behaviour was incidental — untested because unbroken. That is precisely the state in which a later "simplify this to match its siblings" reintroduces the hole silently, and the siblings are the ones that had it, so that refactor is plausible rather than hypothetical.What the added fixture proves
A record body of one spaces-only line and one tab-only line, expected to fail — and verified by mutation rather than by passing:
/^$/skip@claude-bot-andresmgsl re-ran that mutation independently rather than taking my word for it, and confirmed both directions.
test/release.sh: 132 → 133 passed, 0 failed.The family, now consistent by measurement
All three agree, and all three are now asserted rather than assumed.
Verification on
b234f48test/release.shtest/cli.shtest/labels-reconcile.sh.shdrill-recorded.shon this tree0.2.1-devis a development treeThe cost, stated plainly
This burned a review round on a PR that was ready to merge. I judged it worth it: the entire argument for this PR is that a gate must be pinned by tests rather than trusted, and shipping the one guard in the family whose correctness rested on nobody having touched it would have undercut that argument with its own implementation.
Only
test/release.shchanged since your first approval. The guard, the CI wiring,drill/RUNS.md, CONTRIBUTING and LABELS are byte-identical to what you already read.Expected consequence
#101 (
release: 0.3.0) goes red when this merges — correctly. It carriesblocker:drill-pendingand goes green when the drill record lands.