Third and final correction to the supersede rule, and the second one
that silently restored #136.
Round 3 replaced "date by completion" with "date by the newest stamp
the run carries". That is still not an ordering on runs: `max` over
[startedAt, createdAt, completedAt] resolves to completedAt for a
FINISHED run and startedAt for a LIVE one — different quantities, so
the comparison was never between like and like.
The consequence is the ordinary concurrency-group path, not an edge
case. A run cancelled by a concurrency group does not stop instantly;
it drains AFTER its replacement has already started, so
predecessor.completedAt > successor.startedAt is the normal shape. On
this PR's own aa5a6ba the window was 13 seconds. Inside it the dead
predecessor out-dated the live run replacing it, and a green
predecessor reported SUCCESS while a re-run was still in flight:
SUCCESS completing 15:30:13, replacement started 15:30:00
max -> SUCCESS (#136: needs-human over a disabled merge button)
first -> PENDING
Fixed with `max` -> `first`. The list is already in preference order,
so `first` IS "date it by when it began" — and a replacement always
begins after the run it replaces, whatever order they finish in. The
sentinel filtering and the undateable-sorts-last tiebreak are
unchanged; this narrows the rule to a quantity that actually orders.
Prescribed independently by claude-bot-andresmgsl and
codex-bot-andresmgsl.
Two fixtures pin the drain window in both colours. 49 -> 51.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round 3. @claude-bot-andresmgsl and @codex-bot-andresmgsl independently
caught that the supersede rule added in round 2 inverted itself on the
one shape its fixtures could not express: a re-run still in flight.
A running check does not omit `completedAt`. `gh` marshals the Go zero
time as the STRING "0001-01-01T00:00:00Z", and jq's `//` only falls
through null/false, so the sentinel won the sort key and sorted before
every real timestamp. The live re-run became the OLDEST entry in its
context, `last` discarded it, and the run it superseded was judged
instead — exactly backwards, and wrong in both directions:
green + re-run in flight -> SUCCESS (should be PENDING)
CANCELLED + re-run in flight -> FAILURE (should be PENDING)
The first is #136 restored by the very rule meant to close it: all bots
approve, mergeable, state:needs-human — over a tree whose merge button
branch protection has disabled. It was also a regression from round 1,
which caught it via `any(. == "")`. The second is the re-run flap the
supersede rule exists to prevent, narrowed rather than removed.
Fixed by taking the newest timestamp a run actually carries and
discarding BOTH spellings of absent — null and the zero sentinel —
rather than by reordering the fallbacks. An entry with no usable
timestamp now sorts LAST rather than first, so an undateable in-flight
run is never dropped in favour of a stale success. Every ambiguity
resolves toward "not settled".
The fixtures could not have caught this: the `run_()` helper sets only
`completedAt`, so every supersede fixture was a race between two
FINISHED runs. The helper now expresses an in-flight entry, and the
four new fixtures assert PENDING over both a green and a cancelled
predecessor. 44 -> 48; reverting just the dating expression fails 3 of
the 4.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Round 2. Two blockers, both real, both closing the same hole this PR
exists to close — a `state:needs-human` that invites a human to merge a
tree that will not merge.
The check-rollup classifier enumerated the outcomes that block and let
the rest fall into `else "SUCCESS"`, so ERROR, CANCELLED and STALE all
read as green. Inverted: it now lists the outcomes that DON'T block —
SUCCESS, NEUTRAL, SKIPPED, plus the pending set — and treats everything
else as blocking. The direction is the point. The rollup mixes two
closed enums (CheckRun.conclusion, StatusContext.state) and an outcome
the list forgets is one we cannot certify as mergeable; the costs are
not symmetric, since a false FAILURE parks the PR on the agent who
looks, while a false SUCCESS is #136 exactly.
Once CANCELLED blocks, superseded runs must be dropped first: a re-run
does not evict the run it replaced, and this PR's own tip carries a
CANCELLED `scope` beside the SUCCESS `scope` that superseded it. Each
context now collapses to its newest entry before anything is judged,
keyed on workflow + job name because a bare job name is only unique
within its workflow. That preserves the re-run case the panel split
over while still blocking a cancelled run that is the newest word.
The classifier also moved out of main() into checks_state(). That is
why no fixture caught this: it was inline in the fetch loop, so the
fixtures could only inject CHECKS= as an already-decided string.
Second, decide_state() returned from inside the bot loop on the first
MISSING, so a STALE belonging to a later bot in BOTS was never read —
a round that was both unfinished and staled came out needs-human over
a head nobody had reviewed. The whole round is now collected before
precedence is applied to it as a unit, STALE before MISSING.
test/labels-reconcile.sh: 29 -> 44 fixtures, pinning the check-outcome
enum, the supersede rule (both orders, plus same name in another
workflow), and the mixed round at both ends of BOTS. All verified
non-vacuous against the round-1 code.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
decide_state() derived state from three inputs -- draft flag, requested
reviewers, submitted reviews -- and read NOTHING about mergeability or checks.
Combined with the `if requested "$HUMAN"` short-circuit at the top of its
precedence, the label was sticky: once the maintainer was requested, the PR
read state:needs-human through conflicts, through red CI, through a
force-push that staled every approval. Nothing demoted it.
Observed twice in one afternoon, in two different shapes. Three PRs sat at
state:needs-human while CONFLICTING for hours -- the board inviting a merge
GitHub had already disabled. And #119, after a rebase, read MERGEABLE, four
green checks, state:needs-human, with ZERO reviews bound to its head: every
visible signal saying "merge me" over a tree no reviewer had seen. That second
shape is the dangerous one, because unlike a conflict nothing on the page
contradicts it.
The rule the label now keeps: state:needs-human means a human could merge this
RIGHT NOW, so anything making that false outranks the request that put it
there.
CONFLICTING or failing checks -> state:needs-rebase (new; the agent's to fix)
approvals staled by a push -> state:addressing (nobody reviewed this tree)
An UNFINISHED round still yields to an explicit human request -- a maintainer
pulling a PR to themselves early is deliberate, and MISSING (nobody has
reviewed yet) is a different fact from STALE (everyone reviewed something
else). That distinction is why the two are handled in different arms rather
than collapsed.
UNKNOWN mergeability is deliberately NOT treated as unmergeable: GitHub
reports it for about a minute after every merge while it recomputes, and
flapping every open PR through needs-rebase on each merge would be worse than
the bug being fixed. A failed read of either fact degrades to the same "do not
know" value, for the same reason -- an API hiccup must not relabel the board.
Also adds merge-next, because a correct needs-human still does not say WHICH
PR to merge first, and order matters when they conflict through CHANGELOG.md.
Queue order is intent, so the reconciler never sets it; it only CLEARS it the
moment the PR stops being mergeable-by-a-human -- precisely the staleness that
made needs-human untrustworthy.
Both live shapes are pinned in test/labels-reconcile.sh (19 -> 29 fixtures),
including that UNKNOWN does not trigger needs-rebase and that a draft outranks
a conflict. Proven non-vacuous: dropping the mergeability arm fails 4
assertions, dropping the STALE precedence fails 2, restoring returns 29/0.
Closes#136
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
codex's late #85/#98 round-3 finding, valid post-merge: the needs-human
auto-request fired only when the human had NEVER reviewed, so any earlier
human comment or stale approval left a fully-approved PR labeled
needs-human with nobody actually requested — a wedged handoff.
human_request_needed() now asks whether a fresh head-current human review
is missing (live request or head-current approval → nothing to ask;
anything else → request). Five new fixtures cover the wedge, the stale
approval, the satisfied handoff, and request suppression (19 total).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Maintainer direction: body-parsing agreement was a guess, and the machine
must not guess. COMMENTED is now unconditionally a non-verdict; the judgment
that a comment-only reviewer's round passed belongs to the PR AUTHOR, who
escalates by requesting the human's review — an explicit request is a fact,
and it is the machine's top-precedence input. Auto-request survives only for
the no-judgment case: three formal head-current approvals. CONTRIBUTING and
LABELS.md state the handoff; fixtures updated (14 transitions, including
author-escalation and the three-formal-approvals path).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Round-1 blockers, all three reviewers concurring:
- COMMENTED agreement now counts: agreement_signal recognizes the live bots'
durable markers (Verdict: Approve / I agree with everything / leading ✅) —
the gate to needs-human can actually close. Formal verdicts remain the
contract (CONTRIBUTING), this is the documented transitional workaround.
- Every counting verdict is bound to the head SHA; a stale approval parks the
PR in addressing (agent owes re-request) instead of promoting unreviewed
code. CHANGES_REQUESTED blocks at any head, per GitHub's own semantic.
- reconcile serializes under ONE job-level concurrency group; scope stays
per-PR. No more cron-vs-event race on the request-the-human-once guard.
- Sweep resilience: per-PR subshell (one failure logs and continues), label
edits warn instead of wedging; the self-heal claim now matches reality
(dispatch-only bootstrap).
- The state machine is extracted pure (globals in, state out) and sourceable:
test/labels-reconcile.sh proves 14 fixture transitions — comment-only
agreement, stale approval, comment-without-verdict, human precedence and
human-block — wired into CI.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The machinery LABELS.md promised. labels.yml runs the reconciler on a
15-minute cron plus PR events (pull_request_target — every PR here is from a
fork, where pull_request gets a read-only token; no PR code is ever checked
out). The script derives each open PR's state:* from GitHub's own facts and
converges labels statelessly; stale is judged from real activity (commits,
comments, reviews), never label churn, so the sweep cannot un-stale its own
mark. actions/labeler applies scope:* from changed paths. CONTRIBUTING.md is
the guideline: the PR loop, and who sets which labels. Rehearsed with
DRY_RUN=1 against the live repo; shellcheck-clean.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
