claude-bot-andresmgsl/ceremony
1
0
Fork 0
forked from heavy-duty/ceremony
Code Pull requests Activity

test(forge): assert the distinguishing text, not a surviving substring

Browse source 
@codex-reviewer-andresmgsl (#4727) and @grok-reviewer-andresmgsl (#4734):
"...and the refusal names both totals" searched only for "4", so it stayed
green if the later total vanished from the message. A case named "names
BOTH" must fail when one goes. Now asserts "4 then 9".

Auditing this file's siblings for the same shape found a second, older
instance: "the refusal names the client" searched for "gh", which also
occurs in the explanatory prose ("gh speaks GitHub's /api/v3..."), so it
would have passed even if the client name never reached the message. Now
asserts "the 'gh' client cannot speak it".

Both verified by mutation: removing the second total, and removing the
interpolated client name, each red exactly their own case.

Refs #188
This commit is contained in:
cluade-reviewer-andresmgsl 2026-08-02 19:16:48 +00:00
parent 714a2e0413
commit adf3299192
2 changed files with 11 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
test/forge-backends.test.sh
View file

@ -177,7 +177,11 @@ check "a negative total is refused" 1 "not a non-negative integer" \
fake_forge '4,9' '[{"number":1},{"number":2}]' '[{"number":3},{"number":4}]' fake_forge '4,9' '[{"number":1},{"number":2}]' '[{"number":3},{"number":4}]'
check "a total that changes between pages is refused" 1 "changed between pages" \ check "a total that changes between pages is refused" 1 "changed between pages" \
forge_api --paginate 'repos/o/r/issues' --jq '.[].number' forge_api --paginate 'repos/o/r/issues' --jq '.[].number'
check "...and the refusal names both totals" 1 "4" \ # The distinguishing text, not a substring that survives losing half the
# message: "4" alone stayed green if the later total vanished, which is what
# @codex-reviewer-andresmgsl (#4727) and @grok-reviewer-andresmgsl (#4734)
# both caught. A test named "names BOTH totals" must fail when one goes.
check "...and the refusal names both totals" 1 "4 then 9" \
forge_api --paginate 'repos/o/r/issues' --jq '.[].number' forge_api --paginate 'repos/o/r/issues' --jq '.[].number'
# A 200 whose body is not a collection. `length` on a non-array counted 0, # A 200 whose body is not a collection. `length` on a non-array counted 0,

7
test/forge.test.sh
View file

@ -102,7 +102,12 @@ check "forgejo + gh-only client refuses" 1 "cannot speak" \
preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh
check "the refusal names the forge" 1 "forgejo" \ check "the refusal names the forge" 1 "forgejo" \
preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh
check "the refusal names the client" 1 "gh" \ # The interpolated client, not the bare string "gh" — which also appears in
# the explanatory prose ("gh speaks GitHub's /api/v3…"), so the old assertion
# stayed green even if the client name never reached the message. Same class
# as the "names both totals" weakness the panel caught in the backend suite
# (#4727 / #4734); found by auditing this file for the same shape.
check "the refusal names the client" 1 "the 'gh' client cannot speak it" \
preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh preflight_in CEREMONY_FORGE=forgejo CEREMONY_FORGE_CLIENT=gh
# The refusal must be actionable, not merely loud: #188's whole cost was a # The refusal must be actionable, not merely loud: #188's whole cost was a
# red check that told nobody what to do. # red check that told nobody what to do.