Coolify 4.1.2 validates every checkout-relative path on create and 422s
anything without a leading slash: `docker_compose_location` against
ValidationPatterns::FILE_PATH_PATTERN, `base_directory`/`publish_directory`
against DIRECTORY_PATH_PATTERN. cast passed all three through verbatim and
validated nothing about their shape — and docs/semantics.md taught the exact
value Coolify rejects (`compose_file: docker-compose.yaml`).
The 422 is a property of the manifest, not of the instance, so it is knowable
before a single API call. Refine at parse time, on every verb: a refusal that
names the fix, not a silent normalization — the manifest is the artifact under
review, so a bad value is fixed in the file, once.
Fixes the docs example and the fixtures that carried the broken value.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both of these were found by the same run — the genuinely-from-nothing apply that
#38 was also hiding in, against a box that shares its server with another project.
Neither is a bug in what apply DOES; both are bugs in what it leaves behind and
what it says.
#40 — cast removes the default environment it made Coolify create.
POST /projects hands a new project Coolify's OWN default environment, `production`.
#39 taught apply to create the environment its resources actually name, so a project
cast creates from nothing now ends up carrying two: ours, holding everything, and an
empty `production` that nothing will ever use. That is precisely the shape that makes
a box unreadable later, and we have the live example — on the box being migrated away
from, `production` is empty and everything runs in `staging`, and "the obvious guess
is the wrong one" is a note we had to write down for ourselves. Shipping more of those
is not neutrality.
This is the only delete cast performs, so it argues for itself against apply-never-
deletes: what that rule protects is things cast did not make, and this is a byproduct
of cast's own POST /projects seconds earlier, holding nothing and having never held
anything. Three conditions, jointly, or nothing is touched — cast created the project
in THIS run (never a project someone built by hand), the environment is EMPTY (asked
of Coolify via the details route, the only one that eager-loads resources — not
inferred from the first condition), and its name is NOT ours (an --environment
production keeps its production, since that is where everything is about to live).
Best-effort: a delete that fails is reported and never fails an apply that worked.
#41 — the multi-destination 400 says what to do, and the plan says what it assumed.
A create against a server with more than one destination that names none is rejected
with "Server has multiple destinations and you do not set destination_uuid." — a
message that names neither the remedy nor the file it goes in, arriving at the FIRST
create, after apply has already made the project and the environment.
cast cannot pre-flight it and that half is not fixable: 4.1.2 serves no destinations
API at all, and GET /servers/{uuid} does not carry them either, so a server's
destination COUNT is unknowable until a create has been attempted. The diagnosis is
what is fixable. The 400 is now answered with the failing resource, the server by the
name the operator wrote (not its UUID), the exact path the UUID goes in
(environments.<env>.projects.<org>/<repo>.destination_uuid), the create-time warning —
placement is repaired by delete + recreate, never by a later apply — and Coolify's own
words kept verbatim, so the next person's search still works.
And the assumption behind an undeclared destination is now on screen at the moment it
is made: `placement: server's default destination (none declared)`. This reverses a
judgment cast held explicitly ("a line on every diff that says nothing is how a report
stops being read" — the test it replaces). The line does not say nothing; it says which
network the next create lands on. It stays on a clean run that creates nothing, too,
because the trap is set for projects that are already built: the day their server gains
a second destination, every one of them that declared no destination stops being able
to create, and nothing will have warned them.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
POST /projects hands a new project Coolify's OWN default environment,
`production` — never ours. cast then created every resource with
`environment_name: <our --env>`, so the first apply against a project that
did not exist yet 404'd on its first resource ("Environment not found") and
left the project behind, created and empty.
Two comments in the source already asserted the behaviour as though it were
implemented (cli.ts:716, :808), and the README says it outright — the route
existed in the vendored 4.1.2 spec, cast just never called it. It went unseen
because every environment cast had touched until now was hand-built in a UI
and adopted, so it already existed under whatever name someone typed. The
genuinely-from-nothing apply is the one path nobody had run.
apply now reconciles project + environment once per run, before the first
create. Read-before-write: an environment that already exists is never written
to, so adoption is untouched and this cannot regress an apply that works today.
A 409 is read as "present" (the race between our read and our write).
Coolify's default environment is LEFT ALONE, per apply-never-deletes — deleting
it would be the first delete cast ever performs. An empty `production` beside
the environment everything lives in is reported, the same courtesy an orphan
gets, and removed by hand or not at all.
The regression test drives the real failure, not a call count: the fake Coolify
404s a create whose environment_name it does not carry, exactly as a live box
does — against the old executor it reproduces the reported error verbatim.
`cast inventory` could already see a whole instance (#22). It can now write
down what it sees, in the shape of cast's own inputs:
cast inventory --env prod --instance box-b --emit-draft ./draft
draft/
environments.yaml # bindings as far as they can be read — with the projects: registry (#25)
incubator/.infra/manifest.yaml # one per project
incubator/.infra/env/*.env.template
la-familia/.infra/manifest.yaml # …including the client sites nobody ever declared
secrets/<project>.<env>.env.age # encrypted to a recipient you name
UNCAPTURED.md # ← the important file
Two uses: bootstrapping a project that has no manifest (the third-party sites
on the box being drained were never declared, and never will be unless
something writes the first draft), and a point-in-time blueprint.
A DRAFT IS A PROPOSAL. It is never desired state, and `apply` never reads it:
sweep → emit draft → a human reads it → manifest PR → capture → apply
Same shape as `terraform import` → HCL, and the boundary is enforced, not
merely documented. It never emits into a repo that already has a manifest —
for a declared project the manifest IS the truth, and one regenerated from a
live box would let that box's accumulated cruft overwrite a reviewed spec, in
the one direction nobody reviews. Adoption is one-way. So: a non-empty target
refuses, a manifest at the path it would write refuses, and --emit-draft with
a repo positional refuses (that is the reconcile path, and it is exactly the
case where a draft must not be written).
Two things would make a draft actively dangerous, and both are the point:
1. COPIED PROVIDER-GENERATED VALUES. A DATABASE_URL read off the source points
at the SOURCE box's Postgres; rebuild elsewhere and the new box comes up
WORKING, reading and writing the old box's database, and you find out the
day the old box is deleted. So the draft applies capture's discipline: a
provider-generated name is placeheld with the same GENERATED_PLACEHOLDER
literal, its live value is written into no artifact, and the emitted
manifest declares it under generated_secrets: so a later capture placeholds
it again with no flag to remember. The rule is by NAME — Coolify's SERVICE_*
magic vars, and any name carrying a datastore word and a connection word —
and it errs wide, because over-matching a real secret is loud and
recoverable while under-matching a generated one is silent and is not.
Every other var becomes a ${REF} with its value in the age store, never a
literal in a committed file: cast cannot know which of a box's vars are
secret, and a live key written as a literal is a key in a git repo.
2. SILENT LOSSES. UNCAPTURED.md is a first-class output, emitted on every run:
per resource, every live setting cast saw and could not express —
destinations (#21), service hostnames, Basic Auth/Traefik labels, backup
schedules, database kinds cast does not model, env names a template cannot
hold — plus what no API in 4.1.2 will tell it, and the table of what a
blueprint still cannot restore (the GitHub App private key and the S3 keys:
re-create by hand). A blueprint that omits these without saying so is worse
than no blueprint, because in a disaster you would trust it and rebuild a
different box.
Secrets are encrypted to a recipient you NAME (--recipient, or the
environment's age_recipient binding). With neither, cast refuses rather than
quietly emitting a draft that looks complete and holds not one value;
--no-secrets says so deliberately. A project with resources in two populated
environments is a tie cast will not break — it refuses, and --environment says
which, as a tiebreak rather than a filter (filtering by name would drop the
client sites, each alone in Coolify's default `production`, out of a blueprint
that claims to describe the box).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every cast verb was single-project, so "do this to the whole instance" was a
shell loop the operator wrote from memory — and the project they forgot is the
one that drifted. `cast diff --env prod --all` and `cast apply --env prod --all`
iterate the registry (#25) instead.
The bulk of this is a refactor: the apply/diff block in cli.ts was one long
inline body, and it is now `runProject` — checkout → secrets → desired →
bindings → live → diff → optionally apply. Both the single-repo path and the
`--all` loop call it, so there is exactly ONE implementation of what a project
run is. A second, parallel fleet path is how the two would drift, and drift is
the subject of this tool. `openCoolify` and the team assert are hoisted out of
it: one --env means one instance and one team, so asserting once still lands
strictly before the FIRST project's first read — the read is already the lie.
Fails closed on the aggregate. A registered project cast cannot reach is an
ERROR, never a skip: the clone failing, no manifest block for this environment,
an absent or undecryptable store, an absent Coolify project/environment, any
HTTP error. A silently skipped project reads exactly like a clean one — #12/#18/
#22 at fleet scale — so the report leads with COVERAGE (registered / read /
clean / drifted / unreachable), and:
diff --all 0 every registered project was READ, and every one is clean
1 every one was read, and at least one has drift
2 a project could not be read — outranking drift, because an
unreadable project is not a diff result but the absence of one
apply --all 0 every registered project applied; non-zero otherwise
`diff --all` runs every project to completion (stopping hides the drift in the
projects it never reached); `apply --all` STOPS at the first failure and names
what it applied and what it did not touch (continuing to mutate a fleet after an
unexplained failure is not a thing cast gets to do).
Two refusals. An empty or absent registry refuses rather than printing
"0 projects, clean" — an empty fleet reading as a clean fleet is the whole
failure this is against; the message distinguishes an unmigrated state file from
a registry pointed elsewhere and prints the YAML to write. And `--all` is
mutually exclusive with the repo positional and with every single-project
coordinate (--path, --project, --environment, --resource, --hostname-overlay):
each names ONE project's checkout, ONE project's Coolify name, ONE box's
resource names, and `--project X` across a fleet would point every project at
the same Coolify project — a false report on diff, and on apply every manifest
in the fleet written into one project.
Also: `projectsIn`'s doc-comment guessed that `[]` made a fleet verb over an
unmigrated state file "a clean no-op rather than a crash". It is precisely
backwards, and now says so. And the --path/--env-prod refusal is hoisted to the
CLI's up-front flag validation (one rule, one string, two call sites in
resolve.ts) — it used to be caught only by accident of resolveCheckout running
before the bindings load.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
environments.yaml could say where things deploy to, and how a project you
have already named is placed once it is there. It could not say which
projects exist. "Every project" was a thing the operator remembered — so
fleet operations (#26) had nothing to iterate, and rebuild-from-state (#27)
was an assumption, since you cannot restore what you cannot enumerate.
A new optional top-level block, keyed by the full <org>/<repo> slug:
projects:
heavy-duty/incubator:
environments: [prod, staging]
The key IS the repo — no `repo:` field, because a second place to write the
same string is a second place for it to be wrong. No bare-<repo> fallback,
unlike github_apps and environments.<env>.projects: those carry one because
state files in the wild are keyed that way, and this block has none to
support. A bare <repo> is unique only within an org, which is why it is not
a key (#12, twice learned).
Validated in loadBindings, so every verb refuses a registry that lies:
- an environment no `environments:` block defines is an error — the project
would be registered into an environment no command can visit
- every environments.<env>.projects.<slug> binding must be registered for
that env, or the two blocks describe two different fleets: a destination
or smoke_target real enough for a direct apply, invisible to every fleet
run. Only enforced when `projects:` is present, so pre-registry state
files keep loading unchanged.
Both defend one failure: a silently skipped project reads exactly like a
clean one. Errors render multi-line now — zod's own .message is the issue
array as JSON, which flattened the refusals into a line of \n escapes.
projectsIn(bindings, env) gives an environment's slugs, sorted; [] with no
registry. The --all flag that consumes it is #26's, not here.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A destination is the Docker network a resource is created on. cast never sent
one, so everything landed on the server's default — invisible and harmless while
each server hosts one project, and neither the moment a server hosts two.
The state file had nowhere to say otherwise, either. A destination is scoped
project × environment, and `environments.<env>` is scoped by environment alone:
a `destination:` key there would mean "one network shared by every project in
this environment", which is the isolation it is meant to provide, inverted.
So:
- `environments.<env>.projects.<repo>` — per-project state, keyed by repo, full
`<org>/<repo>` slug first with a bare-`<repo>` fallback, exactly like
`github_apps`. It carries `destination_uuid` and `smoke_target`.
- `smoke_target` moves there. It was state-file-scoped: it named ONE project's
app (`core`) from a key that could not tell two projects apart — or even prod's
app from staging's. The old key is still read (with a warning), so an unmigrated
state file keeps smoking, and `cast smoke` now takes an optional `<org>/<repo>`.
- `apply` sends `destination_uuid` on create, for applications, databases and
services alike — Coolify runs identical destination logic in all three.
The API turns out to be worse than the issue assumed, in a way that changes what
"diff should compare the destination" can honestly mean. Verified against
coollabsio/coolify v4.1.2 (routes/api.php + the three Api controllers), and
written up in reference/README.md:
- There is NO destinations API. Zero routes. A destination cannot be listed, read
or resolved by name — only a raw UUID from the UI identifies one, exactly as
with `s3_destination`. Hence `destination_uuid:` and not `destination:`.
- The field is WRITE-ONLY. Coolify takes `destination_uuid` on write and returns
`destination_id` (an integer PK) on read, with nothing mapping between them.
- On a server with >1 destination, a create that OMITS it is a hard 400. So cast
could not deploy onto a shared box at all — it did not silently misplace there,
it simply failed. On a single-destination server the uuid is ignored entirely
and never validated, so a wrong one is invisible until a second one exists.
A declared UUID therefore cannot be verified against the resource it was sent
for — by cast or by anything else. Diffing it as a field would compare a UUID to
an int and report drift that never clears, so it is reported rather than compared,
and the limit is stated out loud: every diff that declares a destination says it
did not verify it. Silence would make an unverified setting read as a verified
one, which is the failure shape #12/#14/#17/#18 are all about.
What IS comparable is the live side to itself. `diff` groups live resources by the
`destination_id` Coolify does report, and a project whose resources do not all
share one network is drift — non-clean, both sides named, and never repaired
(apply moves nothing between networks). That catches the thing actually worth
catching, including on a box whose destinations were made by hand.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cast was scoped to the steady state: manifest → Coolify, forever. It had no
adoption path — no way to bootstrap the age store from an instance built by
hand, before any manifest existed. The operator did it by hand: curl the envs,
assemble 17 name=value pairs into /dev/shm/prod.env, age -r, shred. Every input
to that pipeline is something cast already has, so a human was shuffling cast's
own inputs through a terminal, with the leak (scrollback, history, a tmp file
that never got shredded) and the silent miss both live.
cast capture <org>/<repo> --env <env> [--generated N] [--override N] [--force]
The required set comes from the MANIFEST, not the box: the ${...} refs in that
environment's env templates, read by the same parser apply uses to demand them.
resolveTemplate and templateRefs now share one grammar — a drift between them
would mean capture collects a different set than apply later requires, which is
exactly the "a name silently missed" failure this verb exists to remove.
The mapping is deliberately NOT mechanical. A DATABASE_URL read off the source
points at the SOURCE box's Postgres: confidently wrong, entirely plausible, and
the target's real URL does not exist until Coolify creates the resource. So the
manifest declares `generated_secrets:` and those names are written as the
literal `pending-coolify-generated`. staging's ADMIN_EMAIL must be the operator,
not the source's — staging and prod share a Mailgun domain, so a staging box
carrying the real address can mail real users; that is --override.
A "capture everything" verb would be wrong in ~4 of 17 entries, silently —
worse than being wrong in all of them. So every name is forced into a
disposition, and two of the four stop the run: a name required by a template but
absent from the source REFUSES (an empty substitutes to nothing and the app
boots misconfigured), as does one name carrying different values on two
resources.
generated_secrets is a manifest property rather than a flag the operator must
remember, because the manifest is what knows DATABASE_URL comes from a database
it declares. An entry no template refers to is a hard error: a guard standing
over nothing reads like a guard, and the likeliest cause is a typo whose real
name is then captured from the source instead of placeheld.
Secret hygiene, all covered by tests asserting on real values:
- the plan prints names and provenance, NEVER values
- an --override's value comes from $CAST_CAPTURE_<NAME>, never argv (`ps`)
- plaintext is piped to age on stdin — never a temp file, stdout, or history
- an existing store is not overwritten without --force: it may hold the only
copy of values the source no longer has (apply's never-delete, applied here)
capture inherits diff's absent-target refusal (D-237) — against a project that
isn't there it would report every secret as missing, an alarming report about
the wrong box — plus the team assert and the --path/--env prod ban. The last
gate is a typed confirmation of the environment's name; there is no --yes.
The end-to-end test decrypts the store cast wrote and asserts on its contents,
so "exactly the names the manifest requires, no more and no fewer" is checked
against real ciphertext rather than against cast's own console output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Coolify API tokens are team-scoped, and a wrong-team token does not error:
the API resolves what it cannot see to `null` (getResourceByUuid walks
resource → environment → project → team_id and returns null on a mismatch).
To cast, `null` is indistinguishable from "this resource does not exist
yet" — an invitation to create it. So an apply with a token minted under the
wrong team would not fail loudly; it would provision a duplicate set of
resources into the wrong team, against whatever server that team owns.
Silent, mutating, discovered late. That makes this a correctness bug, not
hardening.
- environments.yaml carries a required `team:` per environment (id, name, or
both). Required is the point: an environment with no declared team is one
cast cannot verify it is pointed at.
- Every command that reaches a live Coolify (apply, diff, server add, smoke)
resolves GET /teams/current — the only endpoint that answers "what team
does this token act as?" — and aborts on mismatch before its first READ,
not merely its first write: a wrong-team diff reports "everything is
absent", which is the very lie an apply would then act on.
- server add and smoke take --env for this reason. A server belongs to
exactly one team forever (no pivot, no is_system_wide escape hatch), and
smoke writes env vars onto a live app.
- New read-only `cast team` prints the token's team, so the binding can be
filled in without a chicken-and-egg. With --env it also checks the
binding: the dry run for "would apply refuse?".
Team id 0 is a first-class value, not a falsy absent — it is the Root Team
that a single-admin instance keeps everything in (app/Models/User.php).
Also records the #4 investigation in docs/semantics.md: GithubApp
`is_system_wide` IS the supported way to serve every team — list_github_apps
scopes to `team_id = token's team OR is_system_wide`, and POST /github-apps
accepts the flag — so per-team App duplication is unnecessary. Corollary:
resolving a GitHub App by name is NOT a proxy for being in the right team,
which is the second reason the assert has to be explicit.
Closes#9
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This should have shipped with the extraction PR and did not: the file was
written after that commit was already pushed, so it landed nowhere. The
infra PR meanwhile removed the same content from that repo's README, so
between the two merges these notes existed only in git history.
They are the expensive part of the executor — apply's guarantees (never
deletes, never recreates a database, fails loudly on un-updatable drift),
the dockercompose build pack, hostname-overlay shapes, and each place
Coolify 4.1.2 misbehaves, with citations verified against coollabsio/coolify
v4.1.2 and the vendored OpenAPI rather than its published docs.
Linked from the README so it is found before someone changes apply.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
claude-hdb
