The suite allocated temp dirs at 68 sites across 21 files and removed none,
accumulating ~6700 directories and 189MB per machine-day, some holding age
keys. All 68 now go through a single `tmp()` helper allocating inside a
per-run root that vitest's globalSetup teardown removes wholesale, and a
class-guard test fails if `mkdtempSync` appears under test/ outside the
helpers.
The per-worker `process.once("exit")` reaper that suggests itself here does
not work under vitest and fails silently: the pool recycles workers by
killing them, so exit handlers registered in a test file never run. Measured
— a probe test writing from an exit hook produced no file, and a full run
with per-worker hooks still left 750 directories. globalSetup's teardown runs
in the main process, after every worker, and vitest awaits it.
Separately, and contrary to #117's framing that "cast itself does not leak":
resolveCheckout() mkdtemps an `infra-checkout-` dir, clones the infra repo
into it, and never removes it, so every `cast apply`/`diff`/`capture` without
--path leaked a full clone. The box that reported #117 was holding 602 such
directories, 73MB of real .git trees, from the same day. The leak fires on
the failure path too, since the dir is created before the clone runs.
Ephemeral checkouts are now reaped on process exit — the lifetime that fits,
since callers read the tree after resolveCheckout returns; a --path checkout
is the operator's own tree and is never registered.
Empirical: /tmp/cast-* + /tmp/infra-* count is 0 before and 0 after a full
`npm test`, against 750 with the exit-hook design. 626 tests green.
Refs #117
UNCAPTURED.md has said since it existed that Basic Auth is "carried as raw
container labels. cast's manifest has no field for them, so a rebuilt resource
is UNPROTECTED where the original was not." For applications that is a cast
vocabulary gap, not a Coolify one: is_http_basic_auth_enabled,
http_basic_auth_username and http_basic_auth_password are in both the create
and the PATCH allowlists at v4.1.2 (ApplicationsController.php:914, :2368).
An application now declares `basic_auth: { enabled, username, password }`, with
the password a store ${REF} and only a ${REF} — the schema refuses a literal,
because a manifest is a committed file. It resolves out of the environment's
age store through the same mechanism every env-template ref uses, and a missing
or empty entry fails before anything is written.
Managing it is opt-in (the is_static rule): an unconditional `false` would have
the first apply after this ships strip protection off every app enabled by hand
in the UI. Enabling without both credentials is refused at parse time and again
at the wire — Coolify's own rule (:2446-2463), enforced before the request
rather than discovered as a mid-run 422.
The read side is fail-honest. The toggle and username are plain columns and are
compared, so a UI flip is caught. The password is gated behind a
sensitive-data-enabled token at 4.1.2 and read:sensitive on v4.2, and would have
to be printed as a field diff, so it is never projected into the comparison
vocabulary on any box — every diff of an app declaring basic_auth says the
password was NOT compared, in the backup schedule's voice: reported, not drift.
custom_labels stays deliberately unwired: enabling basic auth or changing
domains regenerates labels and overwrites it unless
is_container_label_readonly_enabled, which is not API-settable until v4.2.
The NO_API_COVERAGE row narrows to services, where it is a real API gap on both
releases, plus a separate row for custom_labels on applications.
Closes#76
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Services could not carry hostnames through cast: `desiredFromManifest`
dropped a service's `domains` and warned they were a manual Coolify UI act,
citing a re-checked "no flat `domains` on a 4.1.2 service, on any route."
The audit (#72) disproved that — the same failure mode #51 corrected for
backup schedules. The FLAT shape genuinely has no route; the per-container
CAPABILITY was there at 4.1.2 all along.
`POST /services` and `PATCH /services/{uuid}` both take a `urls` list
([{name, url}], url comma-joined) that `applyServiceUrls` matches to a
`ServiceApplication` by name and stores as its `fqdn`; `GET /services/{uuid}`
loads `applications` and returns each `fqdn` (verified against
ServicesController v4.1.2). So services now speak the SAME per-container
vocabulary a dockercompose app does:
- **Manifest:** `service_domains: { <container>: [url] }` replaces the flat,
unhonorable `domains` on a service (a flat list cannot name which container
a hostname belongs to — exactly what `urls` requires). Canonicalized (keys
and each URL array sorted) so container order never false-drifts.
- **Write:** `serviceApiFields` builds `urls` on create and update.
- **Read/diff:** a supplementary `GET /services/{uuid}` per service
(`attachServiceDomains`, gated to `diff`/`apply` like backups) projects
`applications[].fqdn` back into `service_domains`, so a declared hostname is
compared every run — no perpetual drift, no manual UI step.
- **Pre-flight:** a service create's `service_domains` joins
`desiredDomainsOfCreate`, the more important because a service create whose
domain conflicts is DELETED server-side before the 409 (rollback).
Two limits stated out loud: the read is fail-closed (an unreachable/
unrecognized `GET /services/{uuid}` aborts rather than projecting empty and
re-PATCHing forever), and `inventory --emit-draft` does not yet make the
per-service GET, so a drafted service's hostnames are still declared by hand
(same as backups) — draft/semantics say so.
`npm run check` clean · 514 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A public base URL an app reads (LANDING_BASE_URL, ADMIN_WEB_BASE_URL) is a
fact the manifest already states in `domains`/`service_domains` — the same
fields cast parses to reconcile Coolify domains. Hand-transcribing it into an
env template is a second copy that drifts (incubator's prod LANDING_BASE_URL
silently kept a pre-apex host). So a template can now say it directly:
LANDING_BASE_URL=${domain:landing}
ADMIN_WEB_BASE_URL=${domain:core.admin}
- ${domain:<app>} -> applications.<app>.domains[0]
- ${domain:<app>.<service>} -> applications.<app>.service_domains.<service>[0]
Symmetric with ${resource:...} (#60) — parse -> sentinel -> validate -> fill —
but a domain is PURE MANIFEST DATA, known at plan time, so it resolves fully in
desiredFromManifest against a map built from the manifest: no live read, no
executor deferral, no unresolved-at-write path. Domains are PUBLIC, so they
resolve to secret:false (printed in diffs) and read as plain literals
downstream (no diff.ts change). Not secrets: excluded from templateRefs, never
captured. assertDomainRefs is the single validation gate (apply/diff/capture),
refusing an undeclared app/service, a wrong-shape ref, or an empty/blank domain
list before the sentinel can escape. Applications only (Coolify 4.1.2 can't set
service domains). REPORTING_TZ-style operator literals stay literal.
Closes#66.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a ${resource:<name>.url} env-template ref that resolves to the internal URL
of a database the same manifest declares, read back from the live resource's
internal_db_url — never stored in the age store, never decrypted, never printed.
This deletes the two-pass generated-secret bootstrap for a database's own URL
rather than automating it: no placeholder, no stored copy to drift or overwrite,
and a rotated password is simply followed on the next apply.
Resolution runs in one function (fillDerivedEnv) against two URL maps: at diff
time against databases already on the box (so a matching app shows no drift —
killing the "secret DATABASE_URL differs" noise that ran on every plan), and in
the executor at apply time against a database created earlier in the same run
(the from-nothing case; apply acts databases-before-applications, #45). The
unresolved sentinel is never written — the executor refuses, rather than write a
blank that boots the app pointed at nothing, and re-running once the database is
up resolves it as an ordinary update.
A ${resource:X.url} naming a database the manifest does not declare, or an
attribute other than .url, is a hard plan-time error refused by every verb that
opens a template (apply, diff, capture). generated_secrets and the two-pass
bootstrap remain for the residual class — a provider-generated value that
genuinely is not derivable.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
apply created applications but dropped install_command, build_command, and
is_static — settings the manifest had no field for — so a static site in an
npm-workspace monorepo (landing) was built and RUN from the repo-root
package.json, booting the core API server, which crash-looped on a missing
DATABASE_URL.
The build block gains install_command / build_command / start_command
(free-form strings) and static (-> Coolify is_static). apply writes and diffs
them; draft emits them (they left its NO_HOME list, and is_static was never in
it — the silent loss that caused the crash), and only emits static alongside a
publish_directory so a draft always loads.
Managing is_static is opt-in: declaring `static:` is required to serve a static
app, and NOT emitting is_static by default avoids the first apply PATCHing
static serving OFF on an un-migrated app (or fighting a pack:static coupling
forever). static:true with no publish_directory, and any of the four on a
dockercompose app, are parse-time refusals.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Backup schedules were write-only, filed under "known limitations" on the
claim that "live Coolify state doesn't expose it back". The parenthesis was
load-bearing and false: a schedule is not on the database's own GET, but it
was never meant to be — it has its own route, GET /databases/{uuid}/backups,
which cast had been POSTing to all along and had simply never read.
The cost was exact. A database created before its `backup:` block was
declared never got one (apply set the schedule only inside the create
branch); a schedule deleted in the UI was invisible; and the `--full` diff
that gates a production cutover passed with an unbacked-up production
database.
Shape settled from the source rather than the vendored spec, which documents
the body as "Content is very complex. Will be implemented later.":
DatabasesController@database_backup_details_uuid (v4.1.2) returns a raw
Eloquent collection — a JSON array of ScheduledDatabaseBackup rows, columns
per $fillable (uuid, enabled, frequency,
database_backup_retention_amount_locally). `frequency` round-trips verbatim:
the controller validates it and stores $request->only(...) unchanged, with no
mutator on the model. The "diffing it would flag spurious drift" fear was a
guess about a read nobody had performed.
- `backup` becomes a diffed field like any other (resolve.ts), replacing the
side channel that carried it around the diff.
- The live side reads the route (coolify.ts, fetchLive), and apply sets the
schedule on UPDATE as well as create — POST or PATCH, decided by a read.
- A disabled schedule is a row that backs nothing up: neither clean nor
absent. cast diffs it and re-enables it.
Degrades honestly, since no live box was probed: an unreachable or
unrecognized response can only ever produce "declared, NOT compared — verify
in the Coolify UI", never invented drift and never a clean bill on an
unread database. On the write side the same failure raises rather than
guessing — POSTing blind would duplicate a schedule that may already exist.
Coolify injects SOURCE_COMMIT and the COOLIFY_* family into an application's
runtime environment itself, and SKIPS its own injection of a name the resource
already carries a var of (ApplicationDeploymentJob.php v4.1.2, line 2994 —
`->where('key', 'SOURCE_COMMIT')->isEmpty()`). A resource-level var of that
name therefore SUPPRESSES the platform's value. An empty one suppresses it
just as completely: presence, not value.
And it fails green — the deploy succeeds, health checks pass, and the only
symptom is /version reporting "unknown", the endpoint a production cutover is
gated on (D-266).
The rule is now a property of cast, not of one code path. A new src/reserved.ts
owns it, and every place cast touches an env var honors it:
- resolve — every manifest read (desiredFromManifest, requiredSecrets,
manifestResources) refuses a template declaring a reserved name, before any
write. So apply, diff, capture and inventory all refuse identically.
- draft — a reserved name read off a live box gets its own provenance,
`suppressed`: out of the template, out of the age store, its live value read
into no artifact, and named in UNCAPTURED.md with the consequence.
- diff — promoted out of the remove-candidate orphan list ("apply never removes
these; read them by eye") and printed as a FINDING with its consequence. Not
clean. apply still never deletes: cast reports, the human removes it.
- capture (classify) and cli (syncEnv) carry the same assertion at the file and
at the wire — unreachable through the CLI today, and kept because the
invariant is "cast never writes one", not "the CLI happens to check first".
- smoke writes an env var too; its probe names are asserted outside the space.
The rule lives in cast's code, NOT beside forbidden_var_patterns in private
state: that one is policy an environment may set for itself, this one is a fact
about Coolify, true on every box — nothing a manifest change could lower.
19 tests in test/reserved.test.ts, one per path.
Closes#50.
Coolify 4.1.2 gates the SOURCE_COMMIT *build arg* behind a per-application
setting (ApplicationSetting.include_source_commit_in_build, default false)
that has no API surface: it appears in zero API controllers, and both the
create (l.914) and PATCH (l.2368) allowlists in ApplicationsController.php
reject unrecognized keys outright ("This field is not allowed."), so it
cannot be smuggled through — sending it would 422 the whole request. Its
only writer in v4.1.2 is the Livewire Advanced tab (Advanced.php:128), i.e.
a human in the UI.
So apply says it out loud, once per dockercompose application, via the same
desiredFromManifest mechanism and in the same voice as the existing umami
service-domains warning. A manual step the tool knows about and does not
mention is a manual step that gets forgotten — and this one fails green.
Note the toggle gates the BUILD-time arg only; Coolify's runtime injection
of SOURCE_COMMIT is unconditional (ApplicationDeploymentJob.php:2949), so
a service reading process.env.SOURCE_COMMIT per request never needed it.
The warning therefore does not repeat #46's original (incorrect) premise
that this toggle is why /version reported sha "unknown" — the real cause is
an app-level env var suppressing Coolify's own injection (#50).
Closes#46.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Every cast verb was single-project, so "do this to the whole instance" was a
shell loop the operator wrote from memory — and the project they forgot is the
one that drifted. `cast diff --env prod --all` and `cast apply --env prod --all`
iterate the registry (#25) instead.
The bulk of this is a refactor: the apply/diff block in cli.ts was one long
inline body, and it is now `runProject` — checkout → secrets → desired →
bindings → live → diff → optionally apply. Both the single-repo path and the
`--all` loop call it, so there is exactly ONE implementation of what a project
run is. A second, parallel fleet path is how the two would drift, and drift is
the subject of this tool. `openCoolify` and the team assert are hoisted out of
it: one --env means one instance and one team, so asserting once still lands
strictly before the FIRST project's first read — the read is already the lie.
Fails closed on the aggregate. A registered project cast cannot reach is an
ERROR, never a skip: the clone failing, no manifest block for this environment,
an absent or undecryptable store, an absent Coolify project/environment, any
HTTP error. A silently skipped project reads exactly like a clean one — #12/#18/
#22 at fleet scale — so the report leads with COVERAGE (registered / read /
clean / drifted / unreachable), and:
diff --all 0 every registered project was READ, and every one is clean
1 every one was read, and at least one has drift
2 a project could not be read — outranking drift, because an
unreadable project is not a diff result but the absence of one
apply --all 0 every registered project applied; non-zero otherwise
`diff --all` runs every project to completion (stopping hides the drift in the
projects it never reached); `apply --all` STOPS at the first failure and names
what it applied and what it did not touch (continuing to mutate a fleet after an
unexplained failure is not a thing cast gets to do).
Two refusals. An empty or absent registry refuses rather than printing
"0 projects, clean" — an empty fleet reading as a clean fleet is the whole
failure this is against; the message distinguishes an unmigrated state file from
a registry pointed elsewhere and prints the YAML to write. And `--all` is
mutually exclusive with the repo positional and with every single-project
coordinate (--path, --project, --environment, --resource, --hostname-overlay):
each names ONE project's checkout, ONE project's Coolify name, ONE box's
resource names, and `--project X` across a fleet would point every project at
the same Coolify project — a false report on diff, and on apply every manifest
in the fleet written into one project.
Also: `projectsIn`'s doc-comment guessed that `[]` made a fleet verb over an
unmigrated state file "a clean no-op rather than a crash". It is precisely
backwards, and now says so. And the --path/--env-prod refusal is hoisted to the
CLI's up-front flag validation (one rule, one string, two call sites in
resolve.ts) — it used to be caught only by accident of resolveCheckout running
before the bindings load.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Three fixes at one seam: cast could not READ a box it did not build.
#17 — the environment had no read-side coordinate.
`--project` exists because a hand-built project is called whatever someone
typed. The environment has the identical problem and had no flag, so reading
a legacy box forced a choice between mutating that box's UI and renaming OUR
environment to match it. The second is what happened: `prod` became
`production` across the manifest and environments.yaml — a box being deleted
next week naming the environment of the box that replaces it, permanently
(apply creates the environment from --env), moving the store to
incubator.production.env.age and invalidating every runbook. Reverted.
`--environment` is now the coordinate. `--env` stays OURS: manifest block,
binding, age key, store path, team assert. `--environment` is theirs, on the
wire, and nothing else.
#18 — an absent RESOURCE reported as N missing secrets.
The D-237 lie, one level deeper. A resource that is absent reads back exactly
like one present with no env vars, so capture reported all 15 required names
as individually MISSING — from a box that was serving production and sending
mail at that moment — and offered --override as the remedy. Taking that offer
would have "worked": a valid store, hand-carried values, and the real finding
(the manifest and the box disagree about what the app is called) buried.
capture now refuses on the resource, names what does exist, and only reports
per-name MISSING for resources it actually found — where it means what it says.
#19 — cast inventory: see the box before you adopt it.
The missing first step. cast could describe a box it built, change one, and
take values off one for names a manifest declares — but not tell you what is
on a box you did not build, which is the first thing adoption needs. Every
mismatch above surfaced as a refusal from a verb already committed to a course
of action, and the tempting fix for two of them was to bend the manifest toward
the legacy box.
inventory reads resources and env var KEYS (never values), sorts them into
on-both / manifest-only / box-only, and needs no store, no age key and no
recipient — it runs before adoption exists. Its output is a document:
inventory → human reads → manifest PR → capture → apply
That boundary is what lets capture stay strict. inventory may read everything,
because a person reads its output. capture may only write what the manifest
declares, because `apply` reads its output. Same box, two consumers, two
contracts. A manifest-draft emitter is deliberately NOT included: it would be
one `cp` away from becoming desired state, which is the failure this design
exists to prevent.
Zero drift against a hand-built box is reported as suspicious, not as a pass.
npm run check + build clean; 164 tests passing, 18 files (was 151/16).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
cast was scoped to the steady state: manifest → Coolify, forever. It had no
adoption path — no way to bootstrap the age store from an instance built by
hand, before any manifest existed. The operator did it by hand: curl the envs,
assemble 17 name=value pairs into /dev/shm/prod.env, age -r, shred. Every input
to that pipeline is something cast already has, so a human was shuffling cast's
own inputs through a terminal, with the leak (scrollback, history, a tmp file
that never got shredded) and the silent miss both live.
cast capture <org>/<repo> --env <env> [--generated N] [--override N] [--force]
The required set comes from the MANIFEST, not the box: the ${...} refs in that
environment's env templates, read by the same parser apply uses to demand them.
resolveTemplate and templateRefs now share one grammar — a drift between them
would mean capture collects a different set than apply later requires, which is
exactly the "a name silently missed" failure this verb exists to remove.
The mapping is deliberately NOT mechanical. A DATABASE_URL read off the source
points at the SOURCE box's Postgres: confidently wrong, entirely plausible, and
the target's real URL does not exist until Coolify creates the resource. So the
manifest declares `generated_secrets:` and those names are written as the
literal `pending-coolify-generated`. staging's ADMIN_EMAIL must be the operator,
not the source's — staging and prod share a Mailgun domain, so a staging box
carrying the real address can mail real users; that is --override.
A "capture everything" verb would be wrong in ~4 of 17 entries, silently —
worse than being wrong in all of them. So every name is forced into a
disposition, and two of the four stop the run: a name required by a template but
absent from the source REFUSES (an empty substitutes to nothing and the app
boots misconfigured), as does one name carrying different values on two
resources.
generated_secrets is a manifest property rather than a flag the operator must
remember, because the manifest is what knows DATABASE_URL comes from a database
it declares. An entry no template refers to is a hard error: a guard standing
over nothing reads like a guard, and the likeliest cause is a typo whose real
name is then captured from the source instead of placeheld.
Secret hygiene, all covered by tests asserting on real values:
- the plan prints names and provenance, NEVER values
- an --override's value comes from $CAST_CAPTURE_<NAME>, never argv (`ps`)
- plaintext is piped to age on stdin — never a temp file, stdout, or history
- an existing store is not overwritten without --force: it may hold the only
copy of values the source no longer has (apply's never-delete, applied here)
capture inherits diff's absent-target refusal (D-237) — against a project that
isn't there it would report every secret as missing, an alarming report about
the wrong box — plus the team assert and the --path/--env prod ban. The last
gate is a typed confirmation of the environment's name; there is no --yes.
The end-to-end test decrypts the store cast wrote and asserts on its contents,
so "exactly the names the manifest requires, no more and no fewer" is checked
against real ciphertext rather than against cast's own console output.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
resolveCheckout shelled out to a bare `git clone` and relied entirely on the
ambient credential helper. On a workstation with none configured, git falls
through to its interactive username/password prompt — which GitHub no longer
accepts — and the resulting error talks about *the repository* rather than
about cast's missing credentials. Being logged into `gh` does not help:
`gh auth login` alone does not wire git's helper (that is `gh auth setup-git`,
a separate act most people never run).
Not routable around for prod: resolveCheckout refuses --path with --env prod,
so the clone is the only path and its auth is mandatory.
cast now resolves credentials itself, in order: `gh` borrowed as a
per-invocation credential helper (no mutation of the user's global git
config), then GITHUB_TOKEN / GH_TOKEN, then the ambient helper. The token is
never embedded in the clone URL or in http.extraheader — both leak it into
`ps`, and the latter persists it into the clone's git config. The helper
reads it from the environment at run time, so what lands in argv is the
literal text `$CAST_GIT_TOKEN`, never its value.
GIT_TERMINAL_PROMPT=0 on every path: whichever credential was used, git may
never fall through to a prompt it cannot satisfy — it can only hang, or hide
the real fault. When there were no credentials at all, cast now says so, and
names the fix.
Note that the empty `credential.helper=` reset clears URL-scoped helpers
(`credential.https://github.com.helper`, what `gh auth setup-git` writes) as
well as generic ones — verified against a live private clone, along with all
three acceptance criteria.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Public tool, private state. cast holds no hostnames, no bindings, no
secrets: it joins a product repo's .infra/ manifest with a state directory
you point it at, and makes Coolify match.
Extracted from heavy-duty/infra, which was half tool and half state — the
inconsistency that made it impossible to say whether "infra" named a CLI
or a runbook. rig builds the boxes; cast fills them; infra is what they
are filled with.
Two changes were required to make it genuinely stateless and publishable:
- The implicit cwd contract (environments.yaml / secrets/ / .coolify.env
resolved against the working directory, silently reading the wrong file
from the wrong place) is now an explicit --state <dir> / $CAST_STATE.
- BANNED_IN_PROD — a hardcoded list of one product's ALLOW_* flags, the
only product knowledge in the executor — becomes the generic, operator-
owned environments.<env>.forbidden_var_patterns. The guard now lives in
private state, so a product-side change cannot lower its own guard, and
it is a pattern rather than a list, so it catches unforeseen siblings.
Age identities resolve as $CAST_AGE_KEY_FILE_<ENV> then
~/.config/cast/age-<env>.key — which is the entire attended-vs-unattended
apply mechanism, with no environment names known to the tool.
Instance identity (org names, the GitHub App name, founder domains) is out
of the fixtures and out of register-github-app.sh, which took APP_NAME and
ORG as arguments rather than baking them in.
69 tests green; bin/cast + curl installer mirror rig's shape.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
