--emit-draft still told every reader that backup schedules "are not
exposed by Coolify's API" — the exact pre-#51 claim that issue disproved:
GET /databases/{uuid}/backups is a route, and diff/apply have read it on
every run since. The draft path was never brought along, so it warned
instead of reading, and a rebuild from a draft came up with no backups.
Now the draft loop makes the same supplementary per-database GET
(databaseBackupSchedules) for every DRAFTED database and databaseSpec
emits a real backup: { frequency, retention } block for the one shape
the manifest can express — a single, enabled schedule. Ungated on
purpose: fetchLive's opts.backups gate exists because the read-side
sweeps never look at the answer, and the draft is the sweep that does.
The read stays sequential (like the existing per-resource env GETs) and
a failed read degrades to an UNCAPTURED entry per resource rather than
aborting the whole-instance sweep — a draft's reader is a human, not an
apply about to write.
UNCAPTURED keeps only what the route genuinely cannot answer:
- the S3 target: save_s3 now rides on LiveBackup, and a schedule that
saves to S3 gets a per-database entry saying the target reads back
only as s3_storage_id, an int nothing maps to a storage UUID
- a DISABLED schedule (declaring the block would make apply re-enable it)
- several schedules where a manifest declares one
- an unreadable route (reported, never read as "no backups")
The stale NO_API_COVERAGE "backup schedules" row becomes "a backup
schedule's S3 target", and semantics.md's draft section now tells the
truth about what is captured.
Closes#75
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A prod box with zero drift could not make `cast diff` say clean: sixteen
lines of `live-only (orphan var — apply never removes)`, every one of them a
var Coolify MINTED — `SERVICE_FQDN_API` for a compose app's per-container
domains, `SERVICE_PASSWORD_POSTGRES`/`POSTGRES_*` for the one-click umami
service's bundled datastore. They held two resources permanently in `change`.
`remove-candidate` means "a live-only var the manifest does not declare;
apply never removes it; read it by eye". For a name cast did not put there,
cannot declare in any vocabulary, and will never remove, that is a category
error — and a report that can never say clean is how an operator learns to
stop reading it. #78's own Impact section made the argument: "an operator who
learns these always show change stops trusting the diff."
cast already knew: draft.ts has held this exact judgment since #27 and used
it to refuse copying these into a draft. diffEnv just never asked. So the
vocabulary moves to reserved.ts — which already owns "names the platform, not
the manifest, controls" — and both callers consult it.
TWO WIDTHS, deliberately, because over-matching is safe in a draft and unsafe
in a diff:
- draft (WIDE): over-matching withholds a value for review — loud and
recoverable. Under-matching copies the source box's DATABASE_URL into a
new box that boots against the OLD box's database. It errs wide.
- diff, applications (NARROW): over-matching HIDES a live-only var. A
hand-left DATABASE_URL still pointing at a box nobody declares is the one
orphan most worth printing — and it matches the wide rule. Probed against
prod: the wide bucket on a real application held DATABASE_URL and
REDIS_URL, both of them cast's OWN declared vars.
- diff, services (WIDE): a Coolify service is a vendored bundle whose
internals cast does not model — `type` + `service_domains` + an
env_template is the whole vocabulary, and the rest is the template's.
Also fixes a real gap the #87 tests found: the pair-rule missed `POSTGRES_DB`
outright, because [POSTGRES, DB] is datastore + datastore with no connection
word. A db NAME is a connection coordinate like any other, so `DB` joins them
— it is exactly the var a one-click service mints for its bundled Postgres.
And corrects LiveEnvVar's comment: it still cited #79's "stale real_value, a
stored column Coolify does not refresh". That was false — an accessor cannot
go stale, and real_value tracks value on every row of a real box. The split
is still right (real_value is an ESCAPED rendering: 'true' is not true); only
its motivation was wrong. The drift it chased was #85's preview shadow.
Tests: an application carrying only SERVICE_* reads clean; a hand-left
DATABASE_URL on an application is STILL reported; a service carrying the
one-click template's wiring reads clean; a non-generated live-only var on a
service is STILL reported.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`inventory --emit-draft` wrote the `github_apps` binding by guessing: with
exactly one App on the instance it bound every repo to it ("no other it
could be"), and with none or several it left a REVIEW marker on all of them.
The audit (#72) showed the binding is READABLE, so the guess was both
unnecessary and, on a single-App instance, silently WRONG for any public
repo (a repo cloned without a GitHub App got bound to the one App anyway).
Every application carries the `source_id`/`source_type` of the App that
clones it — `removeSensitiveData` hides neither (ApplicationsController
v4.1.2) — and `GET /github-apps` returns each App's `id` and `name` (only
`client_secret`/`webhook_secret` are hidden). So the draft now matches the
two: each repo binds to the App its application's `source_id` names. A
GitlabApp/public-repo source (or an instance that will not list its Apps)
resolves to nothing and still gets a REVIEW marker — and a `source_id` that
collides with an App id but carries a non-GithubApp `source_type` is not
mistaken for one.
The biggest gain is the multi-App instance the old heuristic could not
handle at all: it wrote REVIEW on every repo; the lookup resolves each.
semantics.md, the draft header, and the NO_API_COVERAGE row are corrected to
match (the audit's #51 arc: a limitation filed as a defect gets fixed).
`npm run check` clean · 511 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Services could not carry hostnames through cast: `desiredFromManifest`
dropped a service's `domains` and warned they were a manual Coolify UI act,
citing a re-checked "no flat `domains` on a 4.1.2 service, on any route."
The audit (#72) disproved that — the same failure mode #51 corrected for
backup schedules. The FLAT shape genuinely has no route; the per-container
CAPABILITY was there at 4.1.2 all along.
`POST /services` and `PATCH /services/{uuid}` both take a `urls` list
([{name, url}], url comma-joined) that `applyServiceUrls` matches to a
`ServiceApplication` by name and stores as its `fqdn`; `GET /services/{uuid}`
loads `applications` and returns each `fqdn` (verified against
ServicesController v4.1.2). So services now speak the SAME per-container
vocabulary a dockercompose app does:
- **Manifest:** `service_domains: { <container>: [url] }` replaces the flat,
unhonorable `domains` on a service (a flat list cannot name which container
a hostname belongs to — exactly what `urls` requires). Canonicalized (keys
and each URL array sorted) so container order never false-drifts.
- **Write:** `serviceApiFields` builds `urls` on create and update.
- **Read/diff:** a supplementary `GET /services/{uuid}` per service
(`attachServiceDomains`, gated to `diff`/`apply` like backups) projects
`applications[].fqdn` back into `service_domains`, so a declared hostname is
compared every run — no perpetual drift, no manual UI step.
- **Pre-flight:** a service create's `service_domains` joins
`desiredDomainsOfCreate`, the more important because a service create whose
domain conflicts is DELETED server-side before the 409 (rollback).
Two limits stated out loud: the read is fail-closed (an unreachable/
unrecognized `GET /services/{uuid}` aborts rather than projecting empty and
re-PATCHing forever), and `inventory --emit-draft` does not yet make the
per-service GET, so a drafted service's hostnames are still declared by hand
(same as backups) — draft/semantics say so.
`npm run check` clean · 514 tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
`draft.ts` still told the operator that a database's backup schedule "is
not exposed by Coolify's API" and is "create-time-only in cast" — in the
`backup` uncaptured flag, its comment, and the NO_API_COVERAGE table. #51
disproved both: `GET /databases/{uuid}/backups` answers, and `diff` and
`apply` now read and write it.
The audit (#72) flags this as the #51 failure mode repeating — a defect
filed as a limitation does not get fixed. semantics.md was already
corrected when #51 landed; this brings draft.ts's three copies of the old
claim in line with it.
What is still true, and now said accurately: the DRAFT path
(`inventory --emit-draft`) does not yet read that route, so no `backup:`
block is captured and a rebuild from a draft still has no backups until
the operator declares one — not because the API cannot express it.
Text-only; no behavior change. `npm run check` clean, tests pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
apply created applications but dropped install_command, build_command, and
is_static — settings the manifest had no field for — so a static site in an
npm-workspace monorepo (landing) was built and RUN from the repo-root
package.json, booting the core API server, which crash-looped on a missing
DATABASE_URL.
The build block gains install_command / build_command / start_command
(free-form strings) and static (-> Coolify is_static). apply writes and diffs
them; draft emits them (they left its NO_HOME list, and is_static was never in
it — the silent loss that caused the crash), and only emits static alongside a
publish_directory so a draft always loads.
Managing is_static is opt-in: declaring `static:` is required to serve a static
app, and NOT emitting is_static by default avoids the first apply PATCHing
static serving OFF on an un-migrated app (or fighting a pack:static coupling
forever). static:true with no publish_directory, and any of the four on a
dockercompose app, are parse-time refusals.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Coolify injects SOURCE_COMMIT and the COOLIFY_* family into an application's
runtime environment itself, and SKIPS its own injection of a name the resource
already carries a var of (ApplicationDeploymentJob.php v4.1.2, line 2994 —
`->where('key', 'SOURCE_COMMIT')->isEmpty()`). A resource-level var of that
name therefore SUPPRESSES the platform's value. An empty one suppresses it
just as completely: presence, not value.
And it fails green — the deploy succeeds, health checks pass, and the only
symptom is /version reporting "unknown", the endpoint a production cutover is
gated on (D-266).
The rule is now a property of cast, not of one code path. A new src/reserved.ts
owns it, and every place cast touches an env var honors it:
- resolve — every manifest read (desiredFromManifest, requiredSecrets,
manifestResources) refuses a template declaring a reserved name, before any
write. So apply, diff, capture and inventory all refuse identically.
- draft — a reserved name read off a live box gets its own provenance,
`suppressed`: out of the template, out of the age store, its live value read
into no artifact, and named in UNCAPTURED.md with the consequence.
- diff — promoted out of the remove-candidate orphan list ("apply never removes
these; read them by eye") and printed as a FINDING with its consequence. Not
clean. apply still never deletes: cast reports, the human removes it.
- capture (classify) and cli (syncEnv) carry the same assertion at the file and
at the wire — unreachable through the CLI today, and kept because the
invariant is "cast never writes one", not "the CLI happens to check first".
- smoke writes an env var too; its probe names are asserted outside the space.
The rule lives in cast's code, NOT beside forbidden_var_patterns in private
state: that one is policy an environment may set for itself, this one is a fact
about Coolify, true on every box — nothing a manifest change could lower.
19 tests in test/reserved.test.ts, one per path.
Closes#50.
`cast inventory` could already see a whole instance (#22). It can now write
down what it sees, in the shape of cast's own inputs:
cast inventory --env prod --instance box-b --emit-draft ./draft
draft/
environments.yaml # bindings as far as they can be read — with the projects: registry (#25)
incubator/.infra/manifest.yaml # one per project
incubator/.infra/env/*.env.template
la-familia/.infra/manifest.yaml # …including the client sites nobody ever declared
secrets/<project>.<env>.env.age # encrypted to a recipient you name
UNCAPTURED.md # ← the important file
Two uses: bootstrapping a project that has no manifest (the third-party sites
on the box being drained were never declared, and never will be unless
something writes the first draft), and a point-in-time blueprint.
A DRAFT IS A PROPOSAL. It is never desired state, and `apply` never reads it:
sweep → emit draft → a human reads it → manifest PR → capture → apply
Same shape as `terraform import` → HCL, and the boundary is enforced, not
merely documented. It never emits into a repo that already has a manifest —
for a declared project the manifest IS the truth, and one regenerated from a
live box would let that box's accumulated cruft overwrite a reviewed spec, in
the one direction nobody reviews. Adoption is one-way. So: a non-empty target
refuses, a manifest at the path it would write refuses, and --emit-draft with
a repo positional refuses (that is the reconcile path, and it is exactly the
case where a draft must not be written).
Two things would make a draft actively dangerous, and both are the point:
1. COPIED PROVIDER-GENERATED VALUES. A DATABASE_URL read off the source points
at the SOURCE box's Postgres; rebuild elsewhere and the new box comes up
WORKING, reading and writing the old box's database, and you find out the
day the old box is deleted. So the draft applies capture's discipline: a
provider-generated name is placeheld with the same GENERATED_PLACEHOLDER
literal, its live value is written into no artifact, and the emitted
manifest declares it under generated_secrets: so a later capture placeholds
it again with no flag to remember. The rule is by NAME — Coolify's SERVICE_*
magic vars, and any name carrying a datastore word and a connection word —
and it errs wide, because over-matching a real secret is loud and
recoverable while under-matching a generated one is silent and is not.
Every other var becomes a ${REF} with its value in the age store, never a
literal in a committed file: cast cannot know which of a box's vars are
secret, and a live key written as a literal is a key in a git repo.
2. SILENT LOSSES. UNCAPTURED.md is a first-class output, emitted on every run:
per resource, every live setting cast saw and could not express —
destinations (#21), service hostnames, Basic Auth/Traefik labels, backup
schedules, database kinds cast does not model, env names a template cannot
hold — plus what no API in 4.1.2 will tell it, and the table of what a
blueprint still cannot restore (the GitHub App private key and the S3 keys:
re-create by hand). A blueprint that omits these without saying so is worse
than no blueprint, because in a disaster you would trust it and rebuild a
different box.
Secrets are encrypted to a recipient you NAME (--recipient, or the
environment's age_recipient binding). With neither, cast refuses rather than
quietly emitting a draft that looks complete and holds not one value;
--no-secrets says so deliberately. A project with resources in two populated
environments is a tie cast will not break — it refuses, and --environment says
which, as a tiebreak rather than a filter (filtering by name would drop the
client sites, each alone in Coolify's default `production`, out of a blueprint
that claims to describe the box).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
