diff --git a/README.md b/README.md index 3fc8dfa..f5394fd 100644 --- a/README.md +++ b/README.md @@ -69,6 +69,7 @@ cast diff / --env [--full] cast diff --env --all [--full] # no repo: EVERY registered project cast capture / --env [--generated ] [--override ] cast inventory / --env +cast inventory --env [--emit-draft [--recipient age1…] [--no-secrets]] cast server add --ip --key --env [--user root] [--port 22] cast smoke / --env [--project ] [--environment ] cast team [--env ] @@ -91,6 +92,12 @@ cast team [--env ] no age key, and no recipient — it runs *before* adoption, which is the point of it. A document, read by a person; nothing here is consumed by `apply`. See *Adopting a hand-built instance*. +- **`inventory --emit-draft `** — the sweep, written down as a **draft of + cast's own inputs**: a manifest per project, env templates, an + `environments.yaml` with the registry, an age store, and `UNCAPTURED.md`. A + **proposal**, never desired state — `apply` does not read it. It is how a + project that has *no* manifest gets its first one, and how you take a + point-in-time blueprint of a box. See *Drafting a box that was never declared*. - **`capture`** — the adoption path: reads a hand-built instance's live env and writes the environment's age store from it. See *Adopting a hand-built instance* below. @@ -323,6 +330,126 @@ build pack, the hostname-overlay shapes, and the places Coolify 4.1.2 does not cooperate — each citation verified against `coollabsio/coolify` v4.1.2 and the vendored OpenAPI in `reference/`. Read it before changing `apply`. +## Drafting a box that was never declared + +`inventory` can see a whole instance. `--emit-draft` makes it **write down what +it sees**, in the shape of cast's own inputs: + +```sh +cast inventory --env prod --instance box-b --emit-draft ./draft --recipient age1… +``` + +``` +draft/ + environments.yaml # bindings as far as they can be read — with the projects: registry + incubator/.infra/manifest.yaml # one per project + incubator/.infra/env/*.env.template + la-familia/.infra/manifest.yaml # …including the client sites nobody ever declared + secrets/..env.age # encrypted to a recipient you name + UNCAPTURED.md # ← the important file +``` + +Two uses: **bootstrapping a project that has no manifest** (the third-party sites +on the box being drained were never declared, and never will be unless something +writes the first draft — hand-transcribing them from a UI is exactly the work +cast exists to eliminate), and **a point-in-time blueprint** you could rebuild an +instance from. + +### A draft is a PROPOSAL + +**It is never desired state, and `apply` never reads it.** It is emitted, +reviewed by a human, and lands in a repo as a PR — the same shape as +`terraform import` → HCL: + +**sweep → emit draft → you read it → manifest PR → `capture` → `apply`** + +That boundary is the only reason the verb is allowed to exist, and it is +enforced, not merely documented: + +- It **never emits into a repo that already has a manifest.** For a declared + project the manifest *is* the truth, and one regenerated from a live box would + let that box's accumulated cruft overwrite the reviewed spec — in the one + direction nobody reviews. Adoption is one-way. A non-empty target directory is + refused, and so is writing a manifest over an existing one. +- `--emit-draft` is **sweep-mode only**. With a repo, `inventory` is reconciling + against a manifest that already exists, which is exactly the case where a draft + must not be written. Refused. + +### Two things would make a draft actively dangerous + +**1. Copied provider-generated values.** A `DATABASE_URL` read off the source +points at the *source box's* Postgres. Emit it, rebuild elsewhere, and the new +box comes up **working** — reading and writing the old box's database. You find +out the day the old box is deleted. Same for `REDIS_URL`, and for Coolify's own +magic vars (`SERVICE_FQDN_*`, `SERVICE_URL_*`, `SERVICE_PASSWORD_*`), which are +generated per-instance and mean nothing anywhere else. + +So the draft applies **`capture`'s discipline**: a provider-generated name is +**placeheld** with the same `pending-coolify-generated` literal, its live value +is not written into any artifact, and it is listed for disposition. The emitted +manifest declares it under `generated_secrets:`, so a later `capture` placeholds +it again with no flag to remember. A draft that is confidently wrong in four +entries out of seventeen is worse than one that is obviously incomplete. + +The rule is **by name** — two families: Coolify's `SERVICE_*` magic vars, and any +name carrying a *datastore* word (`DATABASE`, `DB`, `POSTGRES`, `REDIS`, …) and a +*connection* word (`URL`, `HOST`, `PASSWORD`, …) as segments. It errs **wide** on +purpose, because the two errors are not symmetric: over-matching a real secret +placeholds it loudly and you put it back, while under-matching a generated one +copies it silently and rebuilds a box that quietly uses a dead machine's +database. Every value cast read is printed with its disposition — names and +provenance, never values — and a var that points at the source box under a name +cast does not recognize **will** have been copied. Read the table. + +Every other live var becomes a `${REF}`, with its value in the **age store** — +never a literal in a committed file. cast cannot know which of a box's vars are +secret (nobody wrote it down; that is why this verb exists), and a live API key +written as a literal is a key in a git repo. Move the plainly-not-secret ones +back to literals yourself, in review. + +The store is encrypted to a recipient you **name** — `--recipient age1…`, or the +environment's `age_recipient` binding. With neither, cast **refuses**: a draft +whose secrets were silently skipped looks complete and holds not one value. +`--no-secrets` says so deliberately. + +**2. Silent losses.** cast cannot express everything a Coolify holds: +destinations (which Docker network a resource sits on — no API at all in 4.1.2), +service hostnames (they live per-container on `service.applications[].fqdn`), +Basic Auth and custom Traefik labels, the *Include Source Commit in Build* +toggle, whole database kinds (a MySQL is invisible to cast's manifest), backup +schedules, and anything else configured in the UI with no manifest field. + +A blueprint that omits these **without saying so** is worse than no blueprint, +because in a disaster you would trust it and rebuild a *different box*. So +**`UNCAPTURED.md` is a first-class output**, listing per resource every live +setting cast saw and could not express — and it is **written on every run**, even +when it has little to say. + +### What a blueprint still cannot restore + +Worth stating plainly, because "rebuild from the repo" is routinely over-claimed: + +| | | +| --- | --- | +| control plane | `rig coolify install` ✅ | +| structure | draft → manifest PR → `apply` ✅ | +| secret **values** | the age store + your key ✅ | +| **data** | Coolify's DB backups → S3 ✅ (a separate path) | +| **the GitHub App private key** | ❌ re-create by hand | +| **S3 access keys** | ❌ re-mint by hand | + +The last two are **not in the repo** — correctly; it holds no live credentials — +and cannot be regenerated from it. A DR runbook has to say so. The same table is +emitted into every `UNCAPTURED.md`, because that is the file someone will be +reading at the worst possible moment. + +One project per Coolify environment: a project with resources in **two** +populated environments is a tie cast will not break (picking would emit a +blueprint of half a box), so it refuses and `--environment ` says which. It +is a tiebreak, not a filter — a project with only one populated environment is +drafted from it either way, which is what keeps the client sites (each alone in +Coolify's default `production`) in a blueprint that claims to describe the box. + ## Secrets, and attended applies An environment's age identity is resolved in exactly two ways: diff --git a/docs/semantics.md b/docs/semantics.md index aa81ec1..d43d93b 100644 --- a/docs/semantics.md +++ b/docs/semantics.md @@ -424,6 +424,111 @@ the plan. There is no `--yes`: a store written without someone reading the provenance column is the outcome the verb exists to prevent. A closed stdin aborts rather than hanging. +## Drafts (`inventory --emit-draft`) + +`inventory` with no repo sweeps an instance. `--emit-draft ` writes that +sweep down as a draft of cast's **own inputs** — a manifest per project, env +templates, an `environments.yaml` carrying the `projects:` registry, an age store +per project, and `UNCAPTURED.md`. + +**A draft is a PROPOSAL. It is never desired state, and `apply` never reads it.** + + sweep → emit draft → a human reads it → manifest PR → capture → apply + +Same shape as `terraform import` → HCL. Every other rule in this section follows +from that one, and each is enforced rather than merely stated: + +| refusal | why | +| --- | --- | +| a **non-empty** target directory | emitted over a repo that has a manifest, a draft would overwrite a reviewed spec with a live box's accumulated cruft — the one direction nobody reviews. **Adoption is one-way.** | +| an **existing manifest** at the path it would write | the same invariant, once more at the file (`assertNoExistingManifest`). For a declared project the manifest *is* the truth; `cast inventory /` reconciles it instead. | +| `--emit-draft` with a **repo positional** | with a repo, inventory reconciles against a manifest that already exists — exactly the case where a draft must not be written. | +| **no age recipient** (and no `--no-secrets`) | a draft whose store was silently skipped *looks complete*: a manifest, templates full of `${REF}`s, and not one value anywhere. You would find out when `apply` refused, some time after the box those values were on stopped existing. | +| a project with **two populated environments** | a draft carries one environment per project. Picking would emit a blueprint of *half a box* that says nothing about the other half. `--environment` breaks the tie — as a **tiebreak, not a filter**: a project with one populated environment is drafted from it either way, or filtering by name would drop whole projects (each client site sits alone in Coolify's default `production`) out of a blueprint that claims to describe the box. | + +**Provider-generated names are placeheld, never copied.** This is `capture`'s +discipline (see above), applied to a verb that has no manifest to tell it which +names are generated — so it decides **by name**, in two families: + +1. Coolify's per-instance magic vars — `SERVICE_FQDN_*`, `SERVICE_URL_*`, + `SERVICE_PASSWORD_*`, `SERVICE_USER_*`, `SERVICE_BASE64_*`. +2. Any name carrying a **datastore** word (`DATABASE`, `DB`, `POSTGRES`, `PG`, + `REDIS`, `MONGO`, …) *and* a **connection** word (`URL`, `URI`, `DSN`, `HOST`, + `PORT`, `PASSWORD`, `USER`, …) as underscore-delimited segments — + `DATABASE_URL`, `UMAMI_DATABASE_URL`, `REDIS_URL_PROD`, `DB_HOST`. + +Each such name is written as the literal `pending-coolify-generated`, listed in +the run's disposition table, and declared under the emitted manifest's +`generated_secrets:` — so a later `capture` placeholds it again with no flag to +remember. **Its live value is not written into any artifact.** + +The rule errs **wide**, deliberately, because the two errors are not symmetric: + +- over-match a real secret → it is placeheld, reported, and you put the value + back. Noisy, recoverable, **loud**. +- under-match a generated one → it is copied, and a box rebuilt from the draft + comes up **working**, reading and writing the *source box's* database, until + the day that box is deleted. Silent, unrecoverable, **quiet**. + +It is a name-pattern rule, not a promise: a var that points at the source box +under a name cast does not recognize **will** be copied. The disposition table +(names and provenance, **never values** — same contract as the capture plan) is +what a reviewer reads to catch it. + +**Every other live var becomes a `${REF}`**, with its value in the age store — +never a template literal. cast cannot know which of a box's vars are secret +(nobody wrote it down, which is why the verb exists), and the two mistakes are +again asymmetric: a non-secret in an encrypted store is untidy, a live API key +written as a literal into a manifest is a key in a git repo. One name carrying +**different values** on two resources is not a conflict cast resolves (one store +holds one value per name — see `capture`'s CONFLICT refusal): both are kept, +under `_` refs, and the split is reported. + +**`UNCAPTURED.md` is a first-class output, emitted on every run.** cast cannot +express everything a Coolify holds, and a blueprint that omits those things +without saying so is worse than no blueprint — in a disaster you would trust it +and rebuild a *different box*. Per resource, it names what was seen and could not +be written: `destination_id` (which Docker network — no destinations API in 4.1.2 +to resolve it to the UUID `destination_uuid:` wants, #21), service hostnames (no +flat `domains` on a Coolify 4.1.2 service), Basic Auth / custom Traefik labels, +build and deploy command overrides, backup schedules (not exposed on a database's +GET — **a rebuild has no backups until you declare them**), database kinds cast +does not model (MySQL, MariaDB, MongoDB, KeyDB, Dragonfly, ClickHouse — named, +never silently dropped), env var names a cast template cannot express, and +applications whose build pack the manifest has no vocabulary for (left *out* of +the manifest rather than fabricated into the nearest pack). + +It also carries the table below, because that is the file someone will be reading +at the worst possible moment. + +### What a blueprint still cannot restore + +| | | +| --- | --- | +| control plane | `rig coolify install` ✅ | +| structure | draft → manifest PR → `apply` ✅ | +| secret **values** | the age store + your key ✅ | +| **data** | Coolify's DB backups → S3 ✅ (a separate path) | +| **the GitHub App private key** | ❌ re-create by hand | +| **S3 access keys** | ❌ re-mint by hand | + +The last two are not in the state repo — correctly, it holds no live credentials +— and cannot be regenerated from it. A DR runbook that does not say so is not a +runbook. + +**What the box cannot tell you, and cast therefore does not invent:** the +`/` slug comes from an application's git remote (the only place a live +box knows it), so a project with **no application** — a lone service — has no repo +on the box at all. cast writes the bare project name as the registry key, and the +registry's own parse-time refusal (*"a registry key has no meaning without its +org"*) then stops the file being used until a human supplies it. That refusal is +the design: the alternatives are inventing an org, or leaving the project out of +the registry — and a project missing from the registry is one every fleet run +skips **in silence**. Likewise `github_apps`: nothing Coolify returns about an +application says which App cloned it, so cast binds every repo to the instance's +only GitHub App when there is exactly one (there is no other it could be), and +writes a `REVIEW-…` marker when there is not. + ## Cloning a private manifest `resolveCheckout` resolves git credentials **inside cast**, in a fixed order — diff --git a/src/cli.ts b/src/cli.ts index 1f2017b..7f6d012 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -34,6 +34,17 @@ import { computeDiff, renderDiff, } from "./diff.js"; +import { + type DraftProject, + assertEmptyTarget, + draftResourcesFrom, + emitDraft, + planDraft, + renderAmbiguousEnvironments, + renderDraftPlan, + renderNoRecipient, + renderRepoWithDraft, +} from "./draft.js"; import { assertEnvVarPolicy } from "./envtemplate.js"; import { type ProjectOutcome, @@ -78,6 +89,7 @@ const USAGE = `usage: cast apply / --env [--path ] [-- cast capture / --env [--path ] [--project ] [--environment ] [--generated ] [--override ] [--force] cast inventory / --env [--path ] [--project ] [--environment ] [--resource =] cast inventory --env [--instance ] # no repo: SWEEP the whole instance + cast inventory --env --emit-draft [--recipient age1…] [--no-secrets] cast server add --ip --key --env [--user root] [--port 22] cast smoke / --env [--project ] [--environment ] cast team [--env ] @@ -137,7 +149,27 @@ capture (adopt a hand-built instance into the age secret store): --override supply NAME yourself instead of copying the source's value. The VALUE is read from \$CAST_CAPTURE_, never from the command line — argv is visible in \`ps\`. Repeatable. - --force overwrite an existing store (refused by default).`; + --force overwrite an existing store (refused by default). + +inventory --emit-draft (write down what a box has, as a PROPOSAL): + --emit-draft emit what the sweep saw as a draft of cast's own inputs — a + manifest per project, env templates, an environments.yaml + carrying the \`projects:\` registry, an age store per project, + and UNCAPTURED.md. Into a NEW directory, always: a draft is a + proposal, reviewed by a human and landed as a PR, and \`apply\` + never reads one. SWEEP MODE ONLY — with a repo there is already + a manifest, and a manifest regenerated from a live box would + overwrite a reviewed spec with that box's accumulated cruft. + --recipient age1… the age recipient the draft's stores are encrypted to. Defaults + to the environment's \`age_recipient\` binding. + --no-secrets emit no stores. Required when no recipient is available: cast + will not silently drop the values it read off the box. + --environment a TIEBREAK, not a filter: which environment to draft for a + project that has resources in more than one (cast refuses to + pick). A project with only one populated environment is drafted + from it either way — filtering the instance by an environment + name would drop whole projects out of a blueprint that claims to + describe the box.`; // cast is stateless: every instance-scoped input is read from the state // directory it is pointed at, never from a location the tool itself knows. @@ -568,7 +600,7 @@ export function aliasLive( // capture. async function fetchEnv( client: CoolifyClient, - l: Live, + l: Pick, ): Promise> { const base = l.kind === "database" ? "databases" : `${l.kind}s`; const envs = (await client.get(`/${base}/${l.uuid}/envs`).catch((err) => { @@ -1155,6 +1187,9 @@ async function main(): Promise { environment: { type: "string" }, resource: { type: "string", multiple: true }, instance: { type: "string" }, + "emit-draft": { type: "string" }, + recipient: { type: "string" }, + "no-secrets": { type: "boolean", default: false }, }, }); const orgRepo = positionals[0]; @@ -1163,6 +1198,17 @@ async function main(): Promise { console.error(USAGE); return 2; } + const draftDir = values["emit-draft"]; + // A draft is emitted from the SWEEP, and only from the sweep. With a repo, + // inventory is reconciling against a manifest that already exists — which is + // exactly the case where a draft must not be written: for a declared project + // the manifest IS the truth, and one regenerated from a live box would let + // that box's accumulated cruft overwrite the reviewed spec. Adoption is + // one-way, so the two flags cannot be combined at all. + if (draftDir && orgRepo) { + console.error(renderRepoWithDraft(orgRepo, draftDir)); + return 2; + } const stateDir = stateDirFrom(values.state); const sweepBindings = loadBindings(join(stateDir, "environments.yaml")); const sweepBinding = sweepBindings.environments[envName]; @@ -1176,6 +1222,28 @@ async function main(): Promise { // made inventory a discovery verb that needed you to have already // discovered. if (!orgRepo) { + // Both refusals BEFORE the first live call. A draft that is going to be + // refused should be refused before an operator watches a whole instance be + // swept for it — and, more to the point, before cast reads every env var on + // a box it is then not going to write down. + const recipient = values.recipient ?? sweepBinding.age_recipient; + if (draftDir) { + try { + assertEmptyTarget(draftDir); + } catch (err) { + console.error(err instanceof Error ? err.message : String(err)); + return 2; + } + // No recipient, no store — and cast will not make that decision quietly. + // Silently skipping the secrets would emit a draft that LOOKS complete: a + // manifest, templates full of ${REF}s, and nothing anywhere holding a + // single value. You would find out when `apply` refused, having already + // deleted the box the values were on. + if (!recipient && !values["no-secrets"]) { + console.error(renderNoRecipient(envName)); + return 2; + } + } const { instance, client } = openCoolify( stateDir, values.instance, @@ -1186,8 +1254,9 @@ async function main(): Promise { // truthfully reports that it is empty. const team = await assertTeam(client, sweepBinding.team, envName); console.log(`team ${formatTeam(team)} ✓`); + const live = await client.projects(); const projects: SweepProject[] = []; - for (const p of await client.projects()) { + for (const p of live) { const environments: SweepEnvironment[] = []; for (const name of await client.environments(p.uuid)) { const found = await fetchLive(client, p.name, name); @@ -1206,6 +1275,122 @@ async function main(): Promise { baseUrl: instance.baseUrl, }), ); + if (!draftDir) return 0; + + // --- The draft (#27) --- + // + // The sweep above is a DOCUMENT. This is the same reading, written into the + // shape of cast's own inputs — and it is still a proposal, not desired + // state. See draft.ts for the boundary that lets this verb exist at all. + // + // A project with resources in TWO environments cannot be drafted without + // picking one, and cast does not pick: a blueprint of half a box, silently + // chosen, is the failure mode this whole issue is about. --environment says + // which. + // + // --environment is a TIEBREAK here, not a filter. A project with resources + // in exactly one environment has no tie to break, and is drafted from it + // whatever the flag says — filtering the instance by an environment NAME + // would drop the projects that most need drafting (the third-party sites, + // each sitting in its own Coolify-default `production`) out of a blueprint + // that claims to describe the box. + const populatedIn = (p: SweepProject) => + p.environments.filter((e) => e.resources.length > 0); + const pick = (p: SweepProject) => { + const populated = populatedIn(p); + return populated.length === 1 + ? populated[0] + : populated.find((e) => e.name === values.environment); + }; + const ambiguous = projects.filter( + (p) => populatedIn(p).length > 1 && !pick(p), + ); + if (ambiguous.length > 0) { + console.error( + renderAmbiguousEnvironments( + ambiguous.map((p) => ({ + name: p.name, + environments: populatedIn(p).map( + (e) => `${e.name} (${e.resources.length})`, + ), + })), + envName, + ), + ); + return 2; + } + const draftProjects: DraftProject[] = []; + for (const p of live) { + const swept = projects.find((s) => s.name === p.name); + const populated = swept ? populatedIn(swept) : []; + const chosen = swept ? pick(swept) : undefined; + const others = populated + .filter((e) => e.name !== chosen?.name) + .map((e) => ({ name: e.name, resources: e.resources.length })); + if (!chosen) { + // Not drafted — and SAID, in UNCAPTURED.md, rather than left out of a + // blueprint that a reader would take for the whole box. + draftProjects.push({ + name: p.name, + coolifyEnv: "(none)", + resources: [], + unreadable: [], + otherEnvironments: others, + skipReason: "every environment on it is empty", + }); + continue; + } + // The RAW environment document, not fetchLive's projection: the uncaptured + // pass's whole job is to notice fields cast has no home for, and it cannot + // notice what a projection has already thrown away. + const raw = (await client.get( + `/projects/${p.uuid}/${chosen.name}`, + )) as Record | null; + const { resources, unreadable } = draftResourcesFrom(raw ?? {}); + for (const r of resources) { + // Databases hold no manifest-templated env of their own — their URL is + // what the APPS reference, and that name is generated, not captured. + if (r.kind === "database") continue; + r.env = await fetchEnv(client, { kind: r.kind, uuid: r.uuid }); + } + draftProjects.push({ + name: p.name, + coolifyEnv: chosen.name, + resources, + unreadable, + otherEnvironments: others, + }); + } + // Which GitHub App clones a repo is NOT a property of any resource — no + // field Coolify returns about an application says so. What the instance can + // answer is which Apps exist; with exactly one, there is no other it could + // be. Best-effort: an instance that will not list them still gets a draft, + // with a REVIEW marker where the binding goes. + const githubApps = (await client.get("/github-apps").catch(() => [])) as + | Array<{ name?: unknown }> + | undefined; + const draftCtx = { + env: envName, + instance: instance.name, + baseUrl: instance.baseUrl, + team, + server: sweepBinding.server, + githubApps: (Array.isArray(githubApps) ? githubApps : []) + .map((a) => a?.name) + .filter((n): n is string => typeof n === "string"), + recipient, + generatedAt: new Date().toISOString(), + }; + const storeRecipient = values["no-secrets"] ? undefined : recipient; + const plan = planDraft(draftProjects, draftCtx); + const written = emitDraft(draftDir, plan, { recipient: storeRecipient }); + console.log( + renderDraftPlan(plan, draftCtx, { + dir: draftDir, + recipient: storeRecipient, + written, + }), + ); return 0; } const repoShort = orgRepo.split("/")[1]; diff --git a/src/draft.ts b/src/draft.ts new file mode 100644 index 0000000..8fa0697 --- /dev/null +++ b/src/draft.ts @@ -0,0 +1,1321 @@ +import { existsSync, mkdirSync, readdirSync, writeFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { stringify } from "yaml"; +import { GENERATED_PLACEHOLDER } from "./capture.js"; +import { encryptSecrets } from "./secrets.js"; + +// `inventory` can already SEE a whole instance (#22). This is it writing down +// what it saw, in the shape of cast's own inputs — a manifest per project, an +// env template per resource, a bindings file with the registry, an age store, +// and UNCAPTURED.md. +// +// The verb is only allowed to exist because of one boundary, and everything in +// this file is bent around it: +// +// **A draft is a PROPOSAL. It is never desired state, and `apply` never +// reads it.** +// +// sweep → emit draft → a human reads it → manifest PR → capture → apply +// +// Same shape as `terraform import` → HCL. It is emitted, reviewed, and lands in +// a product repo as a PR; the repo stays the source of truth. Which is why a +// draft is NEVER emitted into a repo that already has a manifest: for a declared +// project the manifest *is* the truth, and regenerating it from a live box would +// let that box's accumulated cruft overwrite the reviewed spec, silently, in the +// one direction nobody reviews. Adoption is one-way. +// +// Two things would make a draft actively dangerous, and they are the two things +// this file spends its length on: +// +// 1. COPIED PROVIDER-GENERATED VALUES. A `DATABASE_URL` read off the source +// points at the SOURCE box's Postgres. Emit it, rebuild elsewhere, and the +// new box comes up WORKING — reading and writing the old box's database. You +// find out the day the old box is deleted. So the draft applies `capture`'s +// discipline (see classify): a provider-generated name is placeheld with the +// same `pending-coolify-generated` literal and listed for disposition, and +// the source's value is not written anywhere. A draft that is confidently +// wrong in four entries out of seventeen is worse than one that is obviously +// incomplete. +// +// 2. SILENT LOSSES. cast cannot express everything a Coolify holds — service +// hostnames, destinations (#21), Basic Auth, build toggles, whole database +// kinds. A blueprint that omits them WITHOUT SAYING SO is worse than no +// blueprint, because in a disaster you would trust it and rebuild a +// *different box*. Hence UNCAPTURED.md, which is emitted on every run, even +// when it has little to say. + +// --- Provider-generated names ------------------------------------------------- +// +// The single most consequential judgment in this file, and it is made by NAME — +// never by value, and never by "it looks like a URL". +// +// Two families: +// +// 1. Coolify's own magic vars. `SERVICE_FQDN_*`, `SERVICE_URL_*`, +// `SERVICE_PASSWORD_*`, `SERVICE_USER_*`, `SERVICE_BASE64_*` are generated +// per-instance by Coolify when it creates a service, and mean nothing +// anywhere else — a `SERVICE_PASSWORD_UMAMI` carried to a new box is the +// OLD box's password, sitting in the new box's config next to a database +// that has a different one. +// +// 2. Connection coordinates for a datastore the PROVIDER creates. A name that +// carries both a datastore word (DATABASE, POSTGRES, REDIS, …) and a +// connection word (URL, HOST, PASSWORD, …) as segments — `DATABASE_URL`, +// `UMAMI_DATABASE_URL`, `REDIS_URL_PROD`, `DB_HOST`. Coolify mints these +// when it creates the resource; the target's real value does not exist +// until it does. +// +// Deliberately erring WIDE. The two errors are not symmetric: +// +// - Over-match a name that was really a secret: it is placeheld, listed in +// UNCAPTURED.md for disposition, and the operator puts the value back. The +// source box is still there. Noisy, recoverable, LOUD. +// - Under-match one that was really provider-generated: it is copied, the +// rebuilt box boots working against the old box's database, and nobody finds +// out until the old box is deleted. Silent, unrecoverable, QUIET. +// +// A name-pattern rule is not a promise, and the docs say so: a var that points +// at the source box under a name cast does not recognize WILL be copied. That is +// what the disposition table printed at the end of a run is for — read it. +const COOLIFY_MAGIC = + /^SERVICE_(FQDN|URL|USER|PASSWORD|BASE64|REALBASE64)(_|$)/; + +const DATASTORE_WORDS = new Set([ + "DATABASE", + "DB", + "POSTGRES", + "POSTGRESQL", + "PG", + "MYSQL", + "MARIADB", + "MONGO", + "MONGODB", + "REDIS", + "VALKEY", + "KEYDB", + "DRAGONFLY", + "CLICKHOUSE", +]); + +const CONNECTION_WORDS = new Set([ + "URL", + "URI", + "DSN", + "HOST", + "HOSTNAME", + "PORT", + "PASSWORD", + "PASS", + "USER", + "USERNAME", +]); + +export function isProviderGenerated(key: string): boolean { + if (COOLIFY_MAGIC.test(key)) return true; + const words = key.split("_"); + return ( + words.some((w) => DATASTORE_WORDS.has(w)) && + words.some((w) => CONNECTION_WORDS.has(w)) + ); +} + +// --- Names ------------------------------------------------------------------- + +// A box names things for a human reading a UI ("La Familia Site", "Incubator +// Stack v2"); a repo path and an age-store key cannot hold that. Slugs are used +// for PATHS only — the manifest keeps the box's own name as the resource key, +// because a draft that renamed everything to our vocabulary and never mentioned +// theirs would be unusable against the UI it describes (see inventory.ts). +export function slug(name: string): string { + return ( + name + .toLowerCase() + .replace(/[^a-z0-9]+/g, "-") + .replace(/^-+|-+$/g, "") || "unnamed" + ); +} + +const constCase = (name: string) => slug(name).toUpperCase().replace(/-/g, "_"); + +// `/` out of whatever Coolify recorded as the app's git remote — the +// only place on a live box that knows which REPO a project belongs to, and the +// registry (#25) is keyed by exactly that. +// +// Three shapes, all real: Coolify stores a bare `org/repo` for an application +// created through a GitHub App (the shape `apply` itself posts), and a full URL — +// https or scp-style — for a public one. The last two segments are the answer in +// every case; anything with no `/` in it at all is not a repo, and gets +// `undefined` rather than a guess. +export function repoFromGitUrl(raw: unknown): string | undefined { + if (typeof raw !== "string" || raw === "") return undefined; + const m = raw + .trim() + .replace(/\.git$/, "") + .match(/([^/:\s]+)\/([^/\s]+)$/); + return m ? `${m[1]}/${m[2]}` : undefined; +} + +// A cast env template's grammar is `KEY=value` with KEY matching this (see +// parseTemplate in envtemplate.ts — ONE grammar, shared by every reader). A live +// var whose key does not match cannot be written into a template at all, so it +// is not silently dropped: it is listed in UNCAPTURED.md. +const TEMPLATE_KEY = /^[A-Z][A-Z0-9_]*$/; + +// --- Inputs ------------------------------------------------------------------ + +export type DraftResourceKind = "application" | "database" | "service"; + +export type DraftResource = { + kind: DraftResourceKind; + // The BOX's name for it. There is no other. + name: string; + uuid: string; + // The live Coolify object, as it came off the wire. Kept whole, because the + // uncaptured pass's whole job is to notice fields cast has no home for — and + // it cannot notice what a projection already threw away. + raw: Record; + // Live env vars. Empty for databases (their URL is what the apps reference, + // and that name is generated, not captured). + env: Record; +}; + +export type DraftProject = { + // The Coolify project's own name. + name: string; + // The BOX's environment the resources below were read from. + coolifyEnv: string; + resources: DraftResource[]; + // Resources of a kind cast cannot model AT ALL (a MySQL, a MongoDB): seen, + // named, and left out — the loudest possible silent loss if it went unsaid. + unreadable: Array<{ kind: string; name: string }>; + // Environments on this project that this draft does NOT carry. + otherEnvironments: Array<{ name: string; resources: number }>; + // Why nothing was drafted from this project. A project the draft passes over + // gets no manifest and no registry entry — and an entry in UNCAPTURED.md + // saying so, because a reader who takes this draft for "the box" would + // otherwise never learn the project was there. (The sweep printed it; a file + // on disk outlives a terminal.) + skipReason?: string; +}; + +// The live environment document, split into what cast can model and what it +// cannot. `fetchLive` maps the same response, but through a projection — and the +// uncaptured pass cannot notice a field a projection has already discarded, so +// the draft reads the RAW document instead. +// +// The `unreadable` half is the point. Coolify's environment_details eager-loads +// mysqls, mariadbs, mongodbs, keydbs, dragonflies and clickhouses too +// (ProjectController@environment_details, v4.1.2); cast's manifest speaks +// postgresql and redis only. A MySQL on the box is therefore invisible to every +// other verb — and a blueprint that quietly leaves out a database is the exact +// artifact that gets someone to rebuild a box and only later find out what is +// missing from it. So it is read, named, and reported as inexpressible. +const KIND_OF: Array<[string, DraftResourceKind]> = [ + ["applications", "application"], + ["postgresqls", "database"], + ["redis", "database"], + ["services", "service"], +]; + +const UNREADABLE_KINDS = [ + "mysqls", + "mariadbs", + "mongodbs", + "keydbs", + "dragonflies", + "clickhouses", +]; + +export function draftResourcesFrom(env: Record): { + resources: DraftResource[]; + unreadable: Array<{ kind: string; name: string }>; +} { + const resources: DraftResource[] = []; + for (const [field, kind] of KIND_OF) { + const items = env[field]; + if (!Array.isArray(items)) continue; + for (const raw of items as Array>) { + resources.push({ + kind, + name: String(raw.name), + uuid: String(raw.uuid), + raw, + env: {}, + }); + } + } + const unreadable: Array<{ kind: string; name: string }> = []; + for (const field of UNREADABLE_KINDS) { + const items = env[field]; + if (!Array.isArray(items)) continue; + for (const raw of items as Array>) { + unreadable.push({ + kind: field.replace(/s$/, ""), + name: String(raw.name), + }); + } + } + return { resources, unreadable }; +} + +export type DraftContext = { + // OUR environment name (--env): the key every emitted artifact is filed under. + env: string; + instance: string; + baseUrl: string; + team: { id: number; name: string }; + server?: string; + // The GitHub Apps configured on the instance, by name. NOT a property of any + // resource: nothing Coolify returns about an application says which App clones + // it. With exactly one on the instance there is no other it could be, and cast + // binds every repo to it; with none or several it writes a REVIEW marker + // instead of picking. See bindingsDoc. + githubApps?: string[]; + recipient?: string; + generatedAt: string; +}; + +export type Provenance = "captured" | "generated"; + +export type DraftDisposition = { + project: string; + ref: string; + provenance: Provenance; + sites: string[]; + // Never rendered. Held so emitDraft can encrypt it, and nowhere else. + value: string; +}; + +export type UncapturedItem = { + project: string; + resource?: string; + setting: string; + detail: string; +}; + +export type DraftFile = { path: string; content: string }; + +export type DraftPlan = { + files: DraftFile[]; + // project slug -> the age store's contents. Kept out of `files` because it is + // the one artifact that is not text on the way out. + stores: Array<{ + project: string; + path: string; + vars: Record; + }>; + dispositions: DraftDisposition[]; + uncaptured: UncapturedItem[]; +}; + +// --- Headers ----------------------------------------------------------------- +// +// Every emitted file says what it is, in its own body. The artifacts leave this +// process and are read by a person deciding whether to TRUST them — as a +// blueprint of a box they may have to rebuild — and a file that does not say it +// was machine-generated from a live box will be read as if someone meant it. + +function header( + ctx: DraftContext, + lines: string[], + // UNCAPTURED.md carries the same header, minus the line telling you to go and + // read UNCAPTURED.md. It IS the thing being pointed at. + opts: { self?: boolean } = {}, +): string[] { + return [ + "PROPOSAL — not desired state. `apply` does not read this file.", + "", + "Machine-generated by `cast inventory --emit-draft` from a LIVE box:", + ` instance: ${ctx.instance} (${ctx.baseUrl})`, + ` team: ${ctx.team.id} (${ctx.team.name})`, + ` generated: ${ctx.generatedAt}`, + "", + ...lines, + "", + "Everything a box accumulates that nobody meant — a hand-edited var, a resource", + "somebody made once — is in here too. Review it, decide what should be declared", + "and what is cruft that must not travel, and land it as a PR. Then: capture → apply.", + ...(opts.self + ? [] + : [ + "", + "Read UNCAPTURED.md first: it lists what cast SAW on this box and could not", + "express. A blueprint that omits things without saying so is worse than none.", + ]), + ]; +} + +const comment = (lines: string[]) => + lines.map((l) => (l ? `# ${l}` : "#")).join("\n"); + +// --- The manifest ------------------------------------------------------------ + +const PACKS = new Set(["nixpacks", "static", "dockerfile", "dockercompose"]); + +type Spec = Record; + +function applicationSpec( + r: DraftResource, + ctx: DraftContext, + project: string, + hasEnv: boolean, + uncaptured: UncapturedItem[], +): Spec | undefined { + const flag = (setting: string, detail: string) => + uncaptured.push({ project, resource: r.name, setting, detail }); + + const pack = String(r.raw.build_pack ?? ""); + if (!PACKS.has(pack)) { + // NOT "pick the closest pack". An application cast's manifest cannot express + // is left OUT of the manifest and named here — a fabricated build pack would + // rebuild a different application, which is the exact failure UNCAPTURED.md + // exists to prevent, dressed up as coverage. + flag( + "the whole application", + `build pack "${pack || "(none)"}" — cast's manifest supports ${[...PACKS].join(", ")}. This application is NOT in the draft; it cannot be expressed, and guessing a pack would rebuild a different app. Its env vars WERE read, and are in the store and an env template beside it — the values are not lost, only the structure.`, + ); + return undefined; + } + + const repo = repoFromGitUrl(r.raw.git_repository); + if (!repo) { + flag( + "source.repo", + `git remote "${String(r.raw.git_repository ?? "")}" — cast could not read an / out of it, and wrote it through verbatim. \`apply\` resolves a GitHub App by that slug; fix it before you trust this.`, + ); + } + const branch = r.raw.git_branch; + if (typeof branch !== "string" || branch === "") { + flag( + "source.branch", + "the box reports no branch for this application. `main` was written; confirm it.", + ); + } + + const compose = pack === "dockercompose"; + const fqdn = String(r.raw.fqdn ?? "") + .split(",") + .filter(Boolean); + if (compose && fqdn.length > 0) { + flag( + "domains", + `the box has a flat fqdn (${fqdn.join(", ")}) on this compose application; a compose app's hostnames are expressed per-container (service_domains) and cast cannot map one onto the other.`, + ); + } + if (!compose && fqdn.length === 0) { + flag( + "domains", + "no hostname is set on this application; `domains: []` was written, which `apply` would create it with — a rebuilt box would serve nothing here.", + ); + } + + // Real settings, present on the live object, that the manifest has no field + // for. Each one changes what the application IS, and each would be silently + // absent from a rebuild. + const NO_HOME: Array<[string, string]> = [ + ["custom_labels", "custom Traefik/Docker labels (Basic Auth lives here)"], + ["ports_mappings", "host port mappings"], + ["install_command", "a custom install command"], + ["build_command", "a custom build command"], + ["start_command", "a custom start command"], + ["pre_deployment_command", "a pre-deployment command"], + ["post_deployment_command", "a post-deployment command"], + ["dockerfile", "an inline Dockerfile"], + ["dockerfile_location", "a non-default Dockerfile location"], + ["watch_paths", "watch paths (which changes trigger a deploy)"], + ["redirect", "a www/non-www redirect policy"], + ]; + for (const [field, what] of NO_HOME) { + const v = r.raw[field]; + if (v === undefined || v === null || v === "") continue; + flag(field, `${what} is set on the box. The manifest has no field for it.`); + } + + // `port` is one number in a manifest and a comma-separated list on the wire. + // The draft writes the first and says so — a rebuilt app exposing one of the + // three ports it used to is the kind of difference that surfaces as a broken + // healthcheck weeks later. + const exposed = String(r.raw.ports_exposes ?? "") + .split(",") + .filter(Boolean); + if (exposed.length > 1) { + flag( + "port", + `the box exposes ${exposed.join(", ")}; a manifest declares ONE port, so only ${exposed[0]} is in the draft.`, + ); + } + + return { + source: { + repo: repo ?? String(r.raw.git_repository ?? ""), + branch: typeof branch === "string" && branch ? branch : "main", + }, + build: { + pack, + base_directory: String(r.raw.base_directory ?? "/"), + ...(compose + ? { + compose_file: String( + r.raw.docker_compose_location ?? "/docker-compose.yaml", + ), + } + : {}), + ...(!compose && r.raw.publish_directory + ? { publish_directory: String(r.raw.publish_directory) } + : {}), + }, + ...(compose + ? {} + : { + ...(r.raw.ports_exposes + ? { port: Number(String(r.raw.ports_exposes).split(",")[0]) } + : {}), + ...(r.raw.health_check_path + ? { healthcheck: String(r.raw.health_check_path) } + : {}), + domains: fqdn, + }), + ...(compose + ? { service_domains: composeDomains(r, project, uncaptured) } + : {}), + ...(hasEnv + ? { env_template: `env/${slug(r.name)}.${ctx.env}.env.template` } + : {}), + }; +} + +function composeDomains( + r: DraftResource, + project: string, + uncaptured: UncapturedItem[], +): Record { + const raw = r.raw.docker_compose_domains; + const map: Record = {}; + if (typeof raw === "string" && raw !== "") { + try { + const parsed = JSON.parse(raw); + if (Array.isArray(parsed)) { + for (const e of parsed) { + const name = (e as { name?: unknown })?.name; + const domain = (e as { domain?: unknown })?.domain; + if (typeof name === "string" && typeof domain === "string") { + map[name] = domain.split(",").filter(Boolean); + } + } + } + } catch { + // Unreadable, not absent. Say so rather than write {} and move on. + } + } + if (Object.keys(map).length === 0) { + uncaptured.push({ + project, + resource: r.name, + setting: "service_domains", + detail: + "this compose application exposes no readable per-container domains; `service_domains: {}` was written. A rebuilt stack would serve no hostnames until they are declared.", + }); + } + return map; +} + +function databaseSpec( + r: DraftResource, + project: string, + uncaptured: UncapturedItem[], +): Spec { + const flag = (setting: string, detail: string) => + uncaptured.push({ project, resource: r.name, setting, detail }); + const rawType = String(r.raw.database_type ?? r.raw.type ?? ""); + const type = + rawType === "standalone-postgresql" + ? "postgresql" + : rawType === "standalone-redis" + ? "redis" + : rawType; + const image = typeof r.raw.image === "string" ? r.raw.image : undefined; + const version = image?.split(":")[1]?.match(/^(\d+(?:\.\d+)*)/)?.[1]; + if (image && !version) { + flag( + "version", + `image "${image}" — no version could be read from its tag, so none was written and \`apply\` would create this database on Coolify's default image.`, + ); + } + // Coolify exposes no backup schedule on a database's GET, so a `backup:` block + // cannot be recovered. It is create-time-only in cast (see README's known + // limitations), so a rebuild from this draft would come up with NO BACKUPS — + // the quietest possible loss, and the one you discover at the worst moment. + flag( + "backup", + "backup schedules are not exposed by Coolify's API and are NOT in this draft. If this database is backed up, a rebuild from here would not be. Check the Coolify UI (Backups tab) and declare `backup: { frequency, retention }` yourself.", + ); + return { type, ...(version ? { version } : {}) }; +} + +function serviceSpec( + r: DraftResource, + ctx: DraftContext, + project: string, + hasEnv: boolean, + uncaptured: UncapturedItem[], +): Spec { + // Coolify 4.1.2's Service model carries no flat `domains` — hostnames live + // per-container on `service.applications[].fqdn`, which no endpoint cast uses + // returns (see projectLiveFields / serviceApiFields in cli.ts, and + // desiredFromManifest's warning). Not readable, not writable, not in the + // draft: a service that serves a hostname today would come back serving none. + uncaptured.push({ + project, + resource: r.name, + setting: "domains (hostnames)", + detail: + "Coolify 4.1.2 exposes no flat `domains` on a service — hostnames live per-container on service.applications[].fqdn, which cast can neither read nor write. Whatever hostnames this service answers on are NOT in this draft. Read them off the Coolify UI and set them there after a rebuild.", + }); + return { + type: String(r.raw.service_type ?? r.raw.type ?? ""), + ...(hasEnv + ? { env_template: `env/${slug(r.name)}.${ctx.env}.env.template` } + : {}), + }; +} + +// --- Secrets ----------------------------------------------------------------- +// +// Every live var becomes a `${REF}` in the template, and its value goes to the +// age store. NOTHING is written as a template literal. +// +// That is a deliberate one-way bet. cast cannot know which of a box's vars are +// secret — nobody wrote it down, which is why this verb exists — and the two +// mistakes are not symmetric: a non-secret in the encrypted store is untidy, and +// a live API key written as a literal into a manifest is a key in a git repo. +// So: no heuristic decides where a VALUE goes. Only where a NAME goes (see +// isProviderGenerated), and that decision withholds the value rather than +// publishing it. +function planSecrets( + p: DraftProject, + uncaptured: UncapturedItem[], +): { + // resource name -> [KEY, ref][] + templates: Map>; + dispositions: DraftDisposition[]; + generated: string[]; +} { + const templates = new Map>(); + // key -> sites, and the distinct values seen for it across the project + const byKey = new Map>(); + + for (const r of p.resources) { + const usable: Array<[string, string]> = []; + for (const [key, value] of Object.entries(r.env)) { + if (!TEMPLATE_KEY.test(key)) { + // A cast template cannot hold this name at all — one grammar, shared by + // every reader of a template (envtemplate.ts). Dropping it quietly would + // rebuild the resource without a var it has today. + uncaptured.push({ + project: p.name, + resource: r.name, + setting: `env var ${key}`, + detail: `this box sets an env var named "${key}", which is not a name a cast env template can express (KEY must match ${TEMPLATE_KEY.source}). It is NOT in this draft.`, + }); + continue; + } + const sites = byKey.get(key) ?? []; + sites.push({ resource: r.name, value }); + byKey.set(key, sites); + usable.push([key, ""]); + } + if (usable.length > 0) templates.set(r.name, usable); + } + + const dispositions: DraftDisposition[] = []; + const generated: string[] = []; + const refOf = new Map(); // `${resource}::${key}` -> ref + + for (const [key, sites] of byKey) { + const provenance: Provenance = isProviderGenerated(key) + ? "generated" + : "captured"; + // A provider-generated name is placeheld everywhere it appears, so two + // resources disagreeing about its value is not a conflict cast has to + // resolve — neither value is being carried. + const distinct = new Set(sites.map((s) => s.value)); + const split = provenance === "captured" && distinct.size > 1; + if (split) { + // The store holds ONE value per name (see classify's CONFLICT refusal), and + // this key carries two. cast will not pick — so it does not: each site gets + // its own ref, both values survive, and the reviewer collapses them if they + // were meant to be the same thing. + uncaptured.push({ + project: p.name, + setting: `env var ${key}`, + detail: `${sites.map((s) => `"${s.resource}"`).join(" and ")} each set ${key}, to DIFFERENT values. One store holds one value per name, so cast split them into ${sites.map((s) => `${constCase(s.resource)}_${key}`).join(" and ")} rather than pick. Collapse them if they were meant to be one.`, + }); + } + for (const s of sites) { + const ref = split ? `${constCase(s.resource)}_${key}` : key; + refOf.set(`${s.resource}::${key}`, ref); + if (!split && dispositions.some((d) => d.ref === ref)) { + // Same ref, same value, second site: record the site, not a second entry. + const d = dispositions.find((x) => x.ref === ref); + d?.sites.push(`${s.resource}.${key}`); + continue; + } + dispositions.push({ + project: p.name, + ref, + provenance, + sites: [`${s.resource}.${key}`], + // THE line this whole file is bent around: a provider-generated name is + // placeheld with the same literal `capture` writes, and the source box's + // value is not written anywhere — not into a template, not into a store, + // not into a log. + value: provenance === "generated" ? GENERATED_PLACEHOLDER : s.value, + }); + if (provenance === "generated" && !generated.includes(ref)) + generated.push(ref); + } + } + + for (const [resource, pairs] of templates) { + templates.set( + resource, + pairs.map(([key]) => [key, refOf.get(`${resource}::${key}`) ?? key]), + ); + } + return { templates, dispositions, generated: generated.sort() }; +} + +// --- Uncaptured, the parts that are not per-resource --------------------------- + +function placementItems(p: DraftProject, uncaptured: UncapturedItem[]): void { + const ids = new Set( + p.resources + .map((r) => r.raw.destination_id) + .filter((v): v is number => typeof v === "number"), + ); + if (ids.size === 0) return; + // #21: `destination_id` is the ONLY thing Coolify tells us about placement — + // an integer primary key. `destination_uuid:` (the binding `apply` needs) takes + // the UUID, and Coolify 4.1.2 has no destinations API at all, so cast cannot + // resolve one to the other. On a server with one destination this is inert; on + // a server with two — which is exactly the box you are draining — it decides + // which Docker network a resource can reach, and getting it wrong builds a + // stack whose app cannot see its own database. + uncaptured.push({ + project: p.name, + setting: "destination (Docker network)", + detail: `these resources sit on destination_id ${[...ids].sort().join(", ")}${ids.size > 1 ? " — MORE THAN ONE, so this project is split across Docker networks" : ""}. cast cannot turn that integer into the UUID \`environments..projects..destination_uuid\` needs: Coolify 4.1.2 has no destinations API (see reference/README.md). Read the UUID off the Coolify UI and bind it yourself, or a rebuild lands on whichever network Coolify picks first.`, + }); +} + +const NO_API_COVERAGE: Array<[string, string]> = [ + [ + "destinations", + "Coolify 4.1.2 serves no destinations endpoint. A resource's `destination_id` comes back; the UUID that names it never does. Placement must be read from the UI (#21).", + ], + [ + "service hostnames", + "no flat `domains` on a service — they live per-container on `service.applications[].fqdn`, which cast can neither read nor write (Coolify 4.1.2).", + ], + [ + "Basic Auth / custom Traefik labels", + "carried as raw container labels. cast's manifest has no field for them, so a rebuilt resource is UNPROTECTED where the original was not.", + ], + [ + '"Include Source Commit in Build"', + "and its neighbours on a resource's Settings tab: no API coverage in 4.1.2, and not returned by the endpoints cast reads.", + ], + [ + "backup schedules", + "a database's backup config is not exposed on its GET. A rebuild from this draft has NO backups until you declare them.", + ], + [ + "database kinds cast does not model", + "MySQL, MariaDB, MongoDB, KeyDB, Dragonfly, ClickHouse. cast's manifest speaks postgresql and redis only; any of the others found on the box are named above and are NOT in this draft.", + ], + [ + "which GitHub App clones a repo", + "nothing Coolify returns about an application says so. cast binds every repo to the instance's only App when there is exactly one, and writes a REVIEW marker when there is not.", + ], + [ + "anything configured in the UI with no manifest field", + "this list is what cast KNOWS it cannot express. It is not a proof that nothing else is missing.", + ], +]; + +// The table that stops "rebuild from the repo" being over-claimed. Two of these +// six rows are `❌`, and they are the two that decide whether a DR runbook is +// true: they are not in the repo (correctly — it holds no live credentials) and +// they cannot be regenerated from it. +const CANNOT_RESTORE: Array<[string, string]> = [ + ["control plane", "`rig coolify install` ✅"], + ["structure", "this draft → a manifest PR → `apply` ✅"], + ["secret **values**", "the age store + your key ✅"], + ["**data**", "Coolify's DB backups → S3 ✅ (a separate path — not this one)"], + [ + "**the GitHub App private key**", + "❌ re-create by hand. It is not in the repo and cannot be regenerated from it.", + ], + [ + "**S3 access keys**", + "❌ re-mint by hand. Same reason: the repo holds no live credentials.", + ], +]; + +export function renderUncaptured( + items: UncapturedItem[], + projects: DraftProject[], + ctx: DraftContext, +): string { + const lines = [ + "# UNCAPTURED — what this draft does NOT carry", + "", + // Fenced, not quoted: the header is aligned text, and a markdown blockquote + // would reflow it into one paragraph. + "```", + ...header( + ctx, + ["This file is the reason the draft beside it is allowed to exist."], + { self: true }, + ), + "```", + "", + "cast cannot express everything a Coolify holds. A blueprint that omits those", + "things **without saying so** is worse than no blueprint at all — in a disaster", + "you would trust it, rebuild from it, and get a *different box*, without ever", + "being told. So everything cast saw on this instance and could not write down is", + "listed here, by resource. Nothing on this page is in the draft.", + "", + "## Seen on this box, not in the draft", + "", + ]; + + if (items.length === 0) { + lines.push( + "Nothing — cast expressed every setting it could see on every resource it", + "drafted. That is a statement about what cast can SEE (the sections below say", + "what it cannot), not a clean bill of health.", + "", + ); + } + + for (const p of projects) { + const mine = items.filter((i) => i.project === p.name); + if (mine.length === 0) continue; + lines.push(`### ${p.name} — Coolify environment \`${p.coolifyEnv}\``, ""); + const projectWide = mine.filter((i) => !i.resource); + for (const i of projectWide) { + lines.push(`- **${i.setting}** — ${i.detail}`); + } + if (projectWide.length > 0) lines.push(""); + const resources = [...new Set(mine.map((i) => i.resource))].filter( + (r): r is string => r !== undefined, + ); + for (const resource of resources) { + lines.push(`**${resource}**`, ""); + for (const i of mine.filter((x) => x.resource === resource)) { + lines.push(`- \`${i.setting}\` — ${i.detail}`); + } + lines.push(""); + } + if (p.otherEnvironments.length > 0) { + lines.push( + `- **other environments** — this project also has ${p.otherEnvironments + .map((e) => `\`${e.name}\` (${e.resources} resource(s))`) + .join( + ", ", + )} on the box. This draft carries \`${p.coolifyEnv}\` only.`, + "", + ); + } + } + + lines.push( + "## Cannot be seen at all — no API coverage in Coolify 4.1.2", + "", + "Not gaps in this draft: gaps in the API it was read through. Nothing cast can", + "do would recover these, so a rebuild must set them by hand.", + "", + "| setting | why |", + "| --- | --- |", + ...NO_API_COVERAGE.map(([k, v]) => `| ${k} | ${v} |`), + "", + "## What a rebuild from this draft still cannot restore", + "", + "| | |", + "| --- | --- |", + ...CANNOT_RESTORE.map(([k, v]) => `| ${k} | ${v} |`), + "", + "The last two rows are the ones a DR runbook has to say out loud. They are not", + "in the state repo — correctly, it holds no live credentials — and they cannot be", + "regenerated from it. Re-create them by hand, and expect to.", + "", + "## How values were treated", + "", + "Every live env var in this draft became a `${REF}` in a template, with its", + "value in the age store — never a literal in a committed file. Names that look", + "**provider-generated** (Coolify's `SERVICE_*` magic vars; anything carrying a", + "datastore word and a connection word, like `DATABASE_URL` or `DB_HOST`) were", + `**placeheld** with \`${GENERATED_PLACEHOLDER}\` and their live values were NOT`, + "read into any artifact: a `DATABASE_URL` copied off this box points at THIS", + "box's Postgres, and a rebuilt box carrying it would come up *working*, reading", + "and writing the old box's database — until the day the old box is deleted.", + "", + "That rule is by NAME. A var that points at this box under a name cast does not", + "recognize **will have been copied**. The disposition table printed at the end of", + "the run is the list to read.", + "", + ); + return lines.join("\n"); +} + +// --- The plan ---------------------------------------------------------------- + +export function planDraft( + projects: DraftProject[], + ctx: DraftContext, +): DraftPlan { + const files: DraftFile[] = []; + const stores: DraftPlan["stores"] = []; + const dispositions: DraftDisposition[] = []; + const uncaptured: UncapturedItem[] = []; + + for (const p of projects) { + const dir = projectDir(p); + for (const u of p.unreadable) { + uncaptured.push({ + project: p.name, + resource: u.name, + setting: "the whole resource", + detail: `a ${u.kind} — cast's manifest speaks postgresql and redis only. It is NOT in this draft, and a rebuild from here would not have it.`, + }); + } + // Nothing to express: no manifest, no store, and NO REGISTRY ENTRY — a + // registry that claimed a project this draft carries nothing for would send + // every future fleet run at a project with no manifest to run against. It is + // still named, in UNCAPTURED.md, because it is on the box. + if (p.resources.length === 0) { + uncaptured.push({ + project: p.name, + setting: "the whole project", + detail: `nothing was drafted from it: ${p.skipReason ?? "it has no resources cast can express"}. It EXISTS on this box${p.otherEnvironments.length > 0 ? "" : " and is not in this draft"}.`, + }); + continue; + } + placementItems(p, uncaptured); + + const secrets = planSecrets(p, uncaptured); + dispositions.push(...secrets.dispositions); + + const applications: Record = {}; + const databases: Record = {}; + const services: Record = {}; + for (const r of p.resources) { + const hasEnv = secrets.templates.has(r.name); + if (r.kind === "application") { + const spec = applicationSpec(r, ctx, p.name, hasEnv, uncaptured); + if (spec) applications[r.name] = spec; + } else if (r.kind === "database") { + databases[r.name] = databaseSpec(r, p.name, uncaptured); + } else { + services[r.name] = serviceSpec(r, ctx, p.name, hasEnv, uncaptured); + } + } + + const manifest = { + // The BOX's project name, not a slug: it is what `--project` takes, and the + // one string that connects this file to the UI it was read from. + project: p.name, + environments: { + [ctx.env]: { + applications, + ...(Object.keys(databases).length > 0 ? { databases } : {}), + ...(Object.keys(services).length > 0 ? { services } : {}), + ...(secrets.generated.length > 0 + ? { generated_secrets: secrets.generated } + : {}), + }, + }, + }; + files.push({ + path: join(dir, ".infra", "manifest.yaml"), + content: `${comment( + header(ctx, [ + ` project: "${p.name}"`, + ` environment: "${p.coolifyEnv}" on the box → filed here under \`${ctx.env}\`, YOUR name for it`, + "", + "Resources keep the names the BOX gives them. They are what someone typed into", + "a UI, and renaming them here would make this file unusable against that UI —", + "so a rename is a review decision, made with `--resource =` on", + "the read side until the names agree.", + ]), + )}\n\n${stringify(manifest)}`, + }); + + for (const [resource, pairs] of secrets.templates) { + files.push({ + path: join( + dir, + ".infra", + "env", + `${slug(resource)}.${ctx.env}.env.template`, + ), + content: `${comment( + header(ctx, [ + ` resource: "${resource}"`, + "", + "Every live var is a ${REF}: the VALUES are in the age store, never here. cast", + "cannot know which of a box's vars are secret — nobody wrote it down, which is", + "why this verb exists — and a live key written as a literal is a key in a git", + "repo. Move the ones that are plainly not secret back to literals yourself.", + ]), + )}\n${pairs.map(([key, ref]) => `${key}=\${${ref}}`).join("\n")}\n`, + }); + } + + const vars = Object.fromEntries( + secrets.dispositions.map((d) => [d.ref, d.value]), + ); + if (Object.keys(vars).length > 0) { + stores.push({ + project: p.name, + path: join("secrets", `${storeKey(p)}.${ctx.env}.env.age`), + vars, + }); + } + } + + files.push({ + path: "environments.yaml", + content: `${comment( + header(ctx, [ + "The state file this instance implies — bindings as far as they can be READ, plus", + "`projects:`, the registry: the list of what exists, which nothing before a", + "whole-instance sweep was able to write down.", + "", + "`github_apps` is NOT readable from a box: nothing Coolify returns about an", + `application says which App clones it. This instance has ${ctx.githubApps?.length ?? 0}, so cast`, + ctx.githubApps?.length === 1 + ? `bound every repo to the only one there is (${ctx.githubApps[0]}) — there is no other it could be.` + : "left a REVIEW marker on every repo rather than pick. `apply` will refuse until you fix them.", + "", + "Do not copy it over a state file you already have. Merge the registry into", + "yours, by hand, having decided which of these projects are yours to declare.", + ]), + )}\n\n${stringify(bindingsDoc(projects, ctx))}`, + }); + + files.push({ + path: "UNCAPTURED.md", + content: renderUncaptured(uncaptured, projects, ctx), + }); + + return { files, stores, dispositions, uncaptured }; +} + +// The repo short name when the box knows the repo, else the project's slug — +// `incubator/`, `la-familia-site/`. Only a path; nothing resolves by it. +function projectDir(p: DraftProject): string { + const repo = registryKey(p); + const short = repo.includes("/") ? repo.split("/")[1] : repo; + return slug(short); +} + +const storeKey = (p: DraftProject) => projectDir(p); + +// The `/` slug the registry is keyed by — and the box only knows it +// through an APPLICATION's git remote. A project with no application (a lone +// service, a database somebody made once) has no repo on the box at all, so cast +// writes the bare project name and the registry's own parse-time refusal ("a +// registry key has no meaning without its org", #25) stops the file being used +// until a human supplies the org. +// +// That refusal is the right outcome, and deliberately not worked around: the +// alternatives are inventing an org, or leaving the project out of the list — and +// a project missing from the registry is a project every fleet run skips in +// silence, which reads exactly like a clean one. +function registryKey(p: DraftProject): string { + for (const r of p.resources) { + if (r.kind !== "application") continue; + const repo = repoFromGitUrl(r.raw.git_repository); + if (repo) return repo; + } + return slug(p.name); +} + +// The bindings the box implies — plus `projects:`, THE REGISTRY (#25): the list +// of what exists, which nothing before a whole-instance sweep was in a position +// to write down. A rebuild cannot even be attempted without it, because you +// cannot restore what you cannot enumerate. +function bindingsDoc(projects: DraftProject[], ctx: DraftContext) { + const registry: Record = {}; + const githubApps: Record = {}; + // With exactly one GitHub App on the instance there is no other one an + // application could have been cloned by, so binding every repo to it is a fact, + // not a guess. With none or several it IS a guess, and cast does not make it: + // a wrong App resolves to a real uuid and clones the wrong repo, silently + // (githubAppNameFor, #12). A REVIEW marker resolves to nothing, and `apply` + // says so. + const onlyApp = ctx.githubApps?.length === 1 ? ctx.githubApps[0] : undefined; + for (const p of projects) { + // Only what the draft actually carries a manifest for. See planDraft. + if (p.resources.length === 0) continue; + const repo = registryKey(p); + registry[repo] = { environments: [ctx.env] }; + githubApps[repo] = + onlyApp ?? "REVIEW-which-github-app-in-coolify-clones-this-repo"; + } + return { + environments: { + [ctx.env]: { + // NOT readable off a resource: Coolify's environment_details response + // carries no server. What is written here is the server the environment + // you swept UNDER is bound to — a fact about your state file, not about + // this box. With no binding to read it from, it is left for review, and + // cast's own schema will refuse the file until it is filled in. + server: ctx.server ?? "REVIEW-no-server-is-readable-from-a-live-box", + team: { id: ctx.team.id, name: ctx.team.name }, + instance: ctx.instance, + ...(ctx.recipient ? { age_recipient: ctx.recipient } : {}), + }, + }, + projects: registry, + github_apps: githubApps, + }; +} + +// --- Refusals ---------------------------------------------------------------- + +// A draft is emitted into a NEW directory, and only ever into one. +// +// The refusal is not tidiness. `--emit-draft .` inside a product repo would write +// a manifest generated from a live box straight over the reviewed one — and a +// manifest regenerated from a box carries everything that box has accumulated +// that nobody meant. Adoption is one-way: the repo is the truth for a project it +// already declares, and the draft is a proposal for one it does not. +export function assertEmptyTarget(dir: string): void { + if (!existsSync(dir)) return; + const entries = readdirSync(dir); + if (entries.length === 0) return; + throw new Error( + [ + `refusing to emit a draft: ${dir} is not empty`, + "", + " looked for: an empty or non-existent directory", + ` found: ${entries.slice(0, 8).join(", ")}${entries.length > 8 ? `, … (${entries.length} entries)` : ""}`, + "", + "A draft is a PROPOSAL, machine-generated from a live box, and it is written", + "into a directory of its own so that it can be READ before any of it is", + "believed. Emitted over a repo that already has a manifest, it would overwrite", + "a reviewed spec with whatever that box has accumulated — the one direction", + "nobody reviews. Adoption is one-way.", + "", + "Point --emit-draft at a new directory, and land what survives review as a PR.", + ].join("\n"), + ); +} + +// The same rule, once more at the file. Unreachable through the CLI (an empty +// target cannot hold a manifest) and kept anyway: it is the invariant, not the +// check that happens to enforce it today, and the next caller of emitDraft will +// not have read assertEmptyTarget. +export function assertNoExistingManifest(path: string): void { + if (!existsSync(path)) return; + throw new Error( + [ + `refusing to emit a draft: ${path} already exists`, + "", + "For a declared project the manifest IS the truth. Regenerating it from a live", + "box would let that box's cruft overwrite a reviewed spec, silently. A draft is", + "for a project that has NO manifest; for one that has, `cast inventory /`", + "reconciles the two and you decide, key by key, what the manifest should gain.", + ].join("\n"), + ); +} + +// `--emit-draft` with a repo positional. Not an argument-parsing nicety: the two +// flags mean opposite things about the same box, and the combination is the one +// that would do damage. +export function renderRepoWithDraft(orgRepo: string, dir: string): string { + return [ + "refusing to emit a draft: --emit-draft is a SWEEP-mode flag, and a repo was given", + "", + ` looked at: ${orgRepo}`, + ` emitting to: ${dir}`, + "", + "With a repo, `inventory` reconciles a box against a manifest that ALREADY EXISTS.", + "That is precisely the case where a draft must not be written: for a declared", + "project the manifest is the truth, and one regenerated from a live box would", + "carry back everything that box has accumulated that nobody meant — over the top", + "of a reviewed spec, in the one direction nobody reviews. Adoption is one-way.", + "", + "To reconcile a project that has a manifest:", + "", + ` cast inventory ${orgRepo} --env [--project ] [--environment ]`, + "", + "To draft the projects that have none:", + "", + ` cast inventory --env --emit-draft ${dir}`, + ].join("\n"); +} + +// No recipient, and no explicit opt-out. Refuse — never quietly emit a draft +// with no store in it. +// +// A draft whose secrets were silently skipped looks COMPLETE: a manifest, env +// templates full of ${REF}s, an UNCAPTURED.md — and not one value anywhere. You +// would find out when `apply` refused for want of a store, which is some time +// after the box those values were on stopped existing. +export function renderNoRecipient(envName: string): string { + return [ + `refusing to emit a draft: no age recipient for ${envName}`, + "", + ` looked for: --recipient, then environments.${envName}.age_recipient`, + "", + "The draft's stores are encrypted to a recipient you NAME. cast will not skip", + "them for you: a draft with templates full of ${REF}s and no store behind them", + "looks complete, and the values it did not write are on a box you are about to", + "stop paying for.", + "", + "Name one:", + "", + " cast inventory --env --emit-draft --recipient age1…", + "", + "or bind it (it is the public half — safe to commit next to the bindings):", + "", + " environments:", + ` ${envName}:`, + " age_recipient: age1…", + "", + "or say, explicitly, that you want the structure without the values:", + "", + " cast inventory --env --emit-draft --no-secrets", + ].join("\n"); +} + +// A project with resources in two environments. cast will not pick one. +// +// Picking would produce a blueprint of HALF A BOX that says nothing about the +// other half — the exact artifact this verb exists to not produce. And the +// likeliest split is the one that has already bitten this project once: a +// Coolify-auto-created `production` beside the `staging` where everything +// actually runs (#22). +export function renderAmbiguousEnvironments( + projects: Array<{ name: string; environments: string[] }>, + envName: string, +): string { + return [ + `refusing to emit a draft: ${projects.length} project(s) have resources in MORE THAN ONE environment`, + "", + ...projects.flatMap((p) => [ + ` ${p.name}`, + ...p.environments.map((e) => ` ${e}`), + ]), + "", + "A draft carries ONE environment per project. Picking for you would emit a", + "blueprint of half a box that says nothing about the other half — and the box", + "you are looking at is the one where that already happened once: Coolify", + "auto-creates `production` in every project, so the environment things actually", + "run in is whatever someone typed instead.", + "", + "Say which:", + "", + ` cast inventory --env ${envName} --emit-draft --environment `, + "", + "and run it once per environment you mean to keep, into a directory each.", + ].join("\n"); +} + +// --- Emit -------------------------------------------------------------------- + +export function emitDraft( + dir: string, + plan: DraftPlan, + opts: { recipient?: string }, +): string[] { + assertEmptyTarget(dir); + const written: string[] = []; + for (const f of plan.files) { + const path = join(dir, f.path); + if (path.endsWith("manifest.yaml")) assertNoExistingManifest(path); + mkdirSync(dirname(path), { recursive: true }); + writeFileSync(path, f.content); + written.push(f.path); + } + if (opts.recipient) { + for (const s of plan.stores) { + const path = join(dir, s.path); + mkdirSync(dirname(path), { recursive: true }); + encryptSecrets(opts.recipient, path, s.vars); + written.push(s.path); + } + } + return written; +} + +// --- The plan, printed ------------------------------------------------------- +// +// Names and provenance. NEVER values — same contract as renderCapturePlan, and +// for the same reason: this output is meant to be pasted into a PR discussion. +// The one value-shaped thing here is the GENERATED_PLACEHOLDER literal, which +// carries no information about the box. +export function renderDraftPlan( + plan: DraftPlan, + ctx: DraftContext, + opts: { dir: string; recipient?: string; written: string[] }, +): string { + const lines = [ + "", + `draft — emitted to ${opts.dir}`, + "", + ` source: instance ${ctx.instance} (${ctx.baseUrl}) — read LIVE`, + ` filed as: environment ${ctx.env}`, + ` secrets: ${opts.recipient ? `encrypted to ${opts.recipient}` : "NOT WRITTEN (--no-secrets)"}`, + "", + ...opts.written.map((w) => ` + ${w}`), + "", + ]; + + if (plan.dispositions.length > 0) { + const width = Math.max(...plan.dispositions.map((d) => d.ref.length)); + lines.push( + "every value cast read, and what it did with it — names and provenance, never values:", + "", + ); + for (const d of [...plan.dispositions].sort((a, b) => + a.project === b.project + ? a.ref.localeCompare(b.ref) + : a.project.localeCompare(b.project), + )) { + const note = + d.provenance === "generated" ? ` → ${GENERATED_PLACEHOLDER}` : ""; + lines.push( + ` ${d.ref.padEnd(width)} ${d.provenance.padEnd(9)} ${d.sites.join(", ")}${note}`, + ); + } + const generated = plan.dispositions.filter( + (d) => d.provenance === "generated", + ).length; + lines.push( + "", + `${plan.dispositions.length} name(s): ${plan.dispositions.length - generated} captured, ${generated} placeheld as provider-generated.`, + "", + "A placeheld name's LIVE VALUE WAS NOT READ INTO ANY FILE. A DATABASE_URL copied", + "off this box points at THIS box's Postgres: a rebuilt box carrying it comes up", + "working, against the old box's database, and you find out the day the old box is", + "deleted. The rule is by NAME — read the captured list above and decide whether", + "anything in it is really a pointer at this box.", + "", + ); + } + + lines.push( + `${plan.uncaptured.length} setting(s) cast SAW and could not express — every one is in ${join(opts.dir, "UNCAPTURED.md")}.`, + "Read it before you treat this as a blueprint.", + "", + "This is a PROPOSAL. `apply` does not read it. The path from here:", + "", + " review it → land the manifest in the product repo as a PR → capture → apply", + ); + return lines.join("\n"); +} diff --git a/test/draft-cli.test.ts b/test/draft-cli.test.ts new file mode 100644 index 0000000..986f7d1 --- /dev/null +++ b/test/draft-cli.test.ts @@ -0,0 +1,574 @@ +import { execFileSync, spawn } from "node:child_process"; +import { + existsSync, + mkdirSync, + mkdtempSync, + readFileSync, + readdirSync, + writeFileSync, +} from "node:fs"; +import { createServer } from "node:http"; +import type { AddressInfo } from "node:net"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { afterEach, beforeAll, describe, expect, it } from "vitest"; +import { loadBindings } from "../src/bindings.js"; +import { GENERATED_PLACEHOLDER } from "../src/capture.js"; +import { decryptSecrets } from "../src/secrets.js"; + +// `cast inventory --emit-draft` against a stub shaped like the box that made it +// necessary: a Coolify nobody declared, holding our stack under names someone +// typed, two third-party client sites, and a database cast cannot even model. +// +// The single most important assertion in this file is that POISON — the source +// box's real DATABASE_URL — appears in NO emitted artifact. A draft that carried +// it would rebuild a box that comes up WORKING, reading and writing the old +// box's database, and nobody would find out until the old box was deleted. + +// The live values. If any of these reaches an emitted file, the test fails. +const POISON = + "postgres://postgres:s3cr3t@incubator-database-v2.box-b.internal:5432/app"; +const POISON_REDIS = "redis://:r3d1s@incubator-redis.box-b.internal:6379/0"; +const POISON_SERVICE_PW = "umami-generated-9f3a1c"; +// …and one that MUST survive: an ordinary secret is captured, not placeheld. +const MAILGUN = "key-1a2b3c-real-mailgun"; + +type Stub = { url: string; close: () => Promise }; +const stubs: Stub[] = []; + +const ENVS: Record> = { + a1: [ + { key: "DATABASE_URL", value: POISON }, + { key: "REDIS_URL", value: POISON_REDIS }, + { key: "MAILGUN_KEY", value: MAILGUN }, + { key: "NODE_ENV", value: "production" }, + // Not a name a cast env template can hold. Reported, never dropped in silence. + { key: "legacy.flag", value: "on" }, + ], + s1: [ + { key: "SERVICE_PASSWORD_UMAMI", value: POISON_SERVICE_PW }, + { key: "SERVICE_FQDN_UMAMI", value: "https://umami.box-b.example.com" }, + { key: "UMAMI_APP_SECRET", value: "umami-app-secret-xyz" }, + ], + a9: [{ key: "WP_HOME", value: "https://lafamilia.example.com" }], +}; + +// One project with resources in TWO environments — cast must refuse to pick. +async function stubCoolify(opts: { ambiguous?: boolean } = {}): Promise { + const server = createServer((req, res) => { + const path = (req.url ?? "").replace("/api/v1", ""); + const json = (body: unknown) => { + res.writeHead(200, { "content-type": "application/json" }); + res.end(JSON.stringify(body)); + }; + if (path === "/teams/current") return json({ id: 0, name: "Root Team" }); + // Exactly one App: there is no other one an application could have been + // cloned by, so cast binds every repo to it rather than leave a marker. + if (path === "/github-apps") + return json([{ uuid: "g1", name: "hdb-coolify" }]); + if (path === "/projects") + return json([ + { uuid: "p1", name: "Incubator" }, + { uuid: "p2", name: "La Familia Site" }, + { uuid: "p3", name: "Martin Reyes Barber Shop" }, + ]); + if (path === "/projects/p1/environments") + return json([{ name: "production" }, { name: "staging" }]); + if (path === "/projects/p2/environments") + return json([{ name: "production" }]); + if (path === "/projects/p3/environments") + return json([{ name: "production" }]); + + // Coolify auto-creates `production`. It is empty (unless the ambiguous + // variant puts a resource in it too). Everything real lives in `staging`. + if (path === "/projects/p1/production") + return json( + opts.ambiguous + ? { applications: [{ name: "old-core", uuid: "a7", fqdn: "" }] } + : {}, + ); + if (path === "/projects/p1/staging") + return json({ + applications: [ + { + name: "Incubator Stack v2", + uuid: "a1", + git_repository: "heavy-duty/incubator", + git_branch: "main", + build_pack: "dockercompose", + base_directory: "/", + docker_compose_location: "/docker-compose.yaml", + docker_compose_domains: JSON.stringify([ + { name: "core", domain: "https://app.example.com" }, + ]), + destination_id: 3, + // Basic Auth lives here, and cast's manifest has no field for it. + custom_labels: + "traefik.http.middlewares.auth.basicauth.users=admin:$2y$05$x", + }, + ], + postgresqls: [ + { + name: "Incubator Database v2", + uuid: "d1", + database_type: "standalone-postgresql", + image: "postgres:16-alpine", + destination_id: 3, + }, + ], + services: [ + { + name: "Incubator Umami", + uuid: "s1", + service_type: "umami", + destination_id: 3, + }, + ], + // A database cast's manifest cannot express at all. + mysqls: [{ name: "legacy-analytics", uuid: "m1" }], + }); + if (path === "/projects/p2/production") + return json({ + applications: [ + { + name: "lafamilia-web", + uuid: "a9", + git_repository: "https://github.com/third-party/la-familia.git", + git_branch: "main", + build_pack: "static", + base_directory: "/", + publish_directory: "/dist", + fqdn: "https://lafamilia.example.com", + destination_id: 4, + }, + ], + }); + // A project with no application at all: nothing on the box knows its repo. + if (path === "/projects/p3/production") + return json({ + services: [ + { name: "barber-site", uuid: "s9", service_type: "wordpress" }, + ], + }); + + const envs = path.match(/^\/[a-z]+\/([a-z0-9]+)\/envs$/); + if (envs) return json(ENVS[envs[1]] ?? []); + res.writeHead(404); + res.end("{}"); + }); + await new Promise((r) => { + server.listen(0, "127.0.0.1", r); + }); + const stub: Stub = { + url: `http://127.0.0.1:${(server.address() as AddressInfo).port}`, + close: () => + new Promise((r) => { + server.close(() => r()); + }), + }; + stubs.push(stub); + return stub; +} + +afterEach(async () => { + await Promise.all(stubs.splice(0).map((s) => s.close())); +}); + +let KEY_FILE = ""; +let RECIPIENT = ""; + +beforeAll(() => { + const dir = mkdtempSync(join(tmpdir(), "cast-age-")); + KEY_FILE = join(dir, "key.txt"); + execFileSync("age-keygen", ["-o", KEY_FILE], { stdio: "ignore" }); + RECIPIENT = execFileSync("age-keygen", ["-y", KEY_FILE], { + encoding: "utf8", + }).trim(); +}); + +function fixture(url: string, opts: { recipient?: string } = {}) { + const state = mkdtempSync(join(tmpdir(), "cast-state-")); + writeFileSync( + join(state, ".coolify.env"), + `COOLIFY_BASE_URL="${url}"\nCOOLIFY_ACCESS_TOKEN="t"\n`, + ); + writeFileSync( + join(state, "environments.yaml"), + [ + "environments:", + " prod:", + " server: box-b", + " team: { id: 0, name: Root Team }", + ...(opts.recipient ? [` age_recipient: ${opts.recipient}`] : []), + "github_apps:", + " heavy-duty/incubator: hdb-coolify", + "", + ].join("\n"), + ); + const out = join(mkdtempSync(join(tmpdir(), "cast-out-")), "draft"); + return { state, out }; +} + +function run(args: string[]): Promise<{ code: number; output: string }> { + return new Promise((resolve) => { + const child = spawn("node", ["dist/cli.js", "inventory", ...args], { + stdio: ["pipe", "pipe", "pipe"], + }); + let output = ""; + child.stdout.on("data", (d) => { + output += String(d); + }); + child.stderr.on("data", (d) => { + output += String(d); + }); + child.stdin.end(); + child.on("close", (code) => resolve({ code: code ?? 0, output })); + }); +} + +// Every file in the emitted tree, path -> bytes as text. +function tree(dir: string, prefix = ""): Record { + const out: Record = {}; + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const path = join(dir, entry.name); + const rel = prefix ? `${prefix}/${entry.name}` : entry.name; + if (entry.isDirectory()) Object.assign(out, tree(path, rel)); + else out[rel] = readFileSync(path, "utf8"); + } + return out; +} + +describe("cast inventory --emit-draft (#27)", () => { + it("emits the tree: bindings + registry, a manifest per project, templates, stores, UNCAPTURED", async () => { + const f = fixture((await stubCoolify()).url); + const r = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--recipient", + RECIPIENT, + ]); + expect(r.code).toBe(0); + expect(Object.keys(tree(f.out)).sort()).toEqual([ + "UNCAPTURED.md", + "environments.yaml", + // One per project — INCLUDING the two third-party client sites nobody ever + // declared. They each sit alone in a Coolify-default `production`, and a + // draft that filtered the instance by one environment name would drop + // exactly them: the projects the verb exists to bootstrap. + "incubator/.infra/env/incubator-stack-v2.prod.env.template", + "incubator/.infra/env/incubator-umami.prod.env.template", + "incubator/.infra/manifest.yaml", + "la-familia/.infra/env/lafamilia-web.prod.env.template", + "la-familia/.infra/manifest.yaml", + "martin-reyes-barber-shop/.infra/manifest.yaml", + "secrets/incubator.prod.env.age", + "secrets/la-familia.prod.env.age", + ]); + // Every emitted file says what it is, in its own body. + for (const [path, body] of Object.entries(tree(f.out))) { + if (path.endsWith(".age")) continue; + expect(body, path).toContain("PROPOSAL"); + expect(body, path).toContain("apply` does not read"); + } + }); + + it("NEVER copies a provider-generated value — not into any artifact", async () => { + const f = fixture((await stubCoolify()).url); + const r = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--recipient", + RECIPIENT, + ]); + expect(r.code).toBe(0); + + // 1. Not in any emitted FILE. + for (const [path, body] of Object.entries(tree(f.out))) { + expect(body, path).not.toContain(POISON); + expect(body, path).not.toContain(POISON_REDIS); + expect(body, path).not.toContain(POISON_SERVICE_PW); + } + // 2. Not in the age store either, once decrypted — the store is the one place + // a copied value would actually be, and the one place you cannot grep. + const store = decryptSecrets( + join(f.out, "secrets", "incubator.prod.env.age"), + KEY_FILE, + ); + expect(store.DATABASE_URL).toBe(GENERATED_PLACEHOLDER); + expect(store.REDIS_URL).toBe(GENERATED_PLACEHOLDER); + expect(store.SERVICE_PASSWORD_UMAMI).toBe(GENERATED_PLACEHOLDER); + expect(store.SERVICE_FQDN_UMAMI).toBe(GENERATED_PLACEHOLDER); + expect(Object.values(store)).not.toContain(POISON); + // 3. Not on stdout. + expect(r.output).not.toContain(POISON); + + // …and an ORDINARY secret IS carried. The discipline is placeholding the + // provider's names, not refusing to capture anything. + expect(store.MAILGUN_KEY).toBe(MAILGUN); + expect(store.UMAMI_APP_SECRET).toBe("umami-app-secret-xyz"); + + // The manifest declares the placeheld names, so a later `capture` placeholds + // them again with no flag to remember. + const manifest = readFileSync( + join(f.out, "incubator", ".infra", "manifest.yaml"), + "utf8", + ); + expect(manifest).toContain("generated_secrets:"); + expect(manifest).toContain("- DATABASE_URL"); + expect(manifest).toContain("- REDIS_URL"); + // The plan says so out loud, in names and provenance — never values. + expect(r.output).toContain("generated"); + expect(r.output).toContain(GENERATED_PLACEHOLDER); + expect(r.output).toContain("captured"); + }); + + it("UNCAPTURED.md lists every known-inexpressible setting it SAW", async () => { + const f = fixture((await stubCoolify()).url); + const r = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--recipient", + RECIPIENT, + ]); + expect(r.code).toBe(0); + const md = readFileSync(join(f.out, "UNCAPTURED.md"), "utf8"); + + // Seen on the box, and inexpressible: + expect(md).toContain("Incubator Umami"); // service hostnames (#21 / 4.1.2) + expect(md).toContain("domains (hostnames)"); + expect(md).toContain("destination"); // which Docker network (#21) + expect(md).toContain("destination_id 3"); + expect(md).toContain("legacy-analytics"); // a MySQL cast cannot model + expect(md).toContain("custom_labels"); // Basic Auth / Traefik labels + expect(md).toContain("backup"); // not exposed by the API + expect(md).toContain("legacy.flag"); // not a name a template can hold + + // And the standing sections, emitted on every run whatever was found: + expect(md).toContain("no API coverage in Coolify 4.1.2"); + expect(md).toContain("Include Source Commit in Build"); + expect(md).toContain("What a rebuild from this draft still cannot restore"); + expect(md).toContain("the GitHub App private key"); + expect(md).toContain("S3 access keys"); + + // The run points at it rather than leaving it to be found. + expect(r.output).toContain("UNCAPTURED.md"); + }); + + it("writes the projects: registry — the list of what exists", async () => { + const f = fixture((await stubCoolify()).url); + await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--recipient", + RECIPIENT, + ]); + const yaml = readFileSync(join(f.out, "environments.yaml"), "utf8"); + expect(yaml).toContain("projects:"); + // Read off the application's git remote — the only place a box knows its repo. + expect(yaml).toContain("third-party/la-familia:"); + expect(yaml).toContain("environments:\n - prod"); + // The bindings the sweep could actually read. + expect(yaml).toContain("server: box-b"); + expect(yaml).toContain("name: Root Team"); + // Which GitHub App clones a repo is not on any resource — but this instance + // has exactly one, and there is no other it could be. + expect(yaml).toContain("github_apps:"); + expect(yaml).toContain("third-party/la-familia: hdb-coolify"); + + // The barber shop has no application, so the box knows no repo for it. cast + // writes the bare project name — and the registry's own parse-time refusal + // (#25: "a registry key has no meaning without its org") then stops the file + // being used until a human supplies the org. That refusal IS the design: the + // alternatives are inventing an org, or dropping a project from the list — + // and a project missing from the registry is one every fleet run skips in + // silence. + expect(yaml).toContain("martin-reyes-barber-shop:"); + const path = join(f.out, "environments.yaml"); + expect(() => loadBindings(path)).toThrow(/has no meaning without its org/); + }); + + it("refuses --emit-draft together with a repo — that is the reconcile path", async () => { + const f = fixture((await stubCoolify()).url); + const r = await run([ + "heavy-duty/incubator", + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + ]); + expect(r.code).toBe(2); + expect(r.output).toContain("SWEEP-mode flag"); + expect(r.output).toContain("Adoption is one-way"); + expect(existsSync(f.out)).toBe(false); + }); + + it("refuses a target directory that already holds a repo", async () => { + const f = fixture((await stubCoolify()).url); + mkdirSync(join(f.out, ".infra"), { recursive: true }); + writeFileSync( + join(f.out, ".infra", "manifest.yaml"), + "project: incubator\nenvironments: {}\n", + ); + const r = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--recipient", + RECIPIENT, + ]); + expect(r.code).toBe(2); + expect(r.output).toContain("is not empty"); + // For a declared project the manifest is the truth — a draft must not be able + // to overwrite a reviewed spec with a live box's accumulated cruft. + expect(r.output).toContain("Adoption is one-way"); + expect(readFileSync(join(f.out, ".infra", "manifest.yaml"), "utf8")).toBe( + "project: incubator\nenvironments: {}\n", + ); + }); + + it("refuses to emit secrets nobody can decrypt — and takes an explicit opt-out", async () => { + const f = fixture((await stubCoolify()).url); + const refused = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + ]); + // No --recipient, no age_recipient binding. Silently skipping the store would + // emit a draft that LOOKS complete and holds not one value. + expect(refused.code).toBe(2); + expect(refused.output).toContain("no age recipient"); + expect(refused.output).toContain("--no-secrets"); + expect(existsSync(f.out)).toBe(false); + + const optOut = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--no-secrets", + ]); + expect(optOut.code).toBe(0); + expect(existsSync(join(f.out, "secrets"))).toBe(false); + expect( + existsSync(join(f.out, "incubator", ".infra", "manifest.yaml")), + ).toBe(true); + expect(optOut.output).toContain("NOT WRITTEN"); + }); + + it("takes the recipient from the environment's binding when no flag is given", async () => { + const f = fixture((await stubCoolify()).url, { recipient: RECIPIENT }); + const r = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + ]); + expect(r.code).toBe(0); + const store = decryptSecrets( + join(f.out, "secrets", "incubator.prod.env.age"), + KEY_FILE, + ); + expect(store.MAILGUN_KEY).toBe(MAILGUN); + }); + + it("refuses to pick between two environments that both have resources", async () => { + const f = fixture((await stubCoolify({ ambiguous: true })).url); + const r = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--recipient", + RECIPIENT, + ]); + // Picking would emit a blueprint of half a box that says nothing about the + // other half — and this box is where that already happened once (#22). + expect(r.code).toBe(2); + expect(r.output).toContain("MORE THAN ONE environment"); + expect(r.output).toContain("Incubator"); + expect(r.output).toContain("--environment"); + expect(existsSync(f.out)).toBe(false); + + // …and --environment breaks the tie. For the projects that HAVE no tie (the + // client sites, each alone in its own `production`), it changes nothing: they + // are drafted either way. + const tied = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--recipient", + RECIPIENT, + "--environment", + "staging", + ]); + expect(tied.code).toBe(0); + const files = Object.keys(tree(f.out)); + expect(files).toContain("incubator/.infra/manifest.yaml"); + expect(files).toContain("la-familia/.infra/manifest.yaml"); + // The environment it did NOT draft is named, not dropped in silence. + expect(readFileSync(join(f.out, "UNCAPTURED.md"), "utf8")).toContain( + "other environments", + ); + }); + + it("still sweeps, and still asserts the team, before it writes anything", async () => { + const f = fixture((await stubCoolify()).url); + writeFileSync( + join(f.state, "environments.yaml"), + [ + "environments:", + " prod:", + " server: box-b", + " team: { id: 9, name: Some Other Team }", + "", + ].join("\n"), + ); + const r = await run([ + "--env", + "prod", + "--state", + f.state, + "--emit-draft", + f.out, + "--recipient", + RECIPIENT, + ]); + // A wrong-team token reads back an empty instance — and would draft a + // blueprint of nothing at all, confidently. + expect(r.code).not.toBe(0); + expect(existsSync(f.out)).toBe(false); + }); +}); diff --git a/test/draft.test.ts b/test/draft.test.ts new file mode 100644 index 0000000..b6c4955 --- /dev/null +++ b/test/draft.test.ts @@ -0,0 +1,317 @@ +import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { describe, expect, it } from "vitest"; +import { GENERATED_PLACEHOLDER } from "../src/capture.js"; +import { + type DraftProject, + assertEmptyTarget, + assertNoExistingManifest, + draftResourcesFrom, + isProviderGenerated, + planDraft, + repoFromGitUrl, +} from "../src/draft.js"; +import { templateKeys, templateRefs } from "../src/envtemplate.js"; +import { loadManifest } from "../src/manifest.js"; + +const ctx = { + env: "prod", + instance: "box-b", + baseUrl: "https://coolify.example.com", + team: { id: 0, name: "Root Team" }, + server: "box-b", + recipient: "age1example", + generatedAt: "2026-07-13T00:00:00.000Z", +}; + +// The value that must never leave the box it was read from. +const POISON = "postgres://postgres:pw@incubator-db-v2.box-b.internal:5432/app"; + +const project = (over: Partial = {}): DraftProject => ({ + name: "Incubator", + coolifyEnv: "staging", + resources: [ + { + kind: "application", + name: "Incubator Stack v2", + uuid: "a1", + raw: { + git_repository: "https://github.com/heavy-duty/incubator", + git_branch: "main", + build_pack: "nixpacks", + base_directory: "/", + ports_exposes: "3000", + fqdn: "https://app.example.com", + destination_id: 3, + }, + env: { + DATABASE_URL: POISON, + MAILGUN_KEY: "key-abc123", + NODE_ENV: "production", + }, + }, + ], + unreadable: [], + otherEnvironments: [], + ...over, +}); + +describe("isProviderGenerated — the one judgment that must not be wrong", () => { + it("recognizes the datastore families whose value points at the SOURCE box", () => { + for (const key of [ + "DATABASE_URL", + "DATABASE_URL_PROD", + "UMAMI_DATABASE_URL", + "REDIS_URL", + "POSTGRES_PASSWORD", + "DB_HOST", + "MONGO_URI", + ]) { + expect(isProviderGenerated(key), key).toBe(true); + } + }); + + it("recognizes Coolify's own per-instance magic vars", () => { + for (const key of [ + "SERVICE_FQDN_UMAMI", + "SERVICE_URL_UMAMI", + "SERVICE_PASSWORD_POSTGRES", + "SERVICE_USER_UMAMI", + "SERVICE_BASE64_KEY", + ]) { + expect(isProviderGenerated(key), key).toBe(true); + } + }); + + it("leaves an ordinary secret alone — it is captured, not placeheld", () => { + for (const key of [ + "MAILGUN_KEY", + "OPENROUTER_KEY", + "ADMIN_EMAIL", + "NODE_ENV", + "SERVICE_NAME", + "PORT", + ]) { + expect(isProviderGenerated(key), key).toBe(false); + } + }); +}); + +describe("repoFromGitUrl — the only place a box knows which repo it is", () => { + it("reads a slug out of every remote shape Coolify stores", () => { + expect(repoFromGitUrl("https://github.com/heavy-duty/incubator")).toBe( + "heavy-duty/incubator", + ); + expect(repoFromGitUrl("https://github.com/heavy-duty/incubator.git")).toBe( + "heavy-duty/incubator", + ); + expect(repoFromGitUrl("git@github.com:heavy-duty/incubator.git")).toBe( + "heavy-duty/incubator", + ); + expect(repoFromGitUrl("heavy-duty/incubator")).toBe("heavy-duty/incubator"); + }); + + it("answers undefined rather than guessing", () => { + expect(repoFromGitUrl("")).toBeUndefined(); + expect(repoFromGitUrl(undefined)).toBeUndefined(); + expect(repoFromGitUrl("not-a-remote")).toBeUndefined(); + }); +}); + +describe("draftResourcesFrom — including what cast cannot model", () => { + it("names a MySQL rather than silently omitting it", () => { + const { resources, unreadable } = draftResourcesFrom({ + applications: [{ name: "web", uuid: "a1" }], + postgresqls: [{ name: "db", uuid: "d1" }], + redis: [{ name: "cache", uuid: "d2" }], + services: [{ name: "umami", uuid: "s1" }], + mysqls: [{ name: "legacy-mysql", uuid: "m1" }], + mongodbs: [{ name: "old-mongo", uuid: "m2" }], + }); + expect(resources.map((r) => [r.kind, r.name])).toEqual([ + ["application", "web"], + ["database", "db"], + ["database", "cache"], + ["service", "umami"], + ]); + expect(unreadable).toEqual([ + { kind: "mysql", name: "legacy-mysql" }, + { kind: "mongodb", name: "old-mongo" }, + ]); + }); +}); + +describe("planDraft — the emitted shape", () => { + it("writes a manifest cast itself can load, keyed by --env", () => { + const plan = planDraft([project()], ctx); + const manifest = plan.files.find((f) => f.path.endsWith("manifest.yaml")); + expect(manifest?.path).toBe("incubator/.infra/manifest.yaml"); + // A file that says what it is. It leaves this process and is read by someone + // deciding whether to trust it. + expect(manifest?.content).toContain("PROPOSAL"); + expect(manifest?.content).toContain("`apply` does not read this file"); + expect(manifest?.content).toContain("box-b"); + + const dir = mkdtempSync(join(tmpdir(), "cast-draft-")); + const path = join(dir, "manifest.yaml"); + writeFileSync(path, manifest?.content ?? ""); + const loaded = loadManifest(path); + expect(loaded.project).toBe("Incubator"); + const env = loaded.environments.prod; + // The resource keeps the BOX's name — renaming it here would make the file + // unusable against the UI it was read from. + expect(Object.keys(env.applications)).toEqual(["Incubator Stack v2"]); + expect(env.applications["Incubator Stack v2"].source).toEqual({ + repo: "heavy-duty/incubator", + branch: "main", + }); + // And the manifest DECLARES the placeheld name, so a later `capture` does the + // same placeholding with no flag to remember. + expect(env.generated_secrets).toEqual(["DATABASE_URL"]); + }); + + it("emits an env template every cast reader can parse", () => { + const plan = planDraft([project()], ctx); + const tpl = plan.files.find((f) => f.path.endsWith(".env.template")); + expect(tpl?.path).toBe( + "incubator/.infra/env/incubator-stack-v2.prod.env.template", + ); + const body = tpl?.content ?? ""; + expect(templateKeys(body).sort()).toEqual([ + "DATABASE_URL", + "MAILGUN_KEY", + "NODE_ENV", + ]); + // EVERY var is a ${REF}: values live in the store, never as a literal in a + // file that is about to be committed to a product repo. + expect( + templateRefs(body) + .map((r) => r.ref) + .sort(), + ).toEqual(["DATABASE_URL", "MAILGUN_KEY", "NODE_ENV"]); + expect(body).not.toContain(POISON); + expect(body).not.toContain("key-abc123"); + }); + + it("PLACEHOLDS a provider-generated value and captures an ordinary one", () => { + const plan = planDraft([project()], ctx); + const byRef = Object.fromEntries(plan.dispositions.map((d) => [d.ref, d])); + expect(byRef.DATABASE_URL.provenance).toBe("generated"); + expect(byRef.DATABASE_URL.value).toBe(GENERATED_PLACEHOLDER); + expect(byRef.MAILGUN_KEY.provenance).toBe("captured"); + expect(byRef.MAILGUN_KEY.value).toBe("key-abc123"); + + // The store carries the placeholder, NOT the source box's Postgres. + const store = plan.stores[0]; + expect(store.path).toBe("secrets/incubator.prod.env.age"); + expect(store.vars.DATABASE_URL).toBe(GENERATED_PLACEHOLDER); + expect(JSON.stringify(plan.files)).not.toContain(POISON); + }); + + it("splits one name carrying two values rather than picking", () => { + const p = project(); + p.resources.push({ + kind: "application", + name: "Landing", + uuid: "a2", + raw: { + git_repository: "https://github.com/heavy-duty/incubator", + git_branch: "main", + build_pack: "static", + base_directory: "/", + fqdn: "https://www.example.com", + }, + env: { MAILGUN_KEY: "key-DIFFERENT" }, + }); + const plan = planDraft([p], ctx); + const refs = plan.dispositions.map((d) => d.ref); + // One store holds one value per name (capture refuses a CONFLICT for exactly + // this reason). cast will not pick, so both survive under distinct names. + expect(refs).toContain("INCUBATOR_STACK_V2_MAILGUN_KEY"); + expect(refs).toContain("LANDING_MAILGUN_KEY"); + expect(refs).not.toContain("MAILGUN_KEY"); + expect( + plan.uncaptured.some((u) => u.detail.includes("DIFFERENT values")), + ).toBe(true); + }); + + it("always emits UNCAPTURED.md — even with little to say", () => { + const bare = project({ + resources: [ + { + kind: "application", + name: "web", + uuid: "a1", + raw: { + git_repository: "git@github.com:acme/web.git", + git_branch: "main", + build_pack: "nixpacks", + base_directory: "/", + fqdn: "https://web.example.com", + }, + env: {}, + }, + ], + }); + const md = planDraft([bare], ctx).files.find( + (f) => f.path === "UNCAPTURED.md", + ); + expect(md).toBeDefined(); + // The standing sections are unconditional: what cast CANNOT SEE does not + // depend on what it happened to find. + expect(md?.content).toContain("no API coverage in Coolify 4.1.2"); + expect(md?.content).toContain("Include Source Commit in Build"); + expect(md?.content).toContain("the GitHub App private key"); + expect(md?.content).toContain("S3 access keys"); + }); + + it("registers what it drafted, and only what it drafted", () => { + const empty = project({ + name: "Empty Project", + resources: [], + skipReason: "every environment on it is empty", + }); + const bindings = planDraft([project(), empty], ctx).files.find( + (f) => f.path === "environments.yaml", + ); + expect(bindings?.content).toContain("heavy-duty/incubator"); + expect(bindings?.content).toContain("environments:\n - prod"); + // Not registered — a registry entry for a project with no manifest sends + // every future fleet run at nothing. + expect(bindings?.content).not.toContain("Empty Project"); + // …but it is not lost either. + const md = planDraft([project(), empty], ctx).files.find( + (f) => f.path === "UNCAPTURED.md", + ); + expect(md?.content).toContain("Empty Project"); + }); +}); + +describe("the emit refusals — adoption is one-way", () => { + it("refuses a target directory that is not empty", () => { + const dir = mkdtempSync(join(tmpdir(), "cast-draft-")); + writeFileSync(join(dir, "README.md"), "a repo lives here\n"); + expect(() => assertEmptyTarget(dir)).toThrow(/is not empty/); + expect(() => assertEmptyTarget(dir)).toThrow(/Adoption is one-way/); + }); + + it("allows a directory that does not exist yet, and an empty one", () => { + const dir = mkdtempSync(join(tmpdir(), "cast-draft-")); + expect(() => assertEmptyTarget(dir)).not.toThrow(); + expect(() => assertEmptyTarget(join(dir, "new"))).not.toThrow(); + }); + + it("refuses to write a manifest over one that already exists", () => { + const dir = mkdtempSync(join(tmpdir(), "cast-draft-")); + mkdirSync(join(dir, ".infra"), { recursive: true }); + const path = join(dir, ".infra", "manifest.yaml"); + writeFileSync(path, "project: incubator\n"); + // For a declared project the manifest IS the truth: regenerating it from a + // live box would let that box's cruft overwrite a reviewed spec. + expect(() => assertNoExistingManifest(path)).toThrow(/already exists/); + expect(() => assertNoExistingManifest(path)).toThrow( + /the manifest IS the truth/, + ); + }); +});